Kelly completed a deflected 66-yard touchdown pass and added two more long ones in the fourth quarter, and the Rebels held on to beat No. 2 Alabama 43-37 on Saturday night.
The Rebels (3-0, 1-0 Southeastern Conference) left the topsy-turvy game with their first victory in Bryant-Denny Stadium since 1988, and second straight in a one-sided series. The Crimson Tide (2-1, 0-1) had been 25-1 in Tuscaloosa against Ole Miss and had never lost two straight in the series.
Both games went down to the wire and were settled with the help of Rebels interceptions.
"We certainly think we're at a point now where it's not a shock," Mississippi coach Hugh Freeze said. "I still say it would be an upset in most people's eyes to do it here.
"I don't think it surprises the people in our locker room. Our pregame was as short as it's ever been. I said, 'You're good enough to win.'"
The Rebels made two final defensive stands, the first when Tony Bridges intercepted a deep ball from Jake Coker, who came off the bench to lead a comeback from a 30-10 deficit. Ole Miss scored 24 points off five Alabama turnovers.
Kelly delivered big play after big play following an ineffective first half, none wilder than a fling and a prayer in the third quarter, a 66-yard deflection caught by Quincy Adeboyejo for a touchdown.
Kelly bobbled a high snap and lofted the ball toward a crowd as he was going down before a swarm of Tide defenders. Adeboyejo grabbed a deflected ball and raced toward the end zone in front of Alabama's student section.
"That happened in my prayer time today," Freeze said. "That's the only thing I can say about that."
Kelly said he figured the 6-foot-3 Adeboyejo could "jump the highest out of anybody, and I just gave him a chance."
"Right after that you think, 'Hey, we're winning this,'" he said. "And that's exactly how we all thought, that this was our game."
Kelly added a 73-yarder to Cody Core and a 24-yarder to Laquon Treadwell. Kelly passed for 341 yards and three touchdowns after producing just 36 yards through the air in the first half.
Alabama had won 17 straight at home coming into the game to tie Baylor for the longest active streak in FBS, another milestone snapped by a feisty Ole Miss team that's established itself as an SEC contender under Freeze.
"We thought last year when we played we gave some things away, and I really don't want to take anything away from Ole Miss," Tide coach Nick Saban said. "They played a really good game, but are you going to be a giver or are you going to be a taker? If you're going to be a great competitor, you've got to take what you want.
"I don't know if there's anybody we can beat if we're going to give away (24) points, I really don't."
Coker came up with big plays, and big mistakes. He led two touchdown drives in a span of 2:03 late in the fourth quarter, aided by an onside kick. Derrick Henry scored on a 2-yard run and Richard Mullaney caught a 2-yard touchdown pass. Coker also was picked off by Bridges downfield with 2:36 left.
Coker was 21-of-44 passing for 201 yards with three touchdowns and two interceptions. Henry ran for 127 yards.
The Tide started Cooper Bateman even though Coker opened the previous two games. Regarded as the more athletic option, Bateman was 11 of 14 for 87 yards with an interception.
Coker, meanwhile, ran seven times for 58 yards.
"We had some things in the game plan early that the quarterback was going to pull the ball on and do some things, so we thought Cooper's speed would be sort of a change of pace for them, a little bit of an element of surprise," Saban said.
Alabama got one more chance with 31 seconds left after the defense stuffed Jeremy Liggins on fourth-and-1 from the Tide 31. Replay officials said Jaylen Walton's elbow touched the ground just shy of a first down on the previous play.
Four straight incompletions, including a drop, ended the rally hopes for Alabama.
AP: Ole Miss 43, Alabama 37
OM Spirit Top Stories
Ole Miss Impresses 4-Star OL Saahdiq CharlesNational 300 member and LSU OL commit Saahdiq Charles returned from his trip to Ole Miss with a lot on his mind. Sunday's news that the Tigers had fired Les Miles made things even…
OM Spirit1:13 PM
Who's Todd Harris Feel He Could Play For?Todd Harris is praying to God to help him with his recruitment. Harris knows at least two of his official visits and he gives the very latest here...
Scout Football11:00 AM
Q&A: Baseball With BiancoThe fall baseball season for the Ole Miss Rebels moves into October later this week. Here are some comments from Ole Miss head coach Mike Bianco as the Rebels continue fall ball.
OM SpiritYesterday at 10:00 PM
RYH: Chris Kiffin, Chris MalloyIn this edition of the Rebel Yell Hotline, Chuck Rounsaville and Yancy Porter are joined by Ole Miss defensive line coach Chris Kiffin and Rebel golf coach Chris Malloy.
OM SpiritYesterday at 9:10 PM