The lead changed hands three times in the final 39 seconds, beginning with Ezra Williams' 3-pointer that put Georgia ahead 80-79.
Matt Bonner drove into the lane to hit a basket that put the Gators back on top, 81-80, with 23 seconds left.
Georgia called a timeout and set up a play to isolate Hayes on the left side. Justin Hamilton fell to the court and Hayes knocked down the game-winning shot.
Hayes scored 21 points and Williams added 20 for the Bulldogs, who shot 51.5 percent (34-of-66) from the field.
Florida shot 50 percent, but it wasn't enough to sweep the season series against the Bulldogs. The Gators won the first meeting in Gainesville when Anthony Roberson hit a 3-pointer at the buzzer for a 69-66 triumph.
Bonner led Florida with 24 points, while Lee scored 18 after shooting 7-of-9 from the floor. Walsh added 13 points for the Gators.
In the first half, Lee scored 10 of Florida's first 12 points. The Gators were up by as much as 11 points in the first half, but the Bulldogs cut the lead to 42-37 to go into the locker room at the half.
The Gators led much of the game. Georgia finally took its first lead, 60-58, when Rashad Wright hit a driving layup with 9:07 remaining.
Florida will return home to take on second ranked Kentucky Saturday, March 8 at 2 p.m. The game will be broadcast live on CBS.