South Carolina committed five turnovers, one more than Kentucky (1-5, 0-3 SEC). The Wildcats have lost four straight.
Rathe, a senior who had appeared in only two games this season and completed only one pass, entered with 6:29 left and the Gamecocks trailing 7-6. He was 5-of-7 for 57 yards on the 88-yard drive, and received two breaks on the incompletions, as Kentucky's Vincent Burns and Mike Williams each dropped potential interceptions.
On the touchdown pass - his first in two seasons at South Carolina - Rathe rolled to his right and lobbed the football to Williamson to the back corner of the end zone. Williamson leaped to make the catch and barely got both feet down inbounds.
Kentucky quarterback Shane Boyd threw four incompletions on the Wildcats' final possession.
Kentucky's 23-play touchdown drive consumed 11 minutes, 54 seconds and came after South Carolina punter Josh Brown pinned the Wildcats at their own 6. On the drive, Kentucky converted six times on third down and had only three plays of 10 yards or more.
Kentucky took over at the South Carolina 40 after an interception by freshman Marcus McClinton, but couldn't manage a first down and punted. South Carolina then drove for its only touchdown.
South Carolina emerged from an error-filled first half - the teams combined for six turnovers - with a 6-0 lead thanks to two second-quarter field goals by Brown.
Kentucky reached the South Carolina 31 late in the first quarter, but Boyd threw incomplete on third-and-1 and Begley was well short on a 49-yard field-goal attempt.
Brown's first field goal, a career-high 43-yarder, came one play after the Gamecocks - already without starting quarterback Dondrial Pinkins, who has missed three games with a shoulder injury - lost his replacement, Syvelle Newton, when Newton sprained his left ankle while being sacked.
Blake Mitchell relieved Newton and played until Rathe entered the game for the Gamecocks' touchdown drive.
Boyd lost two fumbles in the half; the second, at the Kentucky 35, led to a 20-yard field goal by Brown.