That's when Phillips started to make a big impact on the Kentucky offense. From 2003-2009, Phillips spent time coaching multiple offensive playmaker positions, eventually taking over as the offensive coordinator from 2005-2009. Head coach Rich Brooks retired after the 2009 season, pushing Phillips into the head coach role of his alma mater.
In three seasons, Kentucky went 13-24 with Phillips as the head coach before he was fired after last season. He wasn't jobless for long, taking the receivers coach position at Florida.
He has earned the respect of the offensive players in Gainesville.
"He's a real strict coach, and he's a stickler for discipline and respect," Florida center Jonotthan Harrison said. "At first when he came in a lot of the receivers were complaining, 'Oh man, he's mean.' Then we just slowly saw them actually appreciating all the extra that he was bringing to the table.
"They really respect him as a coach, really respect the amount of discipline he requires from the receivers, as we're seeing the results every day."
The differences are noticeable. Through three games last season, the Florida wide receivers had 15 catches for 366 yards. Through three games this year, the Gators have 38 catches for 560 yards from the receiver position.
"A huge impact," Florida wide receiver Solomon Patton said about Joker Phillips. "We're learning how to run our routes the right way, at the right depth. Technique-wise and catching the ball, he has made a real huge impact."
Emotions will be high for Phillips this weekend as he returns to the school where he started his coaching career and played college football. During his time at receiver for the Wildcats, Phillips finished his career with 75 catches for 935 yards and nine touchdowns. He was fifth on the school's all-time receiving list when he graduated.
He left his mark on the program while on the field, and while his coaching career didn't work out as he and the Kentucky fan base hoped, it'll be interesting to see the reception he gets on Saturday. Florida offensive coordinator Brent Pease, who served as Kentucky's offensive coordinator from 2001-02, understands the emotion for Phillips this week.
"I think he's got respect for the program because he's an alumnus," Pease said. "I think he understands the kids and where they're at, the ones that he helped recruit. I'm sure we're all a little bit anxious, if we were in that role to do that, but he's not a person that's mad about the situation. I think he enjoys being here. He's excited to go back and coach there and I don't think it's any type of situation to be vengeful or anything like that."
Even though he might not be coaching for revenge on Saturday, the play of the Florida receivers does mean a lot to Phillips this weekend. The players see the difference in practice.
"I can tell he's a little bit more on edge, as far as he wants us to make sure we do everything right," Patton said. "That's understandable coming that he was there last year. He wants to perform at the top level with us going down there to Kentucky."
There is an added bonus to having Phillips on the Florida staff this week. He recruited a large majority of the Kentucky roster and brought them to Lexington. He worked with them every day of practice through the end of last season, helping him get to know the strengths and weaknesses of Kentucky players on both sides of the ball.
Some Florida players said throughout the week that they reached out to Phillips to ask questions about the players they'd be matching up with this week. The Wildcats are running different schemes on both sides of the ball than they did under Phillips, but his opinions can still help with personnel.
"Anytime you spend a lot of time at a place, you have some great memories," Florida head coach Will Muschamp said about Phillips. "I know as a player and coach there, no question it's going to be emotional about that. Anytime you spend as much time as we spend, hours in the office, working hard for the university, there's no question that's part of it."