"Not may people want a 5-foot-11 guard who weighs 135 pounds," U of L coach Rick Pitino jokes about the player from Brooklyn, N.Y.
But Smith didn't worry about what others were talking about him when he arrived at U of L.
"We all told each other, 'Nobody thinks anything of us and nobody could care less about us," Smith recalled. "So we have to make this place something to remember."
Well, four years later he was spot on. Smith not only has made it a place to remember, but as he prepares for Saturday's Senior Day Smith is set to leave as one of the top players in school history - and one of the fan favorites.
The 6-foot Smith is an All-American and currently ranks second in school history in steals with 242 and eighth all-time in scoring at 1,776 career points.
"I just wanted the chance to perform," Smith said. "It's been a great time here. It's amazing where I have come from. I have a lot of my boys from high school coming to Senior Night.
"I actually apologized to them for the way I played in high school because I was a detriment to our team and the success back then. I feel like I could have things differently, so just being able to recognize the way I played, the way I was as a teammate and a person, I feel like I have made a lot of strides."
Smith is the top player among four seniors on the roster not only for his career statistics, but his antics with coach Pitino.
The Hall of Fame coach nicknamed him "Russdiculous" because of his erratic play. Pitino has said the nickname was a term of endearment that summed up Smith's ability to make both good and bad plays on the floor.
Pitino was asked on Friday what his favorite Russ Smith memory was and he responded: "That would be an impossibility, there are so many."
Smith, who nearly transferred after his freshman season, came back for his senior season to help bolster his stock for the NBA.
Pitino said Smith has "really improved" this season and notes that he has told NBA teams they need to find room for the quick guard.
As a senior, Smith is averaging 18.0 points but has raised his assists (4.5), field goal percentage (46.8 percent) and assist/turnover ratio (1.7). He's never hit more than 41 percent from the field during his career.
"I don't think he puts himself in a box as much as he used to," Pitino said. "He used to put himself into difficult situations where you shoot a low percentage."
Smith, who said he "has been underestimated my whole life," said he feels like he will get emotional on Saturday during Senior Day ceremonies at the KFC Yum! Center.
"I feel like I definitely will (get emotional)," he said. "I have been through a lot. Not a lot of people could have kept their head up through all of the things I have been through."
And as for Pitino, Smith said he "knows" his coach will get emotional saying goodbye.
"I always joke around about him," Pitino said. "But I think I will miss him more than any basketball player I have ever coached.
"I have had so many laughs with him, even in tense moments he has afforded me the ability to laugh. He's one-in-a-million, he's one of a kind and he's so unique in whatever he does. He has a great heart and in four years the young man has never gotten into a stitch of trouble. I have been really fortunate to have him for four years."