That all changed in the last year or so, as he hit national camps to get on plenty of major programs' radar. Alabama, Notre Dame, Ohio State, Louisville and Kentucky were among the many that came with scholarship offers, and now the four-star defensive tackle is down to the in-state Wildcats and Crimson Tide.
Elam's journey was originally set to end January 4, while participating in the Army All-American game, but he pushed the decision back over that weekend.
"Letting everyone know right now I, Matt Elam, will be deciding my college decision on Signing Day (February 5)," he posted on Twitter. "Sorry for the inconvenience #Bama #BBN."
The 6-foot-6, 340-pounder felt like making a commitment at John Hardin High School was the better idea.
"I just didn't want to to it there," he said. "I don't know, I felt weird. I didn't like how they did it there, so I would rather do it at my school."
Elam also was unsure about taking one last official visit. He tripped to Lexington in December, but never took the official visit to Tuscaloosa, though he took in the Alabama-LSU game in November.
Plans have changed on the visit front, too.
"I will be taking one on the 17th. Alabama," he said. "Just to see everything for one last time, to give them a chance like I gave UK."
The process has been one the ever-smiling nose guard talent has taken in stride. But deciding between the finalists is exactly four weeks away.
"I can't describe it," he said laughing. "I don't really get stressed, just getting more excited."
Elam is considered the No. 12 defensive tackle prospect in the class of 2014 on Scout.com.