But it wasn't until USC offered last weekend that the floodgates really opened for him.
Once the Trojans offered, seemingly schools came at him left and right with an offer.
"UCLA, Boise State, Nebraska, Texas A&M and Arizona all offered me since," said Brown. "I had no idea it was going to turn out like this. Especially from the big Pac-12 schools. Texas A&M just called (on Tuesday) to offer out of the blue. A month ago, I didn't even expect that."
The Aztecs got a jump on Brown, and he was perfectly content to play there, until things started accelerating for him.
"San Diego State was my first offer in the spring and I was excited about that," said Brown. "Then the other offers came in at the same time, when my film got out. Washington State was my first Pac-12 offer, then more just started coming in."
UCLA offered on Tuesday afternoon before Texas A&M offered him in the evening.
A couple of other schools have reached out to Brown too, though an offer hasn't materialized just yet.
The USC offer, though, was the tipping point for Brown.
With his head coach, Marvin Sanders, a former USC assistant, and several other Loyola assistants having served on the staff at USC, Sanders told Brown that USC wanted him to come by the school over the weekend.
"I went to practice on Sunday and Coach (Ed) Orgeron offered me right when I got there," said Brown. "When Coach Sanders got here to Loyola, he started seeing me on the field and obviously he had some connections to USC. He talked to them and they talked to me and thought I was a good player and they offered."
That offer left a big impression on Brown.
"That was the one that was kind of like ‘wow'," said Brown. "Before USC, I just had San Diego State, Washington State and Fresno State. Then USC offered and then I got Nebraska and Boise State on the same day. So that was the first big one."
Texas A&M's offer on Tuesday also left its mark.
"That one was big because it literally came out of nowhere," said Brown. "I know I want to take an official visit there."
A conversation with Adrian Klemm from UCLA on Tuesday also included an offer.
"Coach Klemm offered me on Tuesday night after we talked a while on the phone," said Brown. "I watch them too because of Anthony Barr. It's exciting to see Anthony play because I remember seeing him at my school."
Growing up in Southern California, Brown said he liked both Los Angeles teams but USC got more of his attention.
"I was pretty even because I have friends and family that are UCLA fans, friends and family that are USC fans but a neighbor who has season tickets to USC so I followed USC a little bit more," said Brown.
Brown said he will try to visit UCLA unofficially soon and will also get to the next USC home game.
As for his position, Brown said each school likes him on the offensive line, with tackle the likely spot.
"Some didn't really say because they don't know yet, but everybody is recruiting me as an offensive lineman," said Brown.
With things being shaken up, Brown said he'll likely be taking his time.
"I do think it's going to be kind of hard, but there is no easy decision to make," said Brown. "I definitely want to weight out all my options. I've gotten 6-7 offers in the last week and I'm anticipating more."
Brown attributes the rise in interest to the coaching he's gotten at Loyola, with former USC lineman and Trojan staffer Lenny Vandermade working with him.
"I don't really think I was overlooked in the spring" said Brown. "I just think the new staff came in and helped. Coach Vandermade improved my technique and I think it's helping. All the coaches are."
Brown said his family was thrilled with how things have quickly transpired.
"My family is definitely excited, especially about USC and UCLA," said Brown. "Part of it is because it's so close to home. But they're excited for every opportunity that comes up."
Loyola faces Encino (Calif.) Crespi this Friday.