Now, the comedic Baltimore Ravens owner is entering his final season as majority owner as minority owner Steve Bisciotti plans to exercise an option to purchase the remaining 51 percent of the club for $325 million for a total purchase price of $600 million.
When Modell was 35 and living with his mother in New York, he bought the original Cleveland Browns for $4.295 million on March 21, 1961.
The hefty markup was engineered partly by inflation and the increase in franchises' value through television contracts and revenue sharing that Modell was responsible for along with late NFL commissioner Pete Rozelle.
It's been a long, successful career for Modell, albeit a journey that includes the controversial move to Baltimore that has cast him as a figure of immense unpopularity in Cleveland.
However, Modell is also widely recognized in league circles for being unselfish and for championing diversity in the front office.
"A great contributor to this league, Art is a man who has always put the NFL and his team first," Ravens general manager Ozzie Newsome said. "He has a lot of passion for football. He is a very loyal man and he rewards loyalty."
And Modell is alternately buoyant and melancholy as he contemplates the reality that this will be his last season as majority owner.
He boldly predicts this will be a wildly successful campaign with a team he considers superior in talent to the one that defeated the New York Giants in Super Bowl XXXV.
"The emotions run the gamut," Modell said. "The highs outnumber the lows. I have no misgivings. It was a great run."
Impact on the game
From modest beginnings in Brooklyn, N.Y., as an electrician's helper who cleaned hulls of ships in a shipyard, Modell kept rising along with the NFL until he held the Vince Lombardi trophy above his head in Tampa, Fla., two years ago.
Modell was 15 when he dropped out of high school to help support his family after his father died. A few years later, he parlayed enlistment in the Air Force and the G.I. bill to enroll in television school after World War II.
Using his savings from time spent producing television shows and working in advertising, Modell bought the Browns for a then-record price.
Along the way, Modell, 78, played a pivotal role in negotiating the television contracts that generated even more popularity for the league.
"Art always took the lead in those negotiations and put the league first," said New York Giants owner Wellington Mara, one of Modell's closest friends. "He should be remembered as someone who made a tremendous contribution."
Modell was the first chairman of NFL Films and instituted doubleheaders. He also broke the impasse for realignment by shifting the Browns into the American Football Conference.
"He's a historical figure in the league, such a leader, gave his life to this sport," said Giants general manager Ernie Accorsi, who worked for Modell in Cleveland. "Such an entertaining individual, too. No matter how serious or critical a situation, he's always a few seconds away from a funny line in any kind of crisis to break the tension. "It just saddens me to think he's going out. I'm sure it saddens him. He left such an amazing legacy."
Modell was also the lone elected president in NFL history, chairing discussions for the initial collective bargaining agreement on a committee that included Green Bay Packers coach Vince Lombardi. He went before Congress with Rozelle to lobby for an antitrust exemption.
Yet, the television contracts may be his lasting hallmark, helping transform a game once played on sandlots into an international event. The revenue sharing from those deals allows small-market cities like Green Bay to compete with a vast metropolis like New York.
"Art changed the game, really revolutionized it," said Ravens kicker Matt Stover, entering his 14th year playing for Modell. "He and Pete Rozelle had the foresight to see what the NFL could become. They spearheaded the expansion of the league. Art helped make a very popular game marketable and profitable without sacrificing his integrity."
During his early years as the Browns owner, Modell made the difficult decision to fire Paul Brown shortly after Brown traded running back Bobby Mitchell to the Redskins. Two years later, the Browns won an NFL title under Blanton Collier.
Modell also had the unenviable task of informing bruising running back Jim Brown that he needed to choose between a budding acting career and football. Brown opted to remain in Hollywood and retired in his prime.
Also, Modell stood by the late Ernie Davis and several other former players during their final days.
"Art Modell is a legend of our game," NFL commissioner Paul Tagliabue said, "and one of life's unforgettable characters for all of us who have been fortunate to know him."
Modell also experienced The Drive, The Fumble and The Interception.
Between John Elway's two-minute drill, Earnest Byner's costly fumble and Brian Sipe's errant pass, Modell had some rough moments.
Yet, he's still here with 27 winning seasons under his belt and an all-time record of 336-298-8.
"I'm really happy that Art won a Super Bowl," Pittsburgh Steelers owner Dan Rooney said. "He truly deserved it. That was so rewarding for me to see even though we're competitors. Art will be missed."
A painful split
When Modell couldn't reach an accord with politicians regarding an untenable stadium situation, he made the difficult choice in 1995 to move the Browns to Baltimore.
He left behind the Browns' trademark, colors and history, still creating a maelstrom of ill will that continues to this day even though Cleveland has since received an expansion franchise and built a pristine stadium with the late billionaire owner Al Lerner, another close friend of Modell's.
Publicly vilified, Modell hasn't been back to Cleveland ever since.
"People there still carry a terrible scar and that's understandable," Modell said. "The politicians drove me out of town. The business community took care of the Indians and the Cavaliers, but not the Browns. Those memories are unpleasant. I will go back when I feel that I'm welcome."
That may never happen. The bitter feelings in Cleveland have yet to subside. Everything Modell built in Cleveland became instantly forgotten by all but his closest colleagues.
"In Cleveland, unfortunately his legacy will be someone who took something away that they loved so much," Mara said. "The trouble is they directed their wrath at the victim rather than the spoiler who drove him out of town."
The move to Maryland actually accelerated his sale of the team as he sold 49 percent of the club to Bisciotti for $275 million to help recoup debt acquired during the relocation.
Modell had to borrow money that first year to pay receiver Andre Rison's $5 million bonus. The Ravens couldn't even afford a practice squad during their first season in Baltimore.
"It's been a blessing for Cleveland in some ways," Stover said. "Art Modell couldn't have existed with the money he was making there as the league changed. Now, they have a cash cow and Art has rolled with the punches."
Cloudy Hall of Fame outlook
Modell was effectively blindsided during an unsuccessful bid for the Hall of Fame in January, 2002.
Modell's candidacy was presented by Cleveland Plain Dealer beat writer Tony Grossi. That's the custom for the Hall as a voter from the area where a finalist spent the majority of his career is supposed to present the individual's merits for induction.
Instead, Grossi went on a long harangue downplaying Modell's career, emphasizing the move throughout his talk. Modell wasn't inducted.
"No question, the move from Cleveland hurt my possibilities," Modell said. "My record speaks for itself, but obviously the move hurt some people and has hurt me."
Modell wasn't a finalist last year. He said the only way he'll return to Ohio is if he's being enshrined in Canton.
"Art has accomplished a tremendous amount for the league," Tagliabue said. "I believe in the near future his entire record should be recognized as entitling him to be in the Hall of Fame in Canton."
Newsome is optimistic that Modell will be inducted at some point. He doesn't believe the path is permanently blocked.
"I think eventually Art will get in," said Newsome, a Hall of Fame tight end for the Browns. "He belongs in there. As time moves on and Cleveland gets their own identity with that team that will allow them not to care so much about Art Modell."
Fairness and generosity
The Ravens organization is populated heavily by employees who have been with Modell dating back to Cleveland, including Newsome and Modell's sons, David Modell, the team president, and John Modell, a vice president for special projects.
Modell has always believed in promoting from within and standing by people when they need him most whether that involves their health, a funeral, family issues or legal problems.
"Art has been a father figure to me, a man's man," said Byner, the Ravens director of player development. "I will forever be grateful and love him for that."
Modell has also been active in philanthropy, contributing to several charities and serving on corporate boards.
Byner said Modell has also been someone whom players could share sensitive information with and have it remain confidential.
"A lot of water has passed under Art's bridge," Byner said. "Art has been through all types of obstacles and adversity in his life. He keeps things within the family."
Hoping for one more memory
Modell's downtown office is akin to a miniature Hall of Fame with framed photographs of gridiron luminaries from the past. Old pals, Modell calls them.
Lombardi is on this wall of fame, demonstrating his trademark wide-toothed smile.
An old friend he used to vacation with, George "Papa Bear" Halas, the late Chicago Bears owner, is there along with former Redskins owner George Preston Marshall.
There's also a prominent picture of Dan Reeves, the late Los Angeles Rams owner who introduced him to his wife, Pat, a former actress in soap operas, movies and Broadway shows.
"I'll get even with you, Dan," Modell quipped while shaking his fist.
Then, Modell turned serious, saying the wall of pictures sometimes makes him sad because it represents many dear friends who have passed away. Modell has had his share of health scares.
He had a heart attack in 1983. He went through a coronary bypass and spent three weeks in intensive care. Last year, Modell suffered a minor heart attack and a mild stroke that affected his vision and movement.
"I'm in pretty good shape now and the high commissioner has extended my option," Modell said. "I tell people the worst blow of my entire life is when Brian Billick took me off the kickoff coverage unit. I can't be the wedge-buster anymore."
Meanwhile, Modell holds a strong belief that this football team will send him out with a memorable season without an extended farewell tour.
"I don't want any fuss made about me or hoopla," Modell said. "I want us to have a great season and then I want to walk away quietly. I'm proud of my career."
Aaron Wilson writes for the Carroll County Times.