Leggett led Clemson to 955 victories, six College World Series trips in 22 seasons and a legacy as one of college baseball's top coaches.
However, the Tigers have not reached Omaha, Nebraska, since 2010 and have gone 5-10 in their past five NCAA tournament trips, including getting swept out of regionals in 2014 and 2015.
The Tigers were 32-29 this past season, losing both games at the Fullerton Regional last weekend. A year ago, Clemson lost two straight at the Nashville Regional.
Clemson athletic director Dan Radakovich met with Leggett earlier this week to assess the team's progress and future.
"After my evaluation, it came down to this: I think we can be better," said Radakovich, who informed Leggett of the dismissal earlier Thursday.
Leggett, 61, did not immediately messages from The Associated Press.
Clemson had been one of the best in college baseball for much of Leggett's tenure.
Leggett took over for the late Bill Wilhelm in 1994 and continued his predecessor's success — Wilhelm led Clemson to six College World Series trips between 1958-93 — with the Tigers.
Leggett's team, led by future Major League Baseball pitchers Kris Benson and Billy Koch, reached the College World Series in 1995 and 1996. His teams also went to Omaha in 2000, 2002, 2006 and 2010.
But since then, the Tigers have had a drastic postseason drop-off. They had not hosted an NCAA regional (Clemson held 11 in Leggett's first 18 seasons) since 2011.
Perhaps worst of all for Clemson fans was the rise of rival South Carolina, which won national titles in 2010 and 2011 and until this season had lost each season series with the Gamecocks since 2006.
Leggett's win totals slid from 40 in 2013 to 36 last year and 32 this season. His program came under scrutiny from Radakovich last season, the AD opting to retain the coach while putting him on notice there had to be significant improvement.
Leggett has one year left on his contract and is owed a buyout of $200,000.
Radakovich said the search for Leggett's successor had not started.
The Tigers' ace, right-hander Matthew Crownover, posted on Twitter that he came to Clemson because of Leggett.
"He will always be my coach. He deserved better," he said.
Crownover, a junior who went 10-3 with a 1.82 ERA this season, is among Baseball America's top 350 prospects for this month's Major League Baseball draft.
Assistant head coach Bradley LeCroy will take over until a new head coach is found. Radakovich said LeCroy expressed interest in the job and was encouraged to apply.
The next coach will be just the third for the baseball team since 1958 — a span of 57 years.
Leggett leaves a lasting mark on the game with former assistants achieving at the top levels of the sport. One-time assistant Tim Corbin led Vanderbilt to the national title last year and Florida coach Kevin O'Sullivan has built a successful program with the Gators since his time as a Clemson assistant.
Leggett, selected to the American Baseball Coaches Association Hall of Fame in 2014, was a driving force behind the flat-seam baseball, which has so far increased offense in the college game in its debut season.
Radakovich was confident that with Clemson's facilities, its strong fan following and winning history, he would have little trouble attracting an energetic leader able to get the Tigers back to the top.
"We need to inject some new momentum and create optimism around our baseball program," Radakovich said. "And I'm confident in our search for a new head coach that we'll find someone who'll bring that excitement to our fans and our student-athletes."
Clemson parts ways with Leggett
CUTigers.com Top Stories
RECRUITING RADARIn our weekly Recruiting Radar, CUTigers Recruiting Analyst Gary McDaniel updates you on the latest news and intel on the Clemson recruiting front!
TIGERS REMAIN #3 AND #5 IN POLLS#3 Clemson is 4-0 (1-0 ACC) headed into a showdown with #4 Louisville in Death Valley.
CUTigers.comYesterday at 9:47 AM
ATLANTA CURSE BROKEN#3 Clemson dominated Georgia Tech from beginning to end on Thursday night and ended a 5-game losing streak to the Yellow Jackets in Atlanta.
CUTigers.comFriday at 8:24 AM
WHAT WE ARE HEARING#3 Clemson is coming off a dominant 59-0 win over SC State and opens ACC play against interdivisional rival Georgia Tech in Atlanta on Thursday night.
CUTigers.comThursday at 5:21 AM