Vic Koenning, Clemson's defensive coordinator of the past three seasons, resigned his position last week and has since been named co-defensive coordinator at his alma mater, Kansas State.
Chavis originally met with Swinney last week in an interview that lasted almost three hours, according to sources.
Sources also tell CUTigers.com Chavis' wife has also been spotted on campus this afternoon as well.
Chavis, along with the rest of Tennessee's coaching staff, was dismissed after Vols' head coach Phil Fulmer was fired near the end of the regular season.
Despite Tennessee's 5-7 record this year, Chavis led the Tennessee defense to a No. 4 ranking in total defense as his unit gave up just over 263 yards per game.
Tennessee was also No. 5 in passing defense, No. 12 in scoring defense and No. 13 in rushing defense.
Among other awards captured by Chavis over the years include the 2006 American Football Coaches Association's Division I-A Assistant Coach of the Year.
Since 1995, Chavis' defenses have held their opponents below a 100-yard rushing average in six seasons, and nine times during that span the Vols have ranked among the top three in the SEC in total defense.
Chavis also has recruiting ties to the Palmetto State and is actually a native of Dillon, S.C.
Current Clemson assistants David Blackwell and Ron West are slated to control Clemson's defense in the upcoming Gator Bowl against Nebraska.
Swinney has already made three hires to his new coaching staff, including former Tiger tight end Danny Pearman, former Mississippi State defensive coordinator Charlie Harbison and former Mississippi State offensive coordinator Woody McCorvey.
Pearman will coach tight ends, Harbison will coach the secondary and McCorvey, who is currently an administrative coach, could be named running backs coach after the bowl game.
Numerous coaches with coordinator experience have been rumored to be interested in the position.
Current Clemson assistant coach Billy Napier is expected to retain the offensive coordinator position next year. Napier took over for Rob Spence, who was let go the same day Swinney was named head coach on Oct. 13.