It was the second straight Saturday No. 1 went down as Duke lost to then-No. 20 North Carolina State 84-76.
It was the Orange's third straight win over the Cardinals.
Carter-Williams had 16 points, Jerami Grant and C.J. Fair both had 10 for the Orange (17-1, 5-0) who won their seventh straight.
Russ Smith's 25 points led Louisville (16-2, 4-1), which had its 11-game winning streak stopped.