Jermaine Marshall added 11 points for Penn State (4-1), which had been blown out by Kentucky, 85-47, in the opening game of the tournament Saturday.
Blake Nash had 14 points and Toarlyn Fitzpatick 11 for South Florida (3-2), which lost forward Augustus Gilchrist to a shoulder injury in the first half.
South Florida led by six points in the second half before Penn State used a 12-1 run, keyed by back-to-back layups from Frazier, and hung on.
Marshall's 3-pointer tied it at 43 with just under five minutes to play and Frazier's back-to-back layups, the second coming after he stole the ball in the backcourt, gave Penn State a 47-44 lead.
Penn State was up 50-44, before Nash hit a 3-pointer for the Bulls.
But Frazier hit a driving layup to extend the lead back to five.
A layup by Nash made it 53-49 with 22 seconds left, but the Bulls missed three 3-pointers in the final seconds.
The teams went into halftime tied at 17 after poor shooting performances on both sides. South Florida hit just 27 percent of its shots from the floor (4 of 15). But Penn State was worse, making just 23 percent from the floor (5 of 22). The Nittany Lions made just 1 of 12 attempts from beyond the arc.
The Lions shot 52 percent in the second half (12 of 23). South Florida shot 38.5 percent after intermission and hit just 14 of 41 shots in the game.
Gilchrist, who came in averaging over 14 points for the Bulls, was 0-for-4 in the half before leaving the game with a shoulder injury after taking a hard foul with just over five minutes to play in the half.
He missed his first foul shot then left the game and went to the locker room. He came back after halftime with an ice bag on his shoulder.
Frazier, who came in averaging almost 21 points and nine assists, had eight of the Lions' 17 points at intermission.
South Florida came into the weekend at 3-0, but lost by two points in overtime to Old Dominion Saturday.
Penn State was blown out Saturday by Kentucky, 85-47. It was the Nittany Lions worst loss since 2004.
Penn State is now 2-3 against the Bulls, a team it had not played since 1990.