Phil Forte's three-pointer a minute into the overtime period gave the Cowboys a 70-68 lead but they would not score again during the extra five minutes as the Jayhawks pulled out the victory to advance.
"We're really proud of our team's effort. I thought they played hard," Cowboys head coach Travis Ford said. "Give Kansas credit. They played really well. (Andrew) Wiggins made a big short in the corner and he played a heck of a game.
"But our guys fought really hard, just didn't have enough left in the tank there late. We fought a lot of adversity. Foul trouble, our 4s and 5s, obviously we don't have a lot of depth there. But, hey, no excuses, Kansas deserved it. They made big shots and big rebounds against us. But our guys fought. I'm proud of them."
Forte's trey was the only points in overtime for the Cowboys, who were 1 of 8 from the field in the extra period. OSU committed two of its season-low five turnovers in overtime.
Freshman Andrew Wiggins scored 30 points and the Jayhawks made 8 of 10 free throws in overtime to secure the victory.
Le'Bryan Nash led the Cowboys with 19 points and 7 rebounds. Forte finished with 16 points but was just 4 of 12 from beyond the three-point line.
Marcus Smart contributed 14 points, 7 rebounds, 3 assists and 2 steals while playing 44 minutes, while Brown finished with 12 points and 3 steals.
The Jayhawks won the season series against the Cowboys with Thursday's victory. During the regular season, KU won 80-78 in Lawrence, Kan., while OSU won 72-65 inside Gallagher-Iba Arena on March 1. It took an extra five minutes Thursday to decide the winner in the deciding game in the season series.
The Cowboys now begin looking ahead to next week.
"I like how we're playing," Ford said. "I think there's some things we can do to adjust to get better. We gotta rebound the ball better, but I think we're playing well. The last five or six games, two games we've lost, we lost in overtime.
"And there's a lot of fight to us. So I think we're playing pretty good basketball."