St. Louis fell into a tie with Pittsburgh for the top spot in the NL Central. Both teams are 85-61 with 16 games remaining. The idle Cincinnati Reds are 2 1/2 back.
Milwaukee broke a three-game losing streak against St. Louis and beat the Cardinals for only the fourth time in 16 meetings this season.
The Cardinals are 5-1 on their nine-game homestand.
Thornburg (2-1), making his eighth career start, gave up two runs and three hits. He spent most of the season with Triple-A Nashville, going 0-9 with a 5.79 ERA in 15 starts.
Thornburg struck out six and walked two. He has allowed two runs or fewer and gone six innings in all five major league starts this season.
Jim Henderson earned his 25th save in 29 opportunities.
Halton, who had three RBIs, hit a two-run homer off Joe Kelly (8-4) in the fourth to put the Brewers up 4-0. Halton also had a run-scoring single in the second.
Kelly allowed four runs, three earned, on seven hits in five innings. He won his previous five starts, and the Cardinals had won his last eight.
Matt Adams hit a two-out solo homer in the ninth, his 13th of the season.
The Cardinals pulled to 4-2 in the fourth on a run-scoring double by Matt Holliday and a sacrifice fly by David Freese. They also put two on with one out in the sixth, but Thornburg retired Carlos Beltran and Freese to end the threat.
Norichika Aoki tripled in Yuniesky Betancourt in the ninth.
NOTES: St. Louis RHP Trevor Rosenthal missed the game. His wife, Lindsey, was in labor awaiting their first child. ... Cardinals C Yadier Molina missed his second successive game due to an illness to his mother, Gladys. General manager John Mozeliak said Molina could be back Friday. ... St. Louis 1B Allen Craig is making progress with his injured left foot. ''We're still optimistic he'll play again,'' Mozeliak said. ''He's feeling a lot better. It's one of those things where you're just going to have to let some time progress.'' Craig sustained a left mid-foot strain on Sept. 4. ... The Brewers begin a 10-game homestand Friday against Cincinnati. Kyle Lohse (9-9, 3.59 ERA) will face Reds RHP Mat Latos (14-5, 3.02) in the opener of a three-game series. ... St. Louis ends its homestand with three games against Seattle beginning Friday. Cardinals RHP Adam Wainwright (16-9, 3.03) faces Hisashi Iwakuma (12-6, 2.97).
Link to box score: Milwaukee 5 at St. Louis 3
Link to game discussion at The Cardinal Nation message board