The Cardinals dropped five of six to begin a 10-game homestand.
Randy Choate (2-1) faced one batter, getting leadoff man David DeJesus to pop out with two men on to end the sixth after the Cubs had tied it at 4 on RBI singles by Castro and Darwin Barney off Seth Maness.
Barney, like Kozma an eighth-place hitter, had two RBI singles. He's batting .215 overall but has a .317 average against St. Louis with two homers and 10 RBIs.
The Cardinals combined two singles, a walk, a steal and a sacrifice fly off Edwin Jackson in a three-run first. Matt Holliday had one of the RBIs plus a steal off Jackson's slow move to the plate, before Jackson finally retaliated by picking off Jay.
Holliday handed the Cubs an unearned run in the third when he camped under Junior Lake's two-out high fly to medium left but then whiffed on the catch and DeJesus scored from first.
The Cubs settled for their first series win in St. Louis since a three-game sweep Sept. 13-15, 2010. They'd been 0-6-1 the previous seven series, and were trying for a three-game sweep.
NOTES: Cardinals OF Carlos Beltran fouled a ball off his right foot in the fifth, got medical attention on the field and then struck out before leaving with a bruise. X-rays were negative. ... Holliday grounded into his major league-leading 26th double play in the fifth. ... The Cubs' Darnell McDonald walked pinch-hitting in the seventh in his first at-bat of the year. He's the 50th player used by the Cubs, most in the majors. ... Cardinals starter Joe Kelly has been the starter the last two St. Louis wins. He allowed three earned runs in 5 1-3 innings and Jackson gave up four runs in five innings.
Link to box score: St. Louis 8, Chicago 4
Link to game discussion at The Cardinal Nation message board