Matt Holliday drew a leadoff walk to start the winning rally. He advanced to third on Matt Adams' single. After Peter Bourjos ran for Holliday, Jhonny Peralta walked to load the bases. Hoover (1-10) then plunked Jay to force in Bourjos with the winning run.
Pat Neshek (6-0) pitched one inning of scoreless relief as the Cardinals won for the fifth time in six games and remained 2 1/2 games behind Milwaukee in the NL Central.
The Reds have lost four in a row.
Jay scored the tying run in the eighth on pinch-hitter Daniel Descalso's RBI double off reliever Jumbo Diaz.
Todd Frazier put the Reds in front 2-1, with a two-run homer in the fourth. It was Frazier's 21st home run of the season, but first since July 20.
Cincinnati starter Alfredo Simon gave up three runs and eight hits over 5 1-3 innings. He struck out two and walked one.
St. Louis starter John Lackey gave up four runs, two earned, in six innings. He struck out seven and walked one.
Cincinnati scored twice in the sixth on a bases-loaded double by former Cardinals outfielder Ryan Ludwick. Errors by Holliday and Matt Carpenter and a walk set the stage for Ludwick's hit, which pushed the lead to 4-1.
Peralta homered in the sixth to cut the deficit to 4-2. It was Peralta's 17th home run of the season, setting the single-season mark for a St. Louis shortstop. Jay and Oscar Taveras followed with hits before pinch-hitter Shane Robinson singled of reliever Sam LeCure to bring the Cardinals to 4-3.
St. Louis took a 1-0 lead on a run-scoring single by Holliday in the third.
Reds: Closer Aroldis Chapman was unavailable Tuesday for the second night in a row with what manager Bryan Price called an, "achy," shoulder. The reliever is listed as day-to-day by the Reds training staff. Chapman has 26 saves in 28 chances. He walked four successive batters on Sunday in Colorado.
Cardinals: INF Mark Ellis was put on the 15-day disabled list with a left oblique strain. Ellis, who is hitting .192 with 12 RBI, last played on Tuesday. INF Greg Garcia was recalled from Triple-A Memphis in time for Tuesday's contest. Garcia was hitting .273 with 40 RBI in 103 games for the Redbirds.
JAY KEEPS ROLLING
St. Louis OF Jon Jay extended his hitting streak to 11 games with a single in the second, the first of two hits. He is 15 for 31 during the run and reached base four times Tuesday.
Reds: RHP Johnny Cueto (15-6, 2.06) will start the final game of the three-game set. Cueto is 7-0 with a 2.24 ERA over his last eight starts. His seven-game winning streak is a career high.
Cardinals: RHP Lance Lynn (13-8, 2.91) will make his team-leading 26th start of the season. He is 5-1 lifetime against the Reds.
Link to box score: St. Louis (68-57) 5, Cincinnati (61-65) 4
Join the pre-game, in-game and post-game discussion about each contest at The Cardinal Nation message board.