Trevor Holder (4-1) allowed just two runs on eight hits and struck out six in 6.0 innings to earn his 19th career victory, moving him into a tie for eighth on the all-time UGA wins list.
Will Harvil pitched two scoreless innings and Dean Weaver worked the ninth and picked up his second save.
"First of all we really appreciate the crowd," said head coach David Perno. "That was great to see. We let a couple of opportunities pass us by in the middle innings, but we were still able to pull it out. I really liked the way we played defense tonight."
Mississippi State (13-7, 2-2 SEC) scored first with a run in the second, then Georgia answered with two runs in the bottom of the inning. Cerione drew a one-out walk, and Michael Demperio doubled down the left field line to put runners at second and third. An infield single by Levi Hyams that barely stayed inside the third base line scored Cerione to make it 1-1. Johnathan Taylor reached on a fielder's choice and drove in another run to give Georgia a 2-1 lead.
In the third, Joey Lewis singled with two outs, then Cerione connected on his first home run of the season to left center, giving the Bulldogs a 4-1 lead.
Cerione saved a run in the fourth inning with a diving catch in center field on a line drive by Mississippi State's Scott DeLoach with a runner at second base, and another great defensive play by Georgia in the seventh kept the Bulldogs' 4-2 lead intact. With two outs and runners at first and second, right fielder Peter Verdin hung on to a fly ball off the bat of Connor Powers as he fell over the fence into the Georgia bullpen for the third out of the inning.
"Peter's play down the right field line gave us a huge lift," said Perno. "He really swung the bat well tonight and gave us a couple of great at-bats."
Georgia added a run in the seventh off a Bryce Massanari single to left that scored Colby May to push the lead to 5-2. Mississippi State scored a run in the ninth and had the tying run at first when Weaver got Ryan Collins to fly out to deep left center to end the game.
Georgia and Mississippi State will play the second game of the series Saturday at 3 p.m. Junior lefty Alex McRee (2-0, 0.77 ERA) will start for Georgia.