Cundiff's kick helped the Browns (1-1) snap a nine-game losing streak in home openers and gave rookie coach Mike Pettine his first NFL win. Cleveland had not won at home since 2004.
Quarterback Brian Hoyer drove the Browns 85 yards in 14 plays in the final 2:46 to set up Cundiff's game-winner. Hoyer completed several big passes, none more crucial than his 28-yarder to Andrew Hawkins with 13 seconds to go.
The Saints (0-2) rallied from a 13-point deficit to take the lead on Mark Ingram's 1-yard TD run with 12:12 left.
Down 24-23, Hoyer took over with the ball at his own 4. Missing suspended wide receiver Josh Gordon and without injured Pro Bowl tight end Jordan Cameron and running back Ben Tate, Hoyer calmly took the Browns down the field.
He completed two passes on third down and kept the drive alive by firing a 10-yard pass over the middle to tight end Gary Barnidge on fourth down. With 15 seconds remaining, Hawkins somehow got behind New Orleans defense and Hoyer lobbed him the ball down to the Cleveland 11.
Out came Cundiff, who drilled his kick to the delight of Browns fans, who seconds earlier feared they would lose another heart-breaker.
Hoyer went 8 of 11 for 77 yards on the last drive and finished 24 of 40 for 204 yards and a TD. He came off the field for a few plays as rookie backup Johnny Manziel made his NFL debut. Manziel handed the ball off twice and had one incompletion.
Safety Tashaun Gipson returned an interception 62 yards for a touchdown for the Browns, who lost their season opener in Pittsburgh last week.
After a terrible start, Brees got into a groove and had the Saints in position to bounce back from a tough overtime loss last week in Atlanta. But New Orleans defense couldn't stop the Browns late and coordinator Rob Ryan and coach Sean Payton had a heated discussion on the sideline in the fourth quarter.
It was a milestone day for Brees, who jumped Hall of Famer John Elway into fourth place in career yardage. Brees went by Elway on a six-yard completion in the second quarter to Graham, who didn't make his first catch until deep into the second quarter.
Brees went 27 of 40 for 237 yards. Graham had 10 catches for 118 yards.
Browns rookie running back Terrance West scored on a 9-yard to give the Browns a 23-17 lead in the third. West took an inside handoff, made a couple cuts and scampered into the end zone in front of the Dawg Pound before performing a little dance. West and rookie Isaiah Crowell combined for 48 yards on nine carries during the drive.
Brees' 1-yard TD pass to Graham in the third put the Saints ahead 17-16. Graham caught the pass near the goal line and reached across as he was being driven out of bounds.
Brees finally found his rhythm -- and Graham -- late in the first half. Brees hooked up with his favorite 6-foot-7 target to pass Elway, and the pair hooked up moments later for a 9-yard TD with three seconds left in the half to pull the Saints within 16-10.
Brees was under duress throughout the first half as Cleveland's defensive front harassed him into making quick throws and forcing others. On the pick, Paul Kruger hit Brees just as he released the ball, sailing it over Graham's outstretched hands and right to Gipson, who picked up some blockers and scored untouched.