Jones posted a time of 21.54, which was .20 seconds behind France's Florent Manandou, who won the gold medal. Brazil's Cesar Cielo, who entered as the favorite, won the bronze.
The silver is the second Jones has won in the 2012 Olympics, as he was on USA's 400-meter freestyle relay team that won the silver earlier in the week.
Jones has now won three Olympic medals. He and the USA won the gold medal in the 400-meter freestyle relay in 2008.