Nelson, who agreed to terms last week, was 3-1 with one save in 59 appearances for the Florida Marlins last season. His 2.00 ERA was third-best among NL relievers behind Hong-Chih Kuo's 1.69 and Brad Lidge's 1.95.
"We feel like Joe can really help our bullpen," Rays executive vice president of baseball operations Andrew Friedman said. "He had a tremendous season in 2008 and has been very effective against both left-handed and right-handed hitters."
The 34-year-old Nelson allowed 42 hits while striking out 60 in 54 innings for the Marlins last season.
Marlins, top starter Johnson agree to new deal
Sergio Busquets Set To Sign Barcelona Contract Extension
Cards finalize deal with ex-Rays reliever Miller