Rolle said he was misquoted.
"I don't call any player out. I'm a professional in this league," said Rolle, who is in his fifth season.
NFC West blog
•Blog network: NFL Nation
The Cardinals face the top-seeded Saints on Saturday in the Superdome.
"Drew Brees has earned his right as one of the most outstanding quarterbacks in this league. His play on the field speaks for itself," Rolle said.
"But it is what it is. If they want to go use that as ammunition, something that I didn't say, then let it be done. I don't back down from anyone. I'm not scared of anyone. The game still has to be played on Saturday, and I'm coming there to play ball, bottom line."
Yahoo! Sports quoted Rolle as saying, "I'll face Drew Brees any day of the week before I face [Rodgers] again" after Rodgers threw for a Green Bay playoff-record 422 yards and four touchdowns in the Cardinals' 51-45 overtime victory in the first-round NFC playoff game.
Rolle said he was having a conversation with teammate Adrian Wilson when his comments were overheard.
"[The reporter] threw Drew Brees' name out there as I was walking out of the locker room, and I'm like, 'I'm not talking about Drew Brees right now.' After a game like that, who is going to be thinking about Drew Brees?" Rolle said.
"I mean, the guy who just ripped us was Aaron Rodgers. That's who I was talking about. That's who I was giving praise to."
Yahoo! reporter Michael Silver sought to clarify the matter, telling ESPN.com's Mike Sando, "Antrel Rolle gave that quote to me at the end of an interview at his locker [as he said, he was starting to walk away]. I wrote it down as he spoke and quoted him exactly. It was related in its proper context and was a completely accurate portrayal of what he said about Aaron Rodgers and Drew Brees.
"I am angered that Rolle would call me a liar publicly and disappointed that his quotes would run without any attempt to contact me for my response."
Rodgers was fourth in the NFL with 4,434 yards passing and 30 touchdown passes in the regular season. Brees threw for 4,388 yards and 34 touchdowns, sitting out the final game of the regular season after the Saints clinched their playoff seeding.
Brees led the NFL with a 109.6 quarterback rating.
Rolle, a Pro Bowl alternate this season, spent his first three years at cornerback before moving to free safety in 2008. He has 13 career interceptions, four this season.
"My image means a lot to me," Rolle said. "I don't do things like that. That's disrespecting another player, someone who has put in more time in the league that I have. Who am I to go and call him out?"
Sources: Kurt Warner of Arizona Cardinals expected to retireBarca book final spot
