Now it looks like the crowd favorite is hearing things from the fans he's never heard before — and they run deeper than just his performance at the plate.
"(Saturday) night was pretty big. A lot of people saying a lot of things I've never heard before," Swisher said Sunday after the Yankees punchless 3-0 loss to the Detroit Tigers in Game 2 of the ALCS. "Prime example — I missed that ball in the lights [in Game 1] and next thing you know, I'm the reason that Jeter got hurt. It's kind of frustrating. They were saying it [Jeter's injury] was my fault."
There were even several inappropriate tweets about him to his wife, actress JoAnna Garcia.
"It's a crazy game. Being here in New York, obviously there's a lot of expectations," said Swisher. "I don't know what to say, bro. I'm just going to kind of let it go."
Swisher is one of the most popular players among Yankees fans — especially among the Bleacher Creatures who usually call Swish one of their own, but he has been struggling over the postseason (batting .154 with one RBI through seven games) and has become one of the least productive postseason players in recent history. A .167 batting average (25-for-150) with 44 strikeouts attest to that fact.
"It hurts," he said. "Sometimes I'm a sensitive guy. Some of the things people say, they get under your skin a little bit but hey, I've been lucky to be here for the last four years, and we're not going to go down like this.
"I'm one of those guys that you give me a hug and I'll run through a brick wall for you. Right now it seems like there's a lot of ... I'm trying to find a way to word this the right way, it's tough. Because you want to play for your city, you want to play for your team. Just right now, it's just really tough."
While it doesn't sound like Swisher will be getting any cuddles from fans, manager Joe Girardi isn't showing him any love either.
Swisher was dropped to the eighth spot in the lineup Sunday — the lowest he's batted since last year.
The slumping Swisher might want to pick his verbal battles with the Yankees fans under better circumstances.
Game 2 could be the lasting image they have of the pending free-agent wearing pinstripes in the Bronx if the Yankees don't win two games in Detroit. There's nothing worse than leaving your former sweetheart(s) an angry breakup message on the answering machine.
"It's not about me," Swisher said when asked about his postseason struggles. "It's about the team."