Hilchos Lashon Hara chapter 6, section 7

Sometimes he knows the lashon hara he was told is true, but the act can be seen in two ways: as something bad, or, if one judges favorably, as nothing bad at all. The speaker judged it the bad way, and so the story came out as something shameful.

A listener who does not judge the person favorably breaks the commandment "You shall judge your fellow charitably," because he did not judge favorably. He also breaks the prohibition of accepting lashon hara, because the person comes out shamed after he did not judge him favorably.