Hilchos Lashon Hara, chapter 3, section 2

Sometimes Chazal (our Sages) seem to allow saying something if the speaker would also say it in front of the person. This is only about something that can be understood in two ways: one way is not bad at all, and the other way sounds bad. It all depends on how the speaker says it.

Chazal said that if someone would say such a thing to the person's face, it shows they do not mean to put them down, because people are normally embarrassed to shame someone to their face.

But if they would not say it in front of the person – even though, if it is understood badly, it is only "the dust of lashon hara" – it is still forbidden.