Hilchos Rechilus, chapter 8, section 4

More things are forbidden because of avak rechilus. For example, if what he tells is not an insult, but people usually mind a little when it is said about them to their face.