Other than not wanting a stone to irritate the wearer is there any other reason for not flush setting stones in thin metal where the back of the stone will be exposed?

I think I remember hearing once that is a stone is exposed at the back it has a much higher risk of falling out if knocked/dropped for example. Is this true? Are there any other reasons to avoid setting stones like this?