sortir par la petite porte

Etymology
lit. 'to go out through the little door', i.e. the back door.

Verb

 * 1)  to leave with one's tail between one's legs defeated; in a cowardly or miserable manner