baby and bathwater

Etymology
From the phrase.