throw the baby out with the bathwater - definition and synonyms

phrase
  1. to get rid of the good and useful aspects of something without intending to, while you try to get rid of its negative aspects
See also main entry: baby