When we see someone break a known rule—like “don’t litter”—and no one else seems to be breaking it, that single “deviant act” sticks out, which makes the rule more vivid and powerful in our minds. But when we see a person break a rule and everyone else seems to be breaking it, we are even more likely to break the rule, too—because there is evidence that we can get away with it, or even are expected to break the espoused rule.
When one or two “bad apples” are kept around—and perhaps rejected, punished, and shunned—everyone else is more conscientious about following the rules.