They had to remove all yeast or leaven from their houses. Passover is also called the feast of unleavened bread.

Source: Exodus 12:15 and several other verses.