They would stay in Egypt until the death of Herod. They moved to Nazareth because they were afraid of Herod’s son Archelaus.

Source:

Mathew 2:19-23