He was talking about intermarriage with gentiles. This was not permitted for Priests and Levites .

Source: Ezra 9 and 10