Go to the '.' options button when in-game > manage words > household management (it's an icon with multiple figures at the top right) > unplayed households > I usually delete most of the 'not in world' Sims, because they are random and usually unknown to my Sims, or I put them in the world if I like them.