This greatly simplifies the cleanup so we don't leave unneeded packages behind.
mentioned in commit d32d74fa