Civilians in sectors remember their owner even when the sector changes owner. Civilians on ships and land units don't. That's because ships and land units don't have an "old owner", unlike sectors.
The game attempts to avoid magical changes of civilian allegiance by letting you load or unload civilians only if the sector old owner matches the ship or land unit owner.
However, such a magical change still happens when the ship or land unit owner changes: ship/land unit boarded, sold, given away via unload, or land unit captured on sector takeover.
Related bug#752775