After entering a directory and exiting out of it, pcmanfm reset back the selection to the first file in the original directory.
For example, say you selected Downloads directory, entered it, then exited out of it (Enter, Alt+Up).
Desktop
Documents
Downloads <--
Music
Pictures
Public
Now pcmanfm assumes previous selection is Desktop (try hitting ArrowDown). It should still assume selection is Downloads. This is how many file managers do it (Dolphin, Nautilus, even Windows File Explorer). Which makes it easier to check the contents of directories one by one (with Enter, Alt+Up, Down combos).
Demonstration gif:

Anonymous