The effect of calling the method window.location.replace(newUrl) is defined as:
Replace the current document with the one at the provided URL. The difference from the assign() method is that after using replace() the current page will not be saved in session history, meaning the user won't be able to use the Back button to navigate to it. (https://developer.mozilla.org/en/DOM/window.location#replace)
The current implementation in HtmlUnit doesn't update the session history correctly since the new URL is just added to it instead of replacing the old one.
Simple test case
Potential Fix
I've attached a patch that works for me but I don't know if there are any issues related to multithreading to take care about.
Fixed in SVN; Thanks for the patch.