#268 XMLHttpRequest is more synchronous than Mozilla

[Steven Grimm]

Mozilla has some different XMLHttpRequest behavior than
Internet Explorer. In particular, Mozilla browsers call
the onReadyStateChange handler for every chunk of
incoming data, whether or not the page has finished
loading yet; the partial results can be inspected in
the object's responseText field. This is valuable for
displaying incremental results from a long-running
query, for example.

Sites that depend on that behavior can determine
whether they're on a Mozilla-derived browser or an
IE-based one by using code like

if (window.XMLHttpRequest) {
... allocate an XMLHttpRequest, you're on Mozilla ...
}
else {
... use a hidden iframe or other mechanism, you're
on IE ...
}

HTMLUnit has IE's XMLHttpRequest semantics and
Mozilla's treatment of XMLHttpRequest as a built-in
type, so the above code fails to do the right thing in
HTMLUnit even though it's fine in the real world.

Ideally HTMLUnit should support both kinds of semantics
depending on the browser version you specify when you
create the WebClient object. But barring that, if it's
going to provide IE's behavior, it should also enforce
IE's object instantiation API. And barring that, it
should at least not allow "new XMLHttpRequest()" to
succeed when the WebClient browser type is set to
Internet Explorer.

[Daniel Gredler]

Logged In: YES
user_id=1109422

The current XMLHttpRequest implementation treats
XMLHttpRequest as a built-in type regardless of the browser
being emulated, as you have pointed out. This will hopefully
go away in the very near future. However, I'm not sure we're
going to be able to change the onReadyStateChange behavior.
How do fewer calls to the onReadyStateChange function affect
your unit testing?

[Steven Grimm]

Logged In: YES
user_id=59197

One of the things I'm testing is a real-time information
service that, on Mozilla browsers, works by keeping a
persistent XMLHttpRequest connection open and sending
updates down the wire as they happen. (On IE, it uses an
iframe instead, but iframes cause browser history problems
on Mozilla so we have to use XMLHttpRequest to keep the user
experience smooth.) Right now the only way I can do an
end-to-end test to make sure the update code is sending out
the expected notifications is to launch a real browser. I
was hoping to use HTMLUnit to integrate that element of the
test process into our JUnit test suite. Obviously we still
need to do real-world browser compatibility testing, but for
day-to-day developer sanity checking HTMLUnit would be perfect.

This is more of an "it would be nice to more exactly emulate
the browsers" than an "HTMLUnit is useless without this"
bug. I will lower its priority. If the XMLHttpRequest type
can be made less built-in-looking, that'd at least solve the
problem of people expecting Mozilla's behavior.

[Daniel Gredler]

Logged In: YES
user_id=1109422
Originator: NO

I had a look at this, and I think I have an idea of what needs to change. Not yet fixed, but I've added a test for it: XMLHttpRequestTest#testStreaming().

[Marc Guillemot]

@Daniel: are you sure of your test?

@Steven Grimm: when you write "chunk", do you just mean a piece of content or a chunked http response (ie with header Transfer-Encoding: chunked)?

[Daniel Gredler]

Yeah, I developed the test in FF and then moved it to the HU unit tests. The idea is that FF streams results and they come, but HU waits for all content to be available before triggering the state changes.

The problem is that the underlying HttpClient library only allows you to get the content stream once, and I don't want this implementation detail to leak through into our API. I read somewhere that they're reworking this for version 4.0, so I'm waiting for that before tackling this bug.

[Daniel Gredler]

"streams results and they come" -> "streams results as they come"