[nekohtml-dev] [ nekohtml-Bugs-2803322 ] 204 response to POST causes unwanted GET on URL

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Bugs item #2803322, was opened at 2009-06-09 15:48
Message generated for change (Comment added) made by leebutts
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=952178&aid=2803322&group_id=195122

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Pending
Resolution: Invalid
Priority: 5
Private: No
Submitted By: Lee Butts (leebutts)
Assigned to: Marc Guillemot (mguillem)
Summary: 204 response to POST causes unwanted GET on URL

Initial Comment:
I'm currently using HtmlUnit to test a site that returns a 204(No Content) response to a POST request.

When neko tries to parse the result it causes another GET request on the url due the input stream being null (as returned by HttpMethodBase.getResponseBodyAsStream). This GET request causes a 403 response as it is a POST only url and requires special authentication headers.

The code in question is in HtmlScanner.setInputSource:

InputStream inputStream = source.getByteStream();
            if (inputStream == null) {
                URL url = new URL(expandedSystemId);
                inputStream = url.openStream();
            }

I'm not sure what the best fix is for this as I presume that behaviour is there for some other use case...

Perhaps you could catch any exceptions that occur from url.openStream() and continue with a null or empty input stream?

----------------------------------------------------------------------

Comment By: Lee Butts (leebutts)
Date: 2009-06-09 18:11

Message:
Ok, I'll raise this issue in HtmlUnit's tracker, thanks for the quick
feedback.

cheers

Lee

----------------------------------------------------------------------

Comment By: Marc Guillemot (mguillem)
Date: 2009-06-09 18:06

Message:
HtmlUnit's code creating the XMLInputSource to call the parser is as
follows:
        final InputStream content = webResponse.getContentAsStream();
        final XMLInputSource in = new XMLInputSource(null, url.toString(),
null, content, charset);

The JavaDoc for this XMLInputSource c'tor says "Constructs an input source
from a byte stream." but doesn't complain if the input source is null and
doesn't store the fact that it has been constructed with this ctor. The
consequence is that users of XMLInputSource like NekoHtml can't make the
difference between an XMLInputSource created with null stream and the usage
of following ctor:
XMLInputSource(String publicId, String systemId,  String baseSystemId)

This is surely not good from XMLInputSource and the consequence is that I
don't see how NekoHTML could change its behaviour here and make a
distinction between null input stream and no input stream provided.

As a consequence, I'm quite sure now that this is a problem on HtmlUnit's
side.

----------------------------------------------------------------------

Comment By: Lee Butts (leebutts)
Date: 2009-06-09 17:42

Message:
I'm not sure, it seems like unexpected behaviour by neko to me.

If I ask it to parse a response I don't expect it to go and send a new
request.

----------------------------------------------------------------------

Comment By: Marc Guillemot (mguillem)
Date: 2009-06-09 17:34

Message:
Shouldn't this be handled by HtmlUnit?

I understand XMLInputSource as a placeholder allowing to specify different
input sources: it may be a stream or an identifier and if no stream is
configured, then it means that the identifier has to be used.
http://xerces.apache.org/xerces2-j/javadocs/xni/org/apache/xerces/xni/parser/XMLInputSource.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=952178&aid=2803322&group_id=195122