bug fix for post redirect

[yairbm]

hi all

i think i found a bug: i m sending an http post request:
<http method="post" url="${loginUrl}">
<http-param name="user"><template>${user}</template></http-param> ...

the url is returning 302 redirect. the webharvest redirects but uses a get method instead of post, so the request fails (the server wants post...).

I've fixed that by changing line 191 in HttpClientManager.java:
old:
method = new GetMethod(newURI);
new:
if ( "post".equalsIgnoreCase(methodType) ) {
method = createPostMethod(newURI, params);
} else {
method = createGetMethod(newURI, params, charset);
}

is that right? who should i send this fix to?

thanks
yair

[Vladimir Nikic]

Thanks Yair, will be fixed in next release.

[Jason]

Speaking of re-directs, I see the following in HttpClientManager:

        // if there is redirection, try to download redirection page
        if ((statusCode == HttpStatus.SC_MOVED_TEMPORARILY) ||
            (statusCode == HttpStatus.SC_MOVED_PERMANENTLY) ||
            (statusCode == HttpStatus.SC_SEE_OTHER) ||
            (statusCode == HttpStatus.SC_TEMPORARY_REDIRECT)) {
            Header header = method.getResponseHeader("location");
            if (header != null) {
                String newURI = header.getValue();
                if ( newURI != null && !newURI.equals("") ) {
                    newURI = CommonUtil.fullUrl(url, newURI);
                    method.releaseConnection();
                    method = new GetMethod(newURI);
                    identifyAsDefaultBrowser(method);
                    client.executeMethod(method);
                }
            }
        }

Why not just use method.setFollowRedirects(true)?

[Vladimir Nikic]

Interesting question :) I cannot remember now, but there are 3 possible answers:

1. first version of HttpClient that was used in Web-Harvest didn't have setFollowRedirects method
2. it did have it, but for some reason it didn't work well
3. somehow I missed that HttpClient have that handy method

Anyway, thanks for the post. I'll surely check how it is impemented inside HttpClient, and I also must collect a number of real-life redirection cases to check if it works well.
If you have some URLs please send me to include them.

Regards, Vladimir.