For me the first thing is using chareset from response. If it is given, it uses the one from server, instead of config.
Second thing is exception. The site I was harveting had several bad links - (spaces or invalid characters). IllegalArgumentException is thrown by HttpClient, if the bad URL is given.
Two minor enhancements. Should be patch - sorry.
For me the first thing is using chareset from response. If it is given, it uses the one from server, instead of config.
Second thing is exception. The site I was harveting had several bad links - (spaces or invalid characters). IllegalArgumentException is thrown by HttpClient, if the bad URL is given.
HttpProcessor:90
Best regards,
Marcin Okraszewski
ok, thanks, I'll try to handle both issues for the following release.