[Icecrawler-devel] Networking works

[Guillaume C. <las...@gm...>]

Mwhahahaha. At last !

Like always, almost, the bug was because of a tiny little thing... Well, for
those of you who care about knowing what was going wrong, here is a little
explanation :
The issue we had with the networking code was simple. Requests got done, but
we couldn't get any data from them. The tests were conducted using the
simple and well known URL "http://www.google.fr". Request any data out of
it, and you'll be given nothing... Almost. And this is wanted behaviour. Try
this little thing in a console (windows / linux / something else... It
should work on any platform) :
do a "telnet www.google.fr 80". Once it reports you are connected, type "GET
/ HTTP/1.0" followed by 2 strikes on your <return> / <enter> key. And look
at what you got :-) A redirect instruction for the user-agent (ICECrawler in
our case). We don't handle them for now and this is why we got no data. I
changed the test URL to "http://www.google.fr/index.html" and guess what ?
It worked !

I'll keep on working on the network module until I can get redirection
working along with some other things. My work should be committed and moved
to the main source tree tonight, moving the Network module from the temp
directory for good !

Have a nice end-of-the day !

Guillaume