htmlparser-1.5
with some URLs on slow servers the parser hangs when parsing the content in the for-loop:
for (NodeIterator e = parser.elements(); e.hasMoreNodes(); )
{
// processing...
}
this happens even though the HttpURLConnection for the URL returns 200 as the reponseCode, meaning it is connected!
my guess is that, despite the URL being connected, the parser will ask for the DataStream from the server for that URL, and since its a very slow server it takes a very long time to process. the longest period i have tried is more than 40 minutes! which is stupid.
one such url is "http://ce.sharif.ac.ir/". I have tried setting some values for the HttpURLConnection, such as:
htmlparser-1.5
with some URLs on slow servers the parser hangs when parsing the content in the for-loop:
for (NodeIterator e = parser.elements(); e.hasMoreNodes(); )
{
// processing...
}
this happens even though the HttpURLConnection for the URL returns 200 as the reponseCode, meaning it is connected!
my guess is that, despite the URL being connected, the parser will ask for the DataStream from the server for that URL, and since its a very slow server it takes a very long time to process. the longest period i have tried is more than 40 minutes! which is stupid.
one such url is "http://ce.sharif.ac.ir/". I have tried setting some values for the HttpURLConnection, such as:
httpCon.setConnectTimeout(30000);
httpCon.setReadTimeout(10000);
but it didnt help at all. somewhere inside the NodeIterator for-loop it waits for something, as i could see by debugging it.
Is there a way to set a time-out value so that it doesnt freeze the whole thing?
all kind of helping is very welcome,!
cheers, KooRosh
Can the Admin please remove this thread, as it was a misunderstanding from me, and I think I know how to fix it!
cheers