I am getting IO Exception like below. My config file is also given. I am not giving any & in the config file it automatically adds in the url. How to avoid it
Config File.
<?xml version="1.0" encoding="UTF-8"?>
<config charset="UTF-8">
<include path="functions.xml"/><!-- defines search keyword and start URL --><var-def name="search">travel chennai</var-def><var-def name="url"><template>http://http://www.guruji.com/local?q=${search}</template></var-def><!-- collects all travel URLs --><var-def name="imgLinks"><call name="download-multipage-list"><call-param name="pageUrl"><var name="url"/></call-param><call-param name="nextXPath">//a[.='Next']/</call-param><call-param name="maxloops">5</call-param></call></var-def><!-- download images and saves them to the files --><loop item="link" index="i" filter="unique"><list><var name="imgLinks"/></list><body><file action="write" type="binary" path="guruji/${search}.txt"><http url="${sys.fullUrl(url,link)}"/></file></body></loop>
</config>
Exception
Exception in thread "main" org.webharvest.exception.HttpException: IO error duri
ng HTTP execution for URL: http://http://www.guruji.com/local?q=travel+chennai&
at org.webharvest.runtime.web.HttpClientManager.execute(Unknown Source)
at org.webharvest.runtime.processors.HttpProcessor.execute(Unknown Sourc
e)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyTextContent(Un
known Source)
at org.webharvest.runtime.processors.HtmlToXmlProcessor.execute(Unknown
Source)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.VarDefProcessor.execute(Unknown Sou
rce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.EmptyProcessor.execute(Unknown Sour
ce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.WhileProcessor.execute(Unknown Sour
ce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.ReturnProcessor.execute(Unknown Sou
rce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.CallProcessor.execute(Unknown Sourc
e)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.VarDefProcessor.execute(Unknown Sou
rce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.Scraper.execute(Unknown Source)
at org.webharvest.runtime.Scraper.execute(Unknown Source)
at CommandLine.main(Unknown Source)
Caused by: java.net.UnknownHostException: http
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.c
reateSocket(DefaultProtocolSocketFactory.java:79)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.c
reateSocket(DefaultProtocolSocketFactory.java:121)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java
:704)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Htt
pMethodDirector.java:382)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMe
thodDirector.java:168)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
a:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
a:324)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The line
<template>http://http://www.guruji.com/local?q=${search}</template>
could be a problem. Having two 'http://'s in the url could be problematic.
cheers.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am getting IO Exception like below. My config file is also given. I am not giving any & in the config file it automatically adds in the url. How to avoid it
Config File.
<?xml version="1.0" encoding="UTF-8"?>
<config charset="UTF-8">
</config>
Exception
Exception in thread "main" org.webharvest.exception.HttpException: IO error duri
ng HTTP execution for URL: http://http://www.guruji.com/local?q=travel+chennai&
at org.webharvest.runtime.web.HttpClientManager.execute(Unknown Source)
at org.webharvest.runtime.processors.HttpProcessor.execute(Unknown Sourc
e)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyTextContent(Un
known Source)
at org.webharvest.runtime.processors.HtmlToXmlProcessor.execute(Unknown
Source)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.VarDefProcessor.execute(Unknown Sou
rce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.EmptyProcessor.execute(Unknown Sour
ce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.WhileProcessor.execute(Unknown Sour
ce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.ReturnProcessor.execute(Unknown Sou
rce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.CallProcessor.execute(Unknown Sourc
e)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.processors.BaseProcessor.executeBody(Unknown S
ource)
at org.webharvest.runtime.processors.BaseProcessor.getBodyListContent(Un
known Source)
at org.webharvest.runtime.processors.VarDefProcessor.execute(Unknown Sou
rce)
at org.webharvest.runtime.processors.BaseProcessor.run(Unknown Source)
at org.webharvest.runtime.Scraper.execute(Unknown Source)
at org.webharvest.runtime.Scraper.execute(Unknown Source)
at CommandLine.main(Unknown Source)
Caused by: java.net.UnknownHostException: http
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.c
reateSocket(DefaultProtocolSocketFactory.java:79)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.c
reateSocket(DefaultProtocolSocketFactory.java:121)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java
:704)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Htt
pMethodDirector.java:382)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMe
thodDirector.java:168)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
a:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
a:324)
The line
<template>http://http://www.guruji.com/local?q=${search}</template>
could be a problem. Having two 'http://'s in the url could be problematic.
cheers.
thanks for the reply.
I found the problem and corrected it.