From: Wolfgang W. <w.w...@mu...> - 2006-01-25 09:57:44
|
Hi, i have a problem with htdig. This is the first time i use htdig, maybe its a very simple error. i start htdig -vvv and get the following output: bdfl01:~ # htdig -vvv 0:1:http://bdis/ New server: bdis, 80 Retrieval command for http://bdis/robots.txt: GET /robots.txt HTTP/1.0 User-Agent: htdig/3.1.6 (unc...@ht...intainer) Host: bdis Header line: HTTP/1.1 200 OK Header line: Date: Wed, 25 Jan 2006 09:46:28 GMT Header line: Server: Apache/2.0.49 (Linux/SuSE) Header line: Last-Modified: Tue, 10 Jan 2006 12:37:48 GMT Converted Tue, 10 Jan 2006 12:37:48 GMT to Tue, 10 Jan 2006 12:37:48 Header line: ETag: "1447f-18-40a00620adf00" Header line: Accept-Ranges: bytes Header line: Content-Length: 24 Header line: Connection: close Header line: Content-Type: text/plain; charset=3DISO-8859-1 Header line: returnStatus =3D 0 Read 24 from document Read a total of 24 bytes Parsing robots.txt file using myname =3D htdig Robots.txt line: User-Agent: * Found 'user-agent' line: * Robots.txt line: Disallow: Found 'disallow' line: Pattern: pushed 1:1:http://bdis/ skipped pick: bdis, # servers =3D 1 0:1:0:http://bdis/: Unable to build connection with bdis:80 no server running pick: bdis, # servers =3D 1 bdfl01:~ # it says "Unable to build connection with bdis:80" but why the robots.txt = is found? any suggestions are welcome. Best regards Mit freundlichen Gr=FC=DFen Wolfgang Winkler --=20 Landeshauptstadt M=FCnchen, Kreisverwaltungsreferat Branddirektion, Sachgebiet Informationsverarbeitung An der Hauptfeuerwache 8, 80331 M=FCnchen Tel.: 089 2353-6169 E-Mail: w.w...@mu... |
From: G. T. Stresen-R. <ted...@ma...> - 2006-01-25 13:33:20
|
Can you send the contents of robots.txt? It looks like indexing is=20 being disallowed. > Found 'disallow' line: > Pattern: > pushed > 1:1:http://bdis/ skipped even though the error message is perhaps misleading. Do you get the same result if you remove the robots.txt file? Just a thought... Ted Stresen-Reuter On Jan 25, 2006, at 9:57 AM, Wolfgang Winkler wrote: > Hi, > i have a problem with htdig. > This is the first time i use htdig, maybe its a very simple error. > i start htdig -vvv and get the following output: > > bdfl01:~ # htdig -vvv > 0:1:http://bdis/ > New server: bdis, 80 > Retrieval command for http://bdis/robots.txt: GET /robots.txt HTTP/1.0 > User-Agent: htdig/3.1.6 (unc...@ht...intainer) > Host: bdis > > Header line: HTTP/1.1 200 OK > Header line: Date: Wed, 25 Jan 2006 09:46:28 GMT > Header line: Server: Apache/2.0.49 (Linux/SuSE) > Header line: Last-Modified: Tue, 10 Jan 2006 12:37:48 GMT > Converted Tue, 10 Jan 2006 12:37:48 GMT to Tue, 10 Jan 2006 12:37:48 > Header line: ETag: "1447f-18-40a00620adf00" > Header line: Accept-Ranges: bytes > Header line: Content-Length: 24 > Header line: Connection: close > Header line: Content-Type: text/plain; charset=3DISO-8859-1 > Header line: > returnStatus =3D 0 > Read 24 from document > Read a total of 24 bytes > Parsing robots.txt file using myname =3D htdig > Robots.txt line: User-Agent: * > Found 'user-agent' line: * > Robots.txt line: Disallow: > Found 'disallow' line: > Pattern: > pushed > 1:1:http://bdis/ skipped > pick: bdis, # servers =3D 1 > 0:1:0:http://bdis/: Unable to build connection with bdis:80 > no server running > pick: bdis, # servers =3D 1 > bdfl01:~ # > > it says "Unable to build connection with bdis:80" but why the=20 > robots.txt is found? > > any suggestions are welcome. > > > Best regards > Mit freundlichen Gr=FC=DFen > > > Wolfgang Winkler > > --=20 > Landeshauptstadt M=FCnchen, Kreisverwaltungsreferat > Branddirektion, Sachgebiet Informationsverarbeitung > An der Hauptfeuerwache 8, 80331 M=FCnchen > Tel.: 089 2353-6169 E-Mail: w.w...@mu... > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log=20= > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD = SPLUNK! > http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=103432&bid#0486&dat=121642= > _______________________________________________ > ht://Dig general mailing list: <htd...@li...> > ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html > List information (subscribe/unsubscribe, etc.) > https://lists.sourceforge.net/lists/listinfo/htdig-general |
From: G. T. Stresen-R. <ted...@ma...> - 2006-01-25 13:57:21
|
I'm at a loss, then. Maybe one of the developers on this list can chime in on what would be causing this error. If you really must get this up and running, then you need to start minimizing the variables: - try indexing another web site on a different server to check that htdig actually works - try indexing a different site on the same server - try running a different version of apache - try upgrading your version of htdig - double-check your config file (although I highly doubt this is the problem) Hope this helps. Ted On Jan 25, 2006, at 1:47 PM, Wolfgang Winkler wrote: > > > G. T. Stresen-Reuter schrieb: >> Can you send the contents of robots.txt? It looks like indexing is >> being disallowed. > > i have only tow lines in robots.txt: > > User-Agent: * > Disallow: > >>> Found 'disallow' line: >>> Pattern: >>> pushed >>> 1:1:http://bdis/ skipped >> even though the error message is perhaps misleading. >> Do you get the same result if you remove the robots.txt file? > > without robots.txt i get the following output: > > bdfl01:/srv/www/htdocs # htdig -vvv > 0:1:http://bdis/ > New server: bdis, 80 > Retrieval command for http://bdis/robots.txt: GET /robots.txt HTTP/1.0 > User-Agent: htdig/3.1.6 (unc...@ht...intainer) > Host: bdis > > Header line: HTTP/1.1 404 Not Found > Header line: Date: Wed, 25 Jan 2006 13:46:08 GMT > Header line: Server: Apache/2.0.49 (Linux/SuSE) > Header line: Vary: accept-language,accept-charset > Header line: Accept-Ranges: bytes > Header line: Connection: close > Header line: Content-Type: text/html; charset=iso-8859-1 > Header line: Content-Language: en > Header line: Expires: Wed, 25 Jan 2006 13:46:08 GMT > Header line: > returnStatus = 1 > pushed > 1:1:http://bdis/ skipped > pick: bdis, # servers = 1 > 0:1:0:http://bdis/: Unable to build connection with bdis:80 > no server running > pick: bdis, # servers = 1 > > >> Just a thought... >> Ted Stresen-Reuter >> On Jan 25, 2006, at 9:57 AM, Wolfgang Winkler wrote: >>> Hi, >>> i have a problem with htdig. >>> This is the first time i use htdig, maybe its a very simple error. >>> i start htdig -vvv and get the following output: >>> >>> bdfl01:~ # htdig -vvv >>> 0:1:http://bdis/ >>> New server: bdis, 80 >>> Retrieval command for http://bdis/robots.txt: GET /robots.txt >>> HTTP/1.0 >>> User-Agent: htdig/3.1.6 (unc...@ht...intainer) >>> Host: bdis >>> >>> Header line: HTTP/1.1 200 OK >>> Header line: Date: Wed, 25 Jan 2006 09:46:28 GMT >>> Header line: Server: Apache/2.0.49 (Linux/SuSE) >>> Header line: Last-Modified: Tue, 10 Jan 2006 12:37:48 GMT >>> Converted Tue, 10 Jan 2006 12:37:48 GMT to Tue, 10 Jan 2006 12:37:48 >>> Header line: ETag: "1447f-18-40a00620adf00" >>> Header line: Accept-Ranges: bytes >>> Header line: Content-Length: 24 >>> Header line: Connection: close >>> Header line: Content-Type: text/plain; charset=ISO-8859-1 >>> Header line: >>> returnStatus = 0 >>> Read 24 from document >>> Read a total of 24 bytes >>> Parsing robots.txt file using myname = htdig >>> Robots.txt line: User-Agent: * >>> Found 'user-agent' line: * >>> Robots.txt line: Disallow: >>> Found 'disallow' line: >>> Pattern: >>> pushed >>> 1:1:http://bdis/ skipped >>> pick: bdis, # servers = 1 >>> 0:1:0:http://bdis/: Unable to build connection with bdis:80 >>> no server running >>> pick: bdis, # servers = 1 >>> bdfl01:~ # >>> >>> it says "Unable to build connection with bdis:80" but why the >>> robots.txt is found? >>> >>> any suggestions are welcome. >> |
From: Wolfgang W. <w.w...@mu...> - 2006-01-31 15:56:01
|
Hi Ted, thnak's for your suggestions. i tried to minimize the variables. I indexed an other server, so htdig works and the config file is ok. i have no other side on my webserver. Apache version is 2.0.49, which should be ok. htdig version is 3.1.6, i will try to update to 3.2.xxx best regards Wolfgang G. T. Stresen-Reuter schrieb: > I'm at a loss, then. Maybe one of the developers on this list can chime > in on what would be causing this error. > > If you really must get this up and running, then you need to start > minimizing the variables: > > - try indexing another web site on a different server to check that > htdig actually works > - try indexing a different site on the same server > - try running a different version of apache > - try upgrading your version of htdig > - double-check your config file (although I highly doubt this is the > problem) > > Hope this helps. > > Ted > |