From: Sven Rahlfs <rahlfs@di...> - 2003-03-25 14:16:40
i have installed htdig 3.16. All went wonderfull the searchmachine worked
good. Then i had to reinitialize the db and do the htdig -i. Suddenly there
are many new domains like http://www.microsoft.com, http://www.dotnet110.com...... but not
My htdig.conf is ok. When i do the htdig -c /path/to/my/conf/htdig.conf it
happens the same. My $[start_url} is set to a local file and there are only
my files listet.
please help me!
Dist: Suse 8.0
From: Jim Cole <lists@yg...> - 2003-03-25 22:36:09
On Tuesday, March 25, 2003, at 07:14 AM, Sven Rahlfs wrote:
> i have installed htdig 3.16. All went wonderfull the searchmachine
> good. Then i had to reinitialize the db and do the htdig -i. Suddenly
> are many new domains like http://www.microsoft.com, http://www.dotnet110.com......
> but not
This sounds like a problem with your limit_urls_to attribute. This
attribute specifies patterns that must be matched by URLs before they
are indexed. In the default configuration file, the attribute is set to
be the same as the start_url attribute. You might want to verify that
limit_urls_to is still present in your config file and that it has a
Another thing to check is that there are not multiple occurrences of
either start_url or limit_urls_to in your config file. Finally,
carefully check your config file for mistakes; you might even want to
run it through cat -v in order to check for non-printing characters
that could potentially interfere with parsing.
If you still can't track down the problem, it might be helpful to
provide the exact settings you are using for the start_url and
Get latest updates about Open Source Projects, Conferences and News.