From: Paul Happ <phapp@yh...> - 2006-02-08 17:09:15
I have htdig running on my shared server environment after working out some
problems. I thought I'd share this in case it might be of help to others .
I have a feeling I'm just missing a basic concept of htdig and this all
makes perfect sense. If someone with a better grasp of htdig would like to
use my situation as an example to perhaps help others, feel free. I would
be willing to generate and forward the rundig -vvvv output in dysfunctional
mode if it would be of help.
My particular situation involves moving a site from a different host. I am
doing all the preparation work prior to moving the site. The existing host,
is somedomain.com. I have a new site, somedomain.biz. I am getting
everything ready, including htdig prior to switching the dns over, so that
somedomain.com will answer up at the site hosting somedomain.biz.
During my troubleshooting, I checked and do not have a robots.txt file. I
also tried disabling my .htaccess file and restarting the server. Neither
helped. I also tried having htdig index both the the old site,
somedomain.com and the new. It did the old site, but not the new.
start_url: somedomain.com somedomain.biz
There are some URLS in files at the new site, (somedomain.biz) that have
references the old site URL (somedomain.com).
The circumstances to get it working:
This did not work. It indexes only the home page and ignores the rest of
This does not work either, it give a 403 Forbidden error.
But much to my surprise, if I include a specific filename in a
subdirectory, it will then index the entire site correctly:
I vaguely remember having the same issues several years ago on other
Get latest updates about Open Source Projects, Conferences and News.