I run a purely .php web site and am trying to set up
htdig ver. 3.1.6 to index my site. I did the process as
explained with the version, did the standard compile, but
it does not index the site and strangely is returning .html
files only as results. In the Document.cc I have set it to
look for .php only and I have no .html files on this server
so I am confused as to where it is finding those.
Secondly it keeps looking for robots.txt. I have now read
the info on your site concerning that file, its format,
semantics etc. Does it have to be at the root? Does it
have a "proper" location that would effect the
performance of htdig if it is not there or in the wrong
Lastly, are there any hidden sub-directories not
identified in the source code that could be effecting the
performance of htdig indexing my site? I want to make
sure I have checked all locations for possible errors.
Log in to post a comment.