Has anyone seen where htdig (appearntly) is ignoring robots.txt.    
 
 

Collin Gayde
Systems Programmer (Unix)
Office of Information Technology
Indiana State University
collin@indstate.edu