Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#161 robots.txt: clfmerged log files maybe not start with /

open
nobody
None
5
2014-12-04
2009-01-11
Che, Dong
No

I collecting log files from many servers and merged with clfmerge, for a pan-domain application: the robots access record might start with domain name
"GET http://foo.bar.com/robots.txt 200 14"

so I tried to remove the rule: must start with /
10385c10385
< my $regrobot = qr/^\/robots.txt$/i;


  my $regrobot       = qr/\/robots\.txt$/i;

Discussion

  • Added in CVS tree for next version.