I have noticed several instances where the system is still tries to index css documents if the site is using variables in the url.
Example domain.com/css/style.css?id=1.2
You seem to have CSS turned off. Please don't fill out this field.
Anonymous
Hi!
Yes, but what's the problem? WHat do you mean with "still"?
By default, phpcrawl tries to crawl everythinbg it finds. Just set a filter (or more filters) for the documents you don't want the crawler to receive.
I was having the same issue you just need to modify the Regular Expression;
$crawler->addURLFilterRule("#.(css|js)?*# i");
This will stop the query string and filter out anything that is .css?
There is a good doc on regular expressions for analytics but it just runs through basic regular expressions - http://www.lunametrics.com/regex-book/Regular-Expressions-Google-Analytics.pdf
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
I have noticed several instances where the system is still tries to index css documents if the site is using variables in the url.
Example
domain.com/css/style.css?id=1.2
Hi!
Yes, but what's the problem?
WHat do you mean with "still"?
By default, phpcrawl tries to crawl everythinbg it finds.
Just set a filter (or more filters) for the documents you don't want
the crawler to receive.
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
I was having the same issue you just need to modify the Regular Expression;
$crawler->addURLFilterRule("#.(css|js)?*# i");
This will stop the query string and filter out anything that is .css?
There is a good doc on regular expressions for analytics but it just runs through basic regular expressions - http://www.lunametrics.com/regex-book/Regular-Expressions-Google-Analytics.pdf