Hi guys,
Can you help me, I need to avoid crawl some url but I dont understand so much about addURLFilterRule
lets say that I dont want to crawl url that begins with http://somedomain.com/blablabla/buuuuu ...
is something like this?
$crawler->addURLFilterRule("^http://somedomain.com/blablabla/"); only with ^ and nothing else in the end?
Thanks for your help!!!!
You seem to have CSS turned off. Please don't fill out this field.
Anonymous
Hi, It depends on what all of your blabla really is, and what you're really trying to filter out. it's regular expression based, so you can be as specific or general as you need to be.
This might be a good place to start: http://www.regular-expressions.info
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Hi guys,
Can you help me, I need to avoid crawl some url but I dont understand so much about addURLFilterRule
lets say that I dont want to crawl url that begins with http://somedomain.com/blablabla/buuuuu ...
is something like this?
$crawler->addURLFilterRule("^http://somedomain.com/blablabla/"); only with ^ and nothing else in the end?
Thanks for your help!!!!
Hi,
It depends on what all of your blabla really is, and what you're really trying to filter out. it's regular expression based, so you can be as specific or general as you need to be.
This might be a good place to start:
http://www.regular-expressions.info