Menu

#82 Filter Rule does not recognise .min.css/.min.js

closed-works-for-me
None
5
2015-09-06
2014-12-24
Anonymous
No

When adding a addURLFilterRule() for .css or .js any .min.js and .min.css files are ignored. Even when adding .min.css or .min.js files to the filter rule they are still crawled causing issues.

Discussion

  • Anonymous

    Anonymous - 2014-12-28

    Hi!

    Do you have an actual example (your complete filter-rules and the site you are crawling)?

     
  • Uwe Hunfeld

    Uwe Hunfeld - 2015-01-06

    Hi!

    Again: ANy examples (URL) on this?
    Not able to reproduce this without any concrete example.

     
  • Uwe Hunfeld

    Uwe Hunfeld - 2015-09-06

    Just can't confirm this, after some tests addURLFilterRule() works as expected in all cases.

    I.e:
    If you set $crawler->addURLFilterRule("#.(css|js)[\?$]# i"), ALL URLs edning with ".css" or ".js" get ignored by the crawler, so "xxx.min.js" for example too.

    I'm closing this for now.

    If someone still has problemss with addURLFilterRule(), please report back.

    THANKS!

     
  • Uwe Hunfeld

    Uwe Hunfeld - 2015-09-06
    • status: open --> closed-works-for-me
     

Anonymous
Anonymous

Add attachments
Cancel





MongoDB Logo MongoDB