Database Crawler

Help
2010-11-18
2012-09-13
  • Richard SINELLE

    Richard SINELLE - 2010-11-18

    Hi

    I have just create a database Crawler.

    How is it possible to launch it for example every 30 minutes ?

    Thanks

    Richard

     
  • Emmanuel Keller

    Emmanuel Keller - 2010-11-19

    Hi Richard,

    There is an HTTP API for that. You can launch any database crawl by using this
    call:

    [url]http://localhost:8080/database?use=indexname&name=dbcrawlername[/url]
    

    Replace indexname and dbcrawlername with your values.

    If you have activated the privileges, you have to add the login and the (api)
    key parameters:

    [url]http://localhost:8080/database?use=indexname&name=dbcrawlername&login=admin&key=29b4317cfc9fa7bf08ca26ae42ee4f66[/url]
    

    If you are on a Unix/Linux environment, you can then add a crontab entry to
    automate the job:

    */30 * * * * wget [url]http://localhost:8080/database?use=indexname&name=dbcrawlername[/url]
    

    Regards,

    Emmanuel

     
  • Richard SINELLE

    Richard SINELLE - 2010-11-19

    Thanks Emmanuel for your quickly answer.

    I'm on Windows plateform.

    If I want to add a scheduler in OSS (like for the web crawler), how can I
    start to study this add on ?

     
  • Richard SINELLE

    Richard SINELLE - 2010-11-22

    Thanks for your answer.

    For my project, it will not possible to use an external scheduler..... no I
    don't want to use an external scheduler because I will have 25 index with for
    each a Database crawler. So I prefer to have the possibility to launch or stop
    the crawler with a web interface.

    For information, I just to finish to update the source code in order to add
    this crawler. Are you interested for a patch file ?

    Richard

     
  • Emmanuel Keller

    Emmanuel Keller - 2010-11-24

    We are always happy to integrate valuables contributions !

    If you want to do that, you can create a related artifact in the new features
    tracker and attach your patch.

    http://sourceforge.net/tracker/?group_id=260863&atid=1127400

    Regards,

    Emmanuel.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks