Es begab sich aber da Kevin C. Bombardier zur rechten Zeit 26.01.2007 03:21 folgendes schrieb:

Is there a genral programmers guide (formal or informal) on how to add a new datasource (compressed files) and crawler ((s)ftp)?


I thought I would ask since the documentation I have read through so far has been excellent and someone may have already written this or have an old email on how to do it.

Thanks for pointing that out, that is nice to hear!

I think we don't have documentation about this, but I may be wrong.
Please contact Antoni Mylka on doing that, he programmed the iCal crawler and can point you to the right things.


I would like to add (s)ftp but I am not sure if that falls under a new one from scratch (and wanted to make sure I knew all the areas to work in to develop it correctly) or would it be part of extending the web one (which does http(s) already).  I feel it is the former.

well, sftp is different from http, because of the clear user/password you need to login,
and also crawling is trivial, just traverse all directories.
Whereas the web crawler does interpret the HTML files and extracts links to find more for crawling

I would look into the filesystem crawler, as a reference, and start from scratch.
Chris Fluit probably has a better idea how to do the sftp...







------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash

_______________________________________________ Aperture-devel mailing list

- DFKI bravely goes where no man has gone before -
We will move to our new building by end of February 2007.

The new address will be as follows:
    Trippstadter Straße 122
    D-67663 Kaiserslautern

My phone/fax numbers will also change:
Phone:    +49 (0)631 20575 - 116
Secr.:    +49 (0)631 20575 - 101
Fax:      +49 (0)631 20575 - 102
Email remains the same
DI Leo Sauermann 
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Trippstadter Strasse 122
P.O. Box 2080          Fon:   +49 631 205-3503
D-67663 Kaiserslautern Fax:   +49 631 205-3472
Germany                Mail:
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff

Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes

Amtsgericht Kaiserslautern, HRB 2313