Re: [Mailmanager-developers] Full text searching in MailManager with Xapian

[Andrew V. <an...@lo...>]

I think we should check this out:

http://divmod.org/projects/quotient

This is a Python/Twisted/Xapian product for personal email  
management, I'm sure it would be a good example of Xapian integration.

A

On 28 Mar 2006, at 13:12, James Gray wrote:

> I thought I'd share what I've found out about Xapian a possible  
> solution to
> better full text searching in MailManager.
>
> Xapian is an open source probabilistic information retrieval libary.
>
>     * written in C++ but has Python bindings.
>
> Highly scalable used in many big content sharing open source  
> websites (Gmane
> the mailing list archive for example).
>
> A system utilising Xapian typically has two parts:
>
>     * Indexer: takes text and processes it for efficent searching
>     * Searcher: accepts queries, reads the database and actually  
> returning the
>       relevant information/
>
> Clearly then,
>
>     Indexer <- database -> Searcher
>
> Database needs to be tailored for specific need. Is high performance a
> priority or should information be added to the database  
> immediately? etc. This
> needs to be formalised. Xapian uses its own database backend for  
> efficient
> searching - it can't work from any SQL database - although it has  
> different
> backends to accodomate different needs.
>
> Therefore, in order for it to work with MailManager the SQL  
> database would
> have to be 'dumped' to the Xapian format every so often (perhaps  
> via a cron
> job) there's a script supplied with an Xapian application Omega  
> that does
> this (dbi2xapian) - as it stands it only works with MySQL but that  
> would
> be trivial to change (shouldn't be much more that changing the  
> connection
> string in fact since it uses Perl's DBI).
>
> I'm not sure how practical this will be to use though. I imagine  
> dumping the
> database every so often will be pracitcally prohibitve in terms of  
> processing.
> Also, it's bumping up the dependencies required for installaiton of
> MailManager.
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting  
> language
> that extends applications into web and mobile media. Attend the  
> live webcast
> and join the prime developer group breaking into this new coding  
> territory!
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Mailmanager-developers mailing list
> Mai...@li...
> https://lists.sourceforge.net/lists/listinfo/mailmanager-developers