witme-ferris — Ferris development

Re: [Witme-ferris] Problem Compiling WebPhotosContext.cpp

[M. D. P. <m....@xm...>]

Hey Ben,

On Sun, 18 May 2008 05:27:35 -0600, Ben Martin  
<mon...@us...> wrote:
>
> Sounds very interesting.

We'll see where it goes, but I think it holds interesting potential.

> I also migrated the site of sf.net very recently. At some point that
> graphic didn't get linked to, so you might have loaded the page at a
> rough point in the migration.

In fact, looking at it in Safari now suggests this could very easily have  
been the case (it's showing up now.)

> Sorry about that :/

What's to be sorry about?  You've built a kick a$$ OSS system in  
libferris*.  A little bit of extra leg work and learning through trial and  
error on my end isn't exactly something that should be seen as an issue.  
;-)

> No worries. I have compiled against gcc 3.4x through recent 4.x versions
> and have a few flavours of ferris for maemo so it can be compiled for
> various distros... though I obviously have a significant advantage of
> knowing the code and dependencies implicitly.

True, though things seem to be pretty straight forward in /most/ cases.   
The troubles I have had thus far are mainly related to finding and linking  
the proper header files and what have you.  Actually, in this regard, a  
couple of questions,

*pccts: The only way I could get this portion of the compile process to  
succees was by tracking down the /old/ pccts source (1.33 I believe),  
building, and then pointing libferris as the proper header files before it  
was happy and decided to move on in the configure process.  I would have  
thought a recent release of antlr would have been adequate, but for some  
reason I couldn't get it to take.  Advice?

*Xerces: The configure script is hardcoded at Xerces 2.7.  The latest  
XQilla requires patches against Xerces 2.8 to compile correctly.  Should I  
move to a previous version of XQilla that uses Xerces 2.7 as its base, run  
the 2.8 patches against the 2.7 code base, or something else altogether?   
The time it takes to build each of these packages out is significant  
enough that I would rather gain your advice before pushing forward.

*STLPort: Are there any clear advantages to building everything againts  
STLPort at this point in time?

*FullTextSearch: No matter what I did, I couldn't get Xapian built in such  
a way that the configure script was satisfied enough to include support  
for full text search.  Any ideas what I should be doing differently?

-- 
/M:D

M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: m....@3r... | m....@am...
Mobile: (206) 999-0588
http://3rdandUrban.com | http://amp.fm |  
http://www.oreillynet.com/pub/au/2354

Re: [Witme-ferris] Problem Compiling WebPhotosContext.cpp

[Ben M. <mon...@us...>]

On Sat, 2008-05-17 at 06:03 -0600, M. David Peterson wrote:
> Hi Ben,
> 
> Thanks for the prompt reply!  As long as I can get things to build
> correctly, I have some fairly extensive plans to integrate libferris
> into the heart of a meta-data search/storage/retrieval system that
> integrates with Amazon SimpleDB.  I'm working directly with the Zorba
> team (see:
> http://www.oreillynet.com/xml/blog/2008/05/zorba_xquery_processor.html ) to utilize Zorba as the backend XQuery engine so at some point I'd like to chat with you about the processing of integrating Zorba with libferris.

Sounds very interesting.
> 
> In the mean time, however, more below,
> 
> On Sat, 17 May 2008 05:46:07 -0600, Ben Martin
> <mon...@us...> wrote:
> 
> > I assume you have read the build order page;
> > http://www.libferris.com/buildorder
> 
> I did, but for some reason the graphic doesn't show up inside of
> Safari.  I clicked on the link inside of Opera and to my surprise what
> I thought had ended with "Build dependencies for libferris and ego"
> was actually preceded by the build order graphic.  I'll start here and
> see where this now takes me.  Thanks!

I also migrated the site of sf.net very recently. At some point that
graphic didn't get linked to, so you might have loaded the page at a
rough point in the migration. Sorry about that :/
> 
> >> I'm compiling this on my own custom Linux build.
> >
> > I've got it compiled on four main distros of Linux so far. Though I
> > can't really test on a custom build ;-)
> 
> Well you can (I've got a VMware version that can be used) but it would
> be a bit much to expect you to take things to that extreme.  I'm happy
> to do all the leg work, so if you wouldn't mind answering a few
> questions now and again, we should be good to go.

No worries. I have compiled against gcc 3.4x through recent 4.x versions
and have a few flavours of ferris for maemo so it can be compiled for
various distros... though I obviously have a significant advantage of
knowing the code and dependencies implicitly.

I'm actually making a few more things optional now during libferris
compiles, which should make building easier. Though this is mainly handy
for embedded targets it should also lower the cost of hand compiles for
folks who are not too interested in having every feature compiled.
> 
> > Assuming that you have followed the buildorder can you send me the
> first
> > 10 error messages in full?
> 
> I'll run through the build order now and let you know the result.
> 
> Thanks for your help!

No problem.

Re: [Witme-ferris] Problem Compiling WebPhotosContext.cpp

[M. D. P. <m....@xm...>]

Hi Ben,

Thanks for the prompt reply!  As long as I can get things to build correctly, I have some fairly extensive plans to integrate libferris into the heart of a meta-data search/storage/retrieval system that integrates with Amazon SimpleDB.  I'm working directly with the Zorba team (see: http://www.oreillynet.com/xml/blog/2008/05/zorba_xquery_processor.html ) to utilize Zorba as the backend XQuery engine so at some point I'd like to chat with you about the processing of integrating Zorba with libferris.

In the mean time, however, more below,

On Sat, 17 May 2008 05:46:07 -0600, Ben Martin <mon...@us...> wrote:

> I assume you have read the build order page;
> http://www.libferris.com/buildorder

I did, but for some reason the graphic doesn't show up inside of Safari.  I clicked on the link inside of Opera and to my surprise what I thought had ended with "Build dependencies for libferris and ego" was actually preceded by the build order graphic.  I'll start here and see where this now takes me.  Thanks!

>> I'm compiling this on my own custom Linux build.
>
> I've got it compiled on four main distros of Linux so far. Though I
> can't really test on a custom build ;-)

Well you can (I've got a VMware version that can be used) but it would be a bit much to expect you to take things to that extreme.  I'm happy to do all the leg work, so if you wouldn't mind answering a few questions now and again, we should be good to go.

> Assuming that you have followed the buildorder can you send me the first
> 10 error messages in full?

I'll run through the build order now and let you know the result.

Thanks for your help!

-- 
/M:D

M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: m....@xm... | m....@am... | Mobile: (206) 999-0588
Web: http://amp.fm/ | http://www.oreillynet.com/pub/au/2354

Re: [Witme-ferris] Problem Compiling WebPhotosContext.cpp

[Ben M. <mon...@us...>]

On Sat, 2008-05-17 at 05:16 -0600, M. David Peterson wrote:
> Hi,
> 
> I'm guessing I'm missing a dependency of some sort, though I'm not
> sure what that might be.  The list of errors I am running into runs
> far and wide starting with "'fh_webPhotos' does not name a type"

I assume you have read the build order page;
http://www.libferris.com/buildorder
For something like fh_webPhotos not being a type it might be that
ferrisloki is not installed or is not being detected correctly, just off
the top of my head.
> 
> I'm compiling this on my own custom Linux build.

I've got it compiled on four main distros of Linux so far. Though I
can't really test on a custom build ;-)
> 
> Ideas?

Assuming that you have followed the buildorder can you send me the first
10 error messages in full?

[Witme-ferris] Problem Compiling WebPhotosContext.cpp

[M. D. P. <m....@xm...>]

Hi,

I'm guessing I'm missing a dependency of some sort, though I'm not sure what that might be.  The list of errors I am running into runs far and wide starting with "'fh_webPhotos' does not name a type"

I'm compiling this on my own custom Linux build.

Ideas?

-- 
/M:D

M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: m....@xm... | m....@am... | Mobile: (206) 999-0588
Web: http://amp.fm/ | http://www.oreillynet.com/pub/au/2354

Re: [Witme-ferris] Using ferris as a file characterization tool

[Ben M. <mon...@us...>]

On Tue, 2008-01-22 at 10:49 +0100, Lars Clausen wrote:
> Hi Ferrisians!

heh, this almost makes it seem like we are a community from Paris ;)

>=20
> As part of the Planets project (http://www.planets-project.eu) I'm
> investigating various tools for identifying and characterizing files.

A very interesting project. Thanks for the link.

> One of the tools suggested is Ferris, which is quite intriguing on many
> levels, but that the suggester hadn't actually tried.  I'd just like to
> let you know that I've successfully installed libferris using the
> xUbuntu deb files provided on a Ubuntu 7.10.  I had to install a number
> of extra libraries for it (I didn't manage to set up the apt-repository
> directly, so had to use dpkg), but now it works. =20

Did the apt repository fail to work or did you just decide to use dpkg?

> A few tests with the
> EAs mimetype (misspelled mime-type in the --help) and

Fixed. Thanks.

> mimetype-from-content shows that it can indeed be used for
> characterization to some extent.  The mimetype-from-content EA is a
> little funny in that things like directories, sockets and unreadable
> files all are called text/plain.

The system to actually work out the mimetype can be selected from a
number of implementations. The packages should be using GNOME's mimetype
detection so it must be reporting the types this way.

A keep throwing around an idea for how to do type detection using a more
formal deduction system like prolog but never get around to coding up
yet another detection engine but using this approach.

>=20
> I am very curious about the overall concept and will probably keep
> experimenting with it in various contexts, especially photography.=20

I am always expanding the support for things with digital pictures. I
recently updated some of the geotagging stuff to be more flexible. One
of the nice things about geotagging in libferris is that it is not file
type based: you can geolocate an office document the same way as an
image file. So notes, images, web pages and everything can be indexed
and overlaid on a map. For example using a bookmark network link in
google earth to get info from libferris.

Let me know how you go and happy ferrising!

[Witme-ferris] Using ferris as a file characterization tool

[Lars C. <lc...@st...>]

Hi Ferrisians!

As part of the Planets project (http://www.planets-project.eu) I'm
investigating various tools for identifying and characterizing files.
One of the tools suggested is Ferris, which is quite intriguing on many
levels, but that the suggester hadn't actually tried.  I'd just like to
let you know that I've successfully installed libferris using the
xUbuntu deb files provided on a Ubuntu 7.10.  I had to install a number
of extra libraries for it (I didn't manage to set up the apt-repository
directly, so had to use dpkg), but now it works.  A few tests with the
EAs mimetype (misspelled mime-type in the --help) and
mimetype-from-content shows that it can indeed be used for
characterization to some extent.  The mimetype-from-content EA is a
little funny in that things like directories, sockets and unreadable
files all are called text/plain.

I am very curious about the overall concept and will probably keep
experimenting with it in various contexts, especially photography. 

-Lars

Re: [Witme-ferris] Libferris-suite for Debian?

[Ben M. <mon...@us...>]

On Thu, 2007-09-06 at 09:44 +1000, Devraj Mukherjee wrote:
> Hi everyone,
>=20
> I am a newbie to libferris and am wondering if there are any binaries
> available for any Debian styled distributions?

I'm not aware of any.=20

I've built libferris in a debian Xen machine here but I don't know
enough about .deb files and what the various packages are called under
debian to make creating binaries easy :( It's sort of been on the todo
list for a long time.

[Witme-ferris] Libferris-suite for Debian?

[Devraj M. <de...@gm...>]

Hi everyone,

I am a newbie to libferris and am wondering if there are any binaries
available for any Debian styled distributions?

Thanks.

-- 
"I never look back darling, it distracts from the now", Edna Mode (The
Incredibles)

Re: [Witme-ferris] libferris and strigi (and hence kde4)

[Ben M. <mon...@us...>]

On Thu, 2007-06-14 at 08:45 +0200, Jos van den Oever wrote:

> After sending this mail, I saw you're going to Ottowa. That's a good
> conference with a high profile albeit very kernel focussed. Lacking
> the possibility to go the desktop conferences easily is definitely a
> problem. The reason the KDE people know about and like Strigi is
> mainly because I went to aKademy and explained to them that although
> Strigi is independent from KDE, it is very useful for getting at
> metadata and searching. Talking to many people in real life really
> makes a difference. Since the code became a dependency for kde, many
> people are helping with improving simply because many people are
> seeing the code.

It would definitely be cool to get to an OLS.

I do tend to get to LCAs sometimes as well. Though the last time I spoke
at one in a talk was 2002.

The problem with guadec/akademy is that they are both a fair distance
away and I don't have anyone other than the conference itself to foot
the air ticket :(

I know of some private companies and a few research groups using
libferris. Nothing the scale of having KDE pick it up as a dep though...
congratulations on that ;), good to see desktop/intranet search moving
along in the open src world.=20

One of the reasons that I think g/k will not be too chummy with
libferris is that I also wrote a VFS as part of the project. Handy for
me to use, but duplication of functionality for larger desktop projects.

>=20
> So my tips would be: 1) meet people 2) try to get big projects to rely
> on your code.
> Both are hard unfortunately.

The hardest part would be (3) get interested parties to somewhat offer
funds so that you can improve your code and thus their experience.
Plugins are something that can be contributed by users, but some of the
core indexing stuff is much less available to the casual weekender
hacker.


> The Xesam spec has a DBus interface that allows metadata retrieval.
> For desktop apps this is nice, but for indexing it is way too slow.
> Out of process is nice for stability, but also adds quite some
> overhead. We've found that even in process we are really stable. Not
> perfect but good. Calling 'xmlindexer' and parsing the output with a
> SAX parser is very easy, so you can do that.

I'd likely implement support in-process and use a deamon to handle
shelling the plugins into subprocesses based on config settings. I admit
that its fairly rare to have for example the xine plugin segv, but when
it does for a desktop app using in-proc its "less than optimal".

I'll take a look at the dbus spec, should save a bunch of needless
redesign :)

Re: [Witme-ferris] libferris and strigi (and hence kde4)

[Jos v. d. O. <jvd...@gm...>]

2007/6/14, Ben Martin <mon...@us...>:
> On Wed, 2007-06-13 at 22:04 +0200, Jos van den Oever wrote:
> > Hello Ben,
> >
> > Today I found out about libferris and was rather shocked. You have been
> > working on something that looks really cool since 2001. Why is this
> > project so unknown?
>
> Firstly, thanks && nice to meet you :)
>
> It's fairly difficult to "promote" libferris. Various conferences want
> to see a G or a K prefix or some definitive link to either of these
> desktops to make accepting a talk easier. Not living in Europe makes
> turning up at places like FOSDEM and other events impossible too
> unfortunately.
>
> But I'm all ears if you think there is another place I can talk about
> libferris to reach more folks.

After sending this mail, I saw you're going to Ottowa. That's a good
conference with a high profile albeit very kernel focussed. Lacking
the possibility to go the desktop conferences easily is definitely a
problem. The reason the KDE people know about and like Strigi is
mainly because I went to aKademy and explained to them that although
Strigi is independent from KDE, it is very useful for getting at
metadata and searching. Talking to many people in real life really
makes a difference. Since the code became a dependency for kde, many
people are helping with improving simply because many people are
seeing the code.

So my tips would be: 1) meet people 2) try to get big projects to rely
on your code.
Both are hard unfortunately.

> > I read large parts of the webpage just now and the features you mention
> > seem pretty impressive. Many of the the goals of you project are similar
> > to the ones from Strigi, a desktop search engine.
>
> I've been keeping an eye on Strigi myself. In fact my TODO list already
> contains an item to add support for index searching with strigi. I
> almost hacked it up for the 1.0.107 release but somehow it floated back
> down the list. But sometime soon I'd expect that to appear in a release.
>
> >
> > Let me briefly introduce Strigi to you. The coolest part of Strigi is the
> > way it handles files containing other files. For this is has a library
> > (libstreams [1]) that can very efficiently read e.g. a jpg embedded in a
> > pdf which is in a zip file that's attached to an email. The interface
> > for these streams is not the stl streams, but it can be wrapped in an
> > stl ostream / iterator combination.
>
> I really like the ability to poke around in an file inside an attachment
> of an Evolution email as a filesystem. I don't know why other VFS don't
> seem too keen on such ideas. The other embedded jpg example that I tend
> to use is for EXIF/thumbnail data, many systems when you plug in a
> camera will download the image to thumbnail it on the machine instead of
> downloading just the embedded thumbnail.

Exactly. The number of uses is huge. The cost is that your VFS has to
support streaming very efficiently to make deeper embedding feasible,
but it sure beats needing to e.g. extract an archive to disk to look
inside it.

> > In Strigi, data is analyzed from these efficient streams and in KDE4
> > already many analyzers are available that use this interface. Most of
> > them have very little dependencies. For analyzing a file, it has to be
> > read at most once, yet multiple analyzers can be applied to it. Since
> > 90% of all analysis is stream based, it is fast and has low memory
> > overhead. Strigi is a dependency of the upcoming KDE4 and will be used
> > to get at metadata and do file searching.
> >
> > The rest of Strigi, i.e. the actual indexing is designed rather nicely
> > and allows for different backends, like libferris does. (here i was
> > thinking this was a unique feature ...)
>
> The coolest part about having multiple back ends I have found is having
> a repeater for yourself as a backend. This makes it really easy to have
> multiple indexes of different formats joined together into a single
> "logical" index. Its also really nice to be able to have strigi
> optionally use libferris for indexing and vice versa.
Yes, this is a big boon. Our current reflector class
CombinedIndexManager is this too, albeit not perfect yet. This is also
a hard problems given the need to combine returned results and dealing
with differing latencies for backends.

> > Strigi comes with a tool called 'xmlindexer' that gives out all the
> > data the analyzers extract as xml. We are currently working with 5
> > desktop search engines on a freedesktop.org standard [2] for an extended
> > attributes ontology and a common query language. Strigi will adhere to
> > this standard once it is ready.
>
> I'm sort of not involved much with fdo. Thanks for pointing this out
> though as I might well peek into see what the EA ontology ends up being
> like. I tend to just type my EA with an extension of the basic XSD data
> types and give them obvious names. The type system is of course
> extensible and you can set types the same way as you set EA themselves.
>
> I would have voted for reusing SPARQL for a common query language. In
> fact you can do this with libferris now if you use the redland backend
> to index your metadata. I lean more towards using XQuery these days. I
> implemented the ability to run XQuery directly against libferris earlier
> in the year and also against its indexes. I'll most likely extend this
> interface in the future.

SPARQL and XQuery are very nice technologies. The Xesam effort is
however a consensus spec from five different search engines and none
of them was based on XML for querying. Also people found that full
text search in XPARQL and XQuery was lacking.

The Nepomuk project which goes beyond the scope of simple file search
and has an ontology that is not only about file attributes does use
SPARQL.

> > Feel free to have a look at the project [3] and I hope we can work
> > together and share ideas and code. Maybe a first step could be an
> > adapter of xmlindexer to libferris or one that uses libstreamanalyzer
> > directly. I'm usually on irc in #strigi.
>
> Sounds good. I'll take anther look at strigi and play around with
> libstreamanalyzer sometime.
>
> An idea from my TODO list is having the ability to run these extractors
> out-of-process. This is because for example the xine extractor sometimes
> segvs inside xine and I really want to isolate that (from the app and
> other plugins) to allow the application to not care which extractor has
> crashed etc. I've been thinking of maybe using dbus to talk to the
> daemon which would also open things up to mixing strigi/libferris/other
> extractors arbitrarily.

The Xesam spec has a DBus interface that allows metadata retrieval.
For desktop apps this is nice, but for indexing it is way too slow.
Out of process is nice for stability, but also adds quite some
overhead. We've found that even in process we are really stable. Not
perfect but good. Calling 'xmlindexer' and parsing the output with a
SAX parser is very easy, so you can do that.

> Unfortunately I've been a little bogged down with rounding out the PhD
> so haven't been able to hack these TODO items into code... yet.
>
> I sometimes hang out in #ferris, though i've rarely been on IRC of late.
>
> I'll be sure to drop into #strigi after I play with the code a little
> more :)

Cheers,
Jos

Re: [Witme-ferris] error linking libferris

[Jos v. d. O. <jvd...@gm...>]

2007/6/14, Ben Martin <mon...@us...>:
> On Thu, 2007-06-14 at 01:04 +0200, Jos van den Oever wrote:
> > Hi Ben,
> >
> > When linking libferris, I get this error:
> >
> > ferris-import-desktop-file.o: In function `main':
> > /home/ferris/src/libferris-1.1.107/apps/importdesktop/ferris-import-desktop-file.cpp:100:
> > undefined reference to
> > `Ferris::Shell::acquireContext(std::basic_string<char,
> > std::char_traits<char>, std::allocator<char> >, int, bool,
> > SigC::Signal1<void, Loki::SmartPtr<Ferris::Context,
> > Ferris::FerrisExRefCountedContext, Loki::DisallowConversion,
> > FerrisLoki::FerrisExSmartPointerChecker,
> > FerrisLoki::FerrisExSmartPtrStorage>, SigC::Marshal<void> >&)'
> >
> > The symbol acquireContext is present in Ferris/.libs/libferris.so, so
> > I'm not quite sure why it is not picked up.
> >
> > Cheers,
> > Jos
>
> This is indeed very strange. I just did a
> make clean; make;
> in apps/importdesktop and all seems to be fine here.
>
> Do other apps also fail to link when you do a make -k  ?

Yes, all apps fail, this is just the first one.
I'm on ubuntu feisty with gcc 4.1.2.

I'll have another look at it tonight.

Cheers, Jos

Re: [Witme-ferris] error linking libferris

[Ben M. <mon...@us...>]

On Thu, 2007-06-14 at 01:04 +0200, Jos van den Oever wrote:
> Hi Ben,
>=20
> When linking libferris, I get this error:
>=20
> ferris-import-desktop-file.o: In function `main':
> /home/ferris/src/libferris-1.1.107/apps/importdesktop/ferris-import-deskt=
op-file.cpp:100:
> undefined reference to
> `Ferris::Shell::acquireContext(std::basic_string<char,
> std::char_traits<char>, std::allocator<char> >, int, bool,
> SigC::Signal1<void, Loki::SmartPtr<Ferris::Context,
> Ferris::FerrisExRefCountedContext, Loki::DisallowConversion,
> FerrisLoki::FerrisExSmartPointerChecker,
> FerrisLoki::FerrisExSmartPtrStorage>, SigC::Marshal<void> >&)'
>=20
> The symbol acquireContext is present in Ferris/.libs/libferris.so, so
> I'm not quite sure why it is not picked up.
>=20
> Cheers,
> Jos

This is indeed very strange. I just did a=20
make clean; make;=20
in apps/importdesktop and all seems to be fine here.=20

Do other apps also fail to link when you do a make -k  ?

Re: [Witme-ferris] libferris and strigi (and hence kde4)

[Ben M. <mon...@us...>]

On Wed, 2007-06-13 at 22:04 +0200, Jos van den Oever wrote:
> Hello Ben,
>=20
> Today I found out about libferris and was rather shocked. You have been
> working on something that looks really cool since 2001. Why is this
> project so unknown?

Firstly, thanks && nice to meet you :)

It's fairly difficult to "promote" libferris. Various conferences want
to see a G or a K prefix or some definitive link to either of these
desktops to make accepting a talk easier. Not living in Europe makes
turning up at places like FOSDEM and other events impossible too
unfortunately.

But I'm all ears if you think there is another place I can talk about
libferris to reach more folks.

>=20
> I read large parts of the webpage just now and the features you mention
> seem pretty impressive. Many of the the goals of you project are similar
> to the ones from Strigi, a desktop search engine.

I've been keeping an eye on Strigi myself. In fact my TODO list already
contains an item to add support for index searching with strigi. I
almost hacked it up for the 1.0.107 release but somehow it floated back
down the list. But sometime soon I'd expect that to appear in a release.

>=20
> Let me briefly introduce Strigi to you. The coolest part of Strigi is the
> way it handles files containing other files. For this is has a library
> (libstreams [1]) that can very efficiently read e.g. a jpg embedded in a
> pdf which is in a zip file that's attached to an email. The interface
> for these streams is not the stl streams, but it can be wrapped in an
> stl ostream / iterator combination.

I really like the ability to poke around in an file inside an attachment
of an Evolution email as a filesystem. I don't know why other VFS don't
seem too keen on such ideas. The other embedded jpg example that I tend
to use is for EXIF/thumbnail data, many systems when you plug in a
camera will download the image to thumbnail it on the machine instead of
downloading just the embedded thumbnail.

>=20
> In Strigi, data is analyzed from these efficient streams and in KDE4
> already many analyzers are available that use this interface. Most of
> them have very little dependencies. For analyzing a file, it has to be
> read at most once, yet multiple analyzers can be applied to it. Since
> 90% of all analysis is stream based, it is fast and has low memory
> overhead. Strigi is a dependency of the upcoming KDE4 and will be used
> to get at metadata and do file searching.
>=20
> The rest of Strigi, i.e. the actual indexing is designed rather nicely
> and allows for different backends, like libferris does. (here i was
> thinking this was a unique feature ...)

The coolest part about having multiple back ends I have found is having
a repeater for yourself as a backend. This makes it really easy to have
multiple indexes of different formats joined together into a single
"logical" index. Its also really nice to be able to have strigi
optionally use libferris for indexing and vice versa.

>=20
> Strigi comes with a tool called 'xmlindexer' that gives out all the
> data the analyzers extract as xml. We are currently working with 5
> desktop search engines on a freedesktop.org standard [2] for an extended
> attributes ontology and a common query language. Strigi will adhere to
> this standard once it is ready.

I'm sort of not involved much with fdo. Thanks for pointing this out
though as I might well peek into see what the EA ontology ends up being
like. I tend to just type my EA with an extension of the basic XSD data
types and give them obvious names. The type system is of course
extensible and you can set types the same way as you set EA themselves.

I would have voted for reusing SPARQL for a common query language. In
fact you can do this with libferris now if you use the redland backend
to index your metadata. I lean more towards using XQuery these days. I
implemented the ability to run XQuery directly against libferris earlier
in the year and also against its indexes. I'll most likely extend this
interface in the future.

>=20
> Feel free to have a look at the project [3] and I hope we can work
> together and share ideas and code. Maybe a first step could be an
> adapter of xmlindexer to libferris or one that uses libstreamanalyzer
> directly. I'm usually on irc in #strigi.

Sounds good. I'll take anther look at strigi and play around with
libstreamanalyzer sometime.

An idea from my TODO list is having the ability to run these extractors
out-of-process. This is because for example the xine extractor sometimes
segvs inside xine and I really want to isolate that (from the app and
other plugins) to allow the application to not care which extractor has
crashed etc. I've been thinking of maybe using dbus to talk to the
daemon which would also open things up to mixing strigi/libferris/other
extractors arbitrarily.

Unfortunately I've been a little bogged down with rounding out the PhD
so haven't been able to hack these TODO items into code... yet.

I sometimes hang out in #ferris, though i've rarely been on IRC of late.

I'll be sure to drop into #strigi after I play with the code a little
more :)

>=20
> Best regards,
> Jos van den Oever
>=20
> [1]
> http://www.kde.org/kdeslides/fosdem2007/2007-02-vandenoever-strigi_intern=
als.pdf
> [2] http://freedesktop.org/wiki/XesamAbout
> [2] http://strigi.sf.net
>=20
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Witme-ferris mailing list
> Wit...@li...
> https://lists.sourceforge.net/lists/listinfo/witme-ferris

[Witme-ferris] error linking libferris

[Jos v. d. O. <jvd...@gm...>]

Hi Ben,

When linking libferris, I get this error:

ferris-import-desktop-file.o: In function `main':
/home/ferris/src/libferris-1.1.107/apps/importdesktop/ferris-import-desktop-file.cpp:100:
undefined reference to
`Ferris::Shell::acquireContext(std::basic_string<char,
std::char_traits<char>, std::allocator<char> >, int, bool,
SigC::Signal1<void, Loki::SmartPtr<Ferris::Context,
Ferris::FerrisExRefCountedContext, Loki::DisallowConversion,
FerrisLoki::FerrisExSmartPointerChecker,
FerrisLoki::FerrisExSmartPtrStorage>, SigC::Marshal<void> >&)'

The symbol acquireContext is present in Ferris/.libs/libferris.so, so
I'm not quite sure why it is not picked up.

Cheers,
Jos

[Witme-ferris] libferris and strigi (and hence kde4)

[Jos v. d. O. <jvd...@gm...>]

Hello Ben,

Today I found out about libferris and was rather shocked. You have been
working on something that looks really cool since 2001. Why is this
project so unknown?

I read large parts of the webpage just now and the features you mention
seem pretty impressive. Many of the the goals of you project are similar
to the ones from Strigi, a desktop search engine.

Let me briefly introduce Strigi to you. The coolest part of Strigi is the
way it handles files containing other files. For this is has a library
(libstreams [1]) that can very efficiently read e.g. a jpg embedded in a
pdf which is in a zip file that's attached to an email. The interface
for these streams is not the stl streams, but it can be wrapped in an
stl ostream / iterator combination.

In Strigi, data is analyzed from these efficient streams and in KDE4
already many analyzers are available that use this interface. Most of
them have very little dependencies. For analyzing a file, it has to be
read at most once, yet multiple analyzers can be applied to it. Since
90% of all analysis is stream based, it is fast and has low memory
overhead. Strigi is a dependency of the upcoming KDE4 and will be used
to get at metadata and do file searching.

The rest of Strigi, i.e. the actual indexing is designed rather nicely
and allows for different backends, like libferris does. (here i was
thinking this was a unique feature ...)

Strigi comes with a tool called 'xmlindexer' that gives out all the
data the analyzers extract as xml. We are currently working with 5
desktop search engines on a freedesktop.org standard [2] for an extended
attributes ontology and a common query language. Strigi will adhere to
this standard once it is ready.

Feel free to have a look at the project [3] and I hope we can work
together and share ideas and code. Maybe a first step could be an
adapter of xmlindexer to libferris or one that uses libstreamanalyzer
directly. I'm usually on irc in #strigi.

Best regards,
Jos van den Oever

[1]
http://www.kde.org/kdeslides/fosdem2007/2007-02-vandenoever-strigi_internals.pdf
[2] http://freedesktop.org/wiki/XesamAbout
[2] http://strigi.sf.net

[Witme-ferris] The list is now moderated for non member posts

[Ben M. <mon...@us...>]

Hi,
  I finally dug up the option to moderate posts from non members by
default. So from now on the only one who gets to enjoy the SPAM that is
hitting the list is me :/

Re: [Witme-ferris] Thanks for your recent refinance application, we are ready to lend some cash regardless of Credit

[DIP S. <zyb...@as...>]

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Diso-8859-1">
<META content=3D"MSHTML 6.00.2900.2912" name=3D"GENERATOR">
</HEAD>
<BODY>
<DIV align=3Dleft><FONT face=3DArial size=3D2><B>Your credit history does not matter to us!</B></FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2>If you OWN real estate and want IMMEDIATE ready money to spend ANY way you like, or simply need to LOWER your current payments by a third or more, here is best deal we can offer you THIS NIGHT (hurry, this offer will expire NOW):</FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2><B>$247,000+ loan</B></FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2>AND EVEN MORE: After further review, our lenders have established the lowest payments!</FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2><B>Hurry, when best deal is gone, it is gone. Simply fill out this easy form... </B></FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2>Don't worry about approval, your credit will not disqualify you!</FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2><a href=3D"http://skelaenyard.com/">http://skelaenyard.com/</a></FONT></DIV></BODY></HTML>

[Witme-ferris] =?gb2312?B?1+7QwkFSTTkvTGludXgvV2luZG93c0NFxeDRtdDFz6Iotdo4MMbao6kh?=

[ǶÈëʽÅàѵ£¡ <jy...@go...>]

xeDRtbi9y81BUk05v6q3oszXvP678kFSTcirz7XB0LfC1ebG9yENCg0KDQotLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQrOorXP1+7QwkFSTTkvTGlu
dXgvV2luZG93c0NFx7bI68q9xeDRtdDFz6Iotdo4MMbao6khIA0KLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQogxeDRtdb3zOIgo7pXaW5kb3dz
IENFLk5FVM+1zbO8sNOm08O/qreixeDRtSANCiDF4NG1z+rH6SCjumh0dHA6Ly93d3cubWRldnNv
ZnQuY29tL2NuL0FydGljbGVfU3BlY2lhbExpc3RfYTAwMi5hc3AgDQogxeDRtcqxvOQgo7ogICAg
IA0KICAgICAgICAgye7b2iCjuiAyMDA3xOo11MIxOS0yMLrFo6i12jIxxtqjqSANCiAgICAgICAg
IMnPuqMgo7ogMjAwN8TqNdTCMTItMTO6xaOotdoyMcbao6kgDQogICAgICAgICCxsb6pIKO6IDIw
MDfE6jXUwjI2LTI3usWjqLXaMjHG2qOpDQogDQogxeDRtdb3zOIgo7pBUk05L2xpbnV4x7bI68q9
v6q3osnovMbF4NG1o6i4vcvNQVJNOb+qt6LM17z+u/JBUk23wtXmxvejqSANCiDF4NG1z+rH6SCj
umh0dHA6Ly93d3cubWRldnNvZnQuY29tL2NuL0FydGljbGVfU3BlY2lhbExpc3RfYTAwMS5hc3Ag
DQogxeDRtcqxvOQgo7ogICAgICAgICAgICAgICANCiAgICAgICAgIMnu29ogo7oyMDA3xOo11MIx
OS0yNLrFICAgo6i12jgwxtqjqSANCiAgICAgICAgIMnPuqMgo7oyMDA3xOo11MIxNC0xN7rFICAg
o6i12jgwxtqjqSANCiAgICAgICAgILGxvqkgo7oyMDA3xOo11MIyOC0zMbrFICAgo6i12jgwxtqj
qSANCg0KIMXg0bW12LXjIKO6DQogICAgICAgICDJ7tvaIKO6ye7b2sTPyb2/xrS01tDQxCjOorXP
ye7b2sXg0bW7+bXYICkgDQogICAgICAgICDJz7qjIKO6yc+6o8bWtqvI7bz+1LDF4NG11tDQxDG6
xcKlM8KlKM6itc/Jz7qjxeDRtbv5tdggKSANCiAgICAgICAgILGxvqkgo7q6o7Xt0MXPosK3MzO6
xTUwOcrSKM6itc+xsb6pxeDRtbv5tdgpIA0KDQoguavLvtb30rMgo7ogd3d3Lm1kZXZzb2Z0LmNv
bS9jbiANCiDF4NG108rP5CCjuiBzYWxlc0BtZGV2c29mdC5jb20NCiDF4NG1yMjP3yCjuiAwNzU1
LTgxMjgzOTMxICAgIA0KICAgwaogz7UgyMsgo7ogu8bPyMn6ICAgDQoNCi0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LSANCtLUyc/Oqrvh1LHQxc+io6y4+MTjtPjAtLK7sePJ7rHtx7jS4qOszcu2qcfru9i4tLW9o7po
dW5hbnNsQHRvbS5jb20g0LvQuyENCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ==

[Witme-ferris] Immense gains in germany.

[Fatbardha s. <sim...@9t...>]

http://img148.imageshack.us/img148/7416/wx4dc8w5gx8.gif

German excnahges are hot.

Ian's ability at telling stories of his heroics was legendary in the Vacuum.

[Witme-ferris] Check out the wonders of pound melting

[Nikki S. <dof...@co...>]

Anatrim – The latest and most exciting product for over-weight people is now easily available – As were told on CNN
Do you understand that superfluous body kilos kill a lot of people around the planet? We know that you hate the unsightly look of those people and the low status they have in modern society. 

Or, maybe, you can’t resist a siege of your terrible eating habits. If you see it sounds familiar, then we got something to propose to you!

We are proudly introduce you Anatrim, the new product for the elimination of your body’s extra weight. The astonishing thing about Anatrim is it improves the quality of your life by soothing your feeling of hunger and giving you better cheer. Here you may see some notes received from our customers:

"This is a great thing! 

Instead of gorging everything close at hand and watching TV constantly 

I became keen on doing exercise.

 Anatrim made me stride on confidently.

 I have a great figure now and there are lots of men following me with their eyes!"

Victoria K., Bellevue WA

"I tried some passive weight losing, but to no purpose. This terrible hunger would just rush in and spoil everything. 

One day I heard on Anatrim in the commercial and I was really impressed at the information.

 I had attempted taking it, and my wife told me I had changed for the better since then, 3 months later. 

30 pounds off and I continue to become slimmer! 

And you know, it’s rather hot in our bedroom now."

Charley Mock, Bellevue WA

Anatrim helps your mind to understand that your body doesn’t need that much food. It raises your spirit, supplies you with energy, and attacks needless kilos. All thanks to its mighty newly-elaborated formula!!
Worm out latest intelligence! 

[Witme-ferris] Next Big market Winner

[Offices D. <hha...@pl...>]

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Diso-8859-1">
<META content=3D"MSHTML 6.00.2900.2912" name=3D"GENERATOR">
</HEAD>
<BODY>
<DIV align=3Dleft><FONT face=3DArial size=3D2>GDKI IS GETTING READY FOR ANOTHER HUGE BREAKDANCE MONDAY!</FONT></DIV>
<DIV align=3Dleft><FONT face=3DArial size=3D2>HIP-HOP is more than just music... HIP-HOP IS A CULTURE!</FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2><I><U>GOLDMARK IND - www [dot] goldmarkentertainment [dot] com</U></I></FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2>CHECK *GDKI* OUT ON - MONDAY FEB 05 2007</FONT></DIV>
<DIV align=3Dleft><FONT face=3DArial size=3D2>RADAR: <B>GDKI</B> </FONT></DIV>
<DIV align=3Dleft><FONT face=3DArial size=3D2>SHARES OPEN MON AT: <B>$0.105</B> </FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2>GOLDMARK INDUSTRIES, specializes in the production and distribution of Music, Feature Films and Television entertainment for North America's most rapidly growing demographic, with a total consumer-based purchasing power of over 1 Trillion dollars: the Hip-Hop community.</FONT></DIV><BR>
<DIV align=3Dleft><FONT face=3DArial size=3D2><U><B>GDKI THE RISING STAR, IS SET FOR SUPERNOVA STATUS ON 03/05/07!</B></U></FONT></DIV>
<DIV align=3Dleft><FONT face=3DArial size=3D2><U><B>HABANA.. HABANA.. HABANA BLUES!!</B></U></FONT></DIV></BODY></HTML>

[Witme-ferris] good dogat

[Evstathios D. <ch...@ba...>]

Hi,

Chaepest CALIS and VAGRA online.

http://parkerokay+now.com

Remove "+" from the link.


youre doing? 
And then Harry heard Krums voice.
Crucio! 

[Witme-ferris] Your Call Centre in China and Hong Kong

[<cnf...@ch...>]

If you are setting up a Call Centre in Hong Kong,
our company can provide:

  1.  Dedicated Hong Kong Fax Line (US$20.00 per month) 
      Unlimited fax receiving and every incoming fax will be 
      converted into TIFF file and forwarded to your email address. 

  2.  Dedicated Hong Kong Phone Line (US$20.00 per month + IDD Charges)
      Every incoming call will be forwarded to your company phone number.

We also provide Dedicated Fax Line and Phone Line services in other China
cities such as Shanghai and ShenZhen

Details please email cnf...@ch... or browse our webpage on
http://www.china.mbfax.com.

Man Bond Communications Limited
Block F1, 13/F., Tuen Mun Industrial Centre
No. 76, Pui To Road, Tuen Mun, Hong Kong

Tel: 852-2780-3257
Fax: 852-8343-7020
Email: in...@ch...
Homepage: http://www.china.mbfax.com       

*********************************************************************
If you wish to be removed from our list, please return our email with
Removal at the Subject: line.
********************************************************************* 

Re: [Witme-ferris] RlotX

[Apolline B. <ga...@ha...>]

Hi,

Vivagra  3. 35
Ciavlis  3. 75
Valvium  1. 25
Sovma    1. 15
Ambvien  2. 90

http://www.zonr*x.com

Important: Remove "*" in the above link



after I plucked his tail. Twelve and a quarter inches... ash...
pleasantly springy. Its in fine condition... You treat it regularly? 
Polished it last night,  said Cedric, grinning.