Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#118 Advice please on extracting embedded files from RTF docs

1.6.0 - features
open
nobody
None
5
2014-03-25
2014-03-21
Chris Bamford
No

Hi

I would like to extract embedded files from RTF docs. Loking through the codebase I have found an example on how to extract text, but nothing more.
Please can you point me in the right direction?

Thanks!

  • Chris

Related

Feature Requests: #118

Discussion

  • Antoni Mylka
    Antoni Mylka
    2014-03-22

    You'd need a Subcrawler for the RTF format. There is none in Aperture, so unfortunately there is no out-of-the-box support for extracting embedded files in RTF docs.

    Note that the last release of Aperture was in December 2011. Since then the codebase hasn't been actively developed.

     
  • P Foomer
    P Foomer
    2014-03-22

    I did this (media wiki crawler) by copying an existing source (web crawler), and calling the library to extract the wiki data elements.

    I would at a guess look at copying the existing RTF extractor and making a new extractor , perhaps using tika, or modify the existing rtf extractor ( probably easier as the aperture model is quite convoluted without a road map) again possibly using tika.

    There were some discussions about using tika instead of aperture, but aperture does two things. extract and then do the RDF stuff (I think!!) where as tika does only the first.

    I feel that in an ideal world the extraction should be divorced from aperture, but at the time tika wasn't readily available.

    Not a perfect solution but it is possible, however you will have to study the code, and its not just java, look down the resource path as well, there be stuff lurking there!!

     
  • P Foomer
    P Foomer
    2014-03-24

    Hi

    if you want to extract stuff from various files, have you considered Tika.

    Aperture was designed as "the best RDF extraction framework".

    Perhaps with hindsight they should have split the Extraction and RDF code.

    Are you using RDF in your project, or just wanting to extract text?

    Some other info (search page for Aperture)

    https://vufind.org/jira/browse/VUFIND-600

    http://incubator.apache.org/projects/tika.html

     
  • Antoni Mylka
    Antoni Mylka
    2014-03-25

    FWIW.

    Tika has graduated from Apache Incubator to a Top-Level Project a long time ago. The homepage now is http://tika.apache.org

    Aperture was founded in 2005. At that time there was no Tika. It was founded by Leo Sauermann from DFKI and Christiaan Fluit from Aduna. Leo left DFKI sometime in 2009. His successor, Christian Reuschling, used Aperture in his projects unti I left DFKI at the end of 2011. Afterwards he decided he'd rather use Tika and develop some crawling functionality on top of it. RDF wasn't his priority. The result of this is the Leech Crawler http://leechcrawler.github.io/leech/. Christiaan Fluit and Aduna changed their focus completely.

    So, the logical continuation of Aperture is Tika for text extraction and Leech Crawler for crawling. The Leech Crawler hasn't been updated for a year though so I don't know how active it is at the moment. Didn't use it myself. It may work though, as long as RDF is not your priority.

    If RDF is your priority, then I'm afraid there is no successor. You could try to use any23.apache.org if you want to extract RDF triples from various sources (my personal experience with any23 output is rather bad though, would recommend to evaluate it carefully before you commit to it), or you could develop your own code to convert Tika's Metadata objects into RDF triples. You could also develop your own bridge between Tika and Aperture. Back in the day we discussed some TikaSubCrawler or something. Nothing tangible emerged out of it though.

    In hindsight, the biggest issue with Aperture was that half of the world did not want RDF. As for the other half, everyone had their own ontologies and the Aperture output wasn't configurable. We tried to push for a "standard" ontology, but that's not an easy task (see http://www.semanticdesktop.org/ontologies/ for the current status).

     
    • Chris Bamford
      Chris Bamford
      2014-03-25

      I see. I am in the half that doesn't want RDF as my priority is text and embedded file extraction only, so I guess Tika is the way forward for me!
      (I originally evaluated Tika v Aperture some 4 years ago and plumped for Aperture as the text extraction on the whole was more accurate.)

      I don't need crawlers either as I deal with files locally - so really I just need the best extractor technology.

      Thanks guys

      Chris

      Sent from my iPhone

      Chris Bamford
      Senior Developer
      m: +44 7860 405292
      p: +44 207 847 8700
      w: www.mimecast.com
      Address click here: www.mimecast.com/About-us/Contact-us/

      On 25 Mar 2014, at 00:09, "Antoni Mylka" mylka@users.sf.net<mailto:mylka@users.sf.net> wrote:

      FWIW.

      Tika has graduated from Apache Incubator to a Top-Level Project a long time ago. The homepage now is http://tika.apache.orghttp://tika.apache.org

      Aperture was founded in 2005. At that time there was no Tika. It was founded by Leo Sauermann from DFKI and Christiaan Fluit from Aduna. Leo left DFKI sometime in 2009. His successor, Christian Reuschling, used Aperture in his projects unti I left DFKI at the end of 2011. Afterwards he decided he'd rather use Tika and develop some crawling functionality on top of it. RDF wasn't his priority. The result of this is the Leech Crawler http://leechcrawler.github.io/leech/.http://leechcrawler.github.io/leech/. Christiaan Fluit and Aduna changed their focus completely.

      So, the logical continuation of Aperture is Tika for text extraction and Leech Crawler for crawling. The Leech Crawler hasn't been updated for a year though so I don't know how active it is at the moment. Didn't use it myself. It may work though, as long as RDF is not your priority.

      If RDF is your priority, then I'm afraid there is no successor. You could try to use any23.apache.orghttp://any23.apache.org if you want to extract RDF triples from various sources (my personal experience with any23 output is rather bad though, would recommend to evaluate it carefully before you commit to it), or you could develop your own code to convert Tika's Metadata objects into RDF triples. You could also develop your own bridge between Tika and Aperture. Back in the day we discussed some TikaSubCrawler or something. Nothing tangible emerged out of it though.

      In hindsight, the biggest issue with Aperture was that half of the world did not want RDF. As for the other half, everyone had their own ontologies and the Aperture output wasn't configurable. We tried to push for a "standard" ontology, but that's not an easy task (see http://www.semanticdesktop.org/ontologies/http://www.semanticdesktop.org/ontologies/ for the current status).


      [feature-requests:#118]http://sourceforge.net/p/aperture/feature-requests/118/ Advice please on extracting embedded files from RTF docs

      Status: open
      Group: 1.6.0 - features
      Created: Fri Mar 21, 2014 09:52 AM UTC by Chris Bamford
      Last Updated: Mon Mar 24, 2014 09:43 PM UTC
      Owner: nobody

      Hi

      I would like to extract embedded files from RTF docs. Loking through the codebase I have found an example on how to extract text, but nothing more.
      Please can you point me in the right direction?

      Thanks!

      • Chris

      Sent from sourceforge.nethttp://sourceforge.net because you indicated interest in https://sourceforge.net/p/aperture/feature-requests/118/https://sourceforge.net/p/aperture/feature-requests/118/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/https://sourceforge.net/auth/subscriptions/

       

      Related

      Feature Requests: #118

    • -----BEGIN PGP SIGNED MESSAGE-----
      Hash: SHA1

      Hi,

      Leech Crawler 'just works' as it is a thin wrapper, simply implementing a recursive Tika Parser
      for some container mimetypes (directory, imap, etc). Thus, there was no need for further
      development. I will migrate to the current version of Tika, but I expect that it works more or
      less out of the box, if Tika didn't changed their interfaces significantly.

      The biggest maintenance issues for crawling libraries are in the extraction support - this comes
      from Tika. As i.e. directories don't change so often, Leech Crawler has a relatively stable,
      static code base.

      best

      Christian

      On 25.03.2014 01:09, Antoni Mylka wrote:

      FWIW.

      Tika has graduated from Apache Incubator to a Top-Level Project a long time ago. The homepage
      now is http://tika.apache.org

      Aperture was founded in 2005. At that time there was no Tika. It was founded by Leo Sauermann
      from DFKI and Christiaan Fluit from Aduna. Leo left DFKI sometime in 2009. His successor,
      Christian Reuschling, used Aperture in his projects unti I left DFKI at the end of 2011.
      Afterwards he decided he'd rather use Tika and develop some crawling functionality on top of
      it. RDF wasn't his priority. The result of this is the Leech Crawler
      http://leechcrawler.github.io/leech/. Christiaan Fluit and Aduna changed their focus
      completely.

      So, the logical continuation of Aperture is Tika for text extraction and Leech Crawler for
      crawling. The Leech Crawler hasn't been updated for a year though so I don't know how active it
      is at the moment. Didn't use it myself. It may work though, as long as RDF is not your
      priority.

      If RDF is your priority, then I'm afraid there is no successor. You could try to use
      any23.apache.org if you want to extract RDF triples from various sources (my personal
      experience with any23 output is rather bad though, would recommend to evaluate it carefully
      before you commit to it), or you could develop your own code to convert Tika's Metadata objects
      into RDF triples. You could also develop your own bridge between Tika and Aperture. Back in the
      day we discussed some TikaSubCrawler or something. Nothing tangible emerged out of it though.

      In hindsight, the biggest issue with Aperture was that half of the world did not want RDF. As
      for the other half, everyone had their own ontologies and the Aperture output wasn't
      configurable. We tried to push for a "standard" ontology, but that's not an easy task (see
      http://www.semanticdesktop.org/ontologies/ for the current status).


      [feature-requests:#118] http://sourceforge.net/p/aperture/feature-requests/118/ Advice
      please on extracting embedded files from RTF docs

      Status: open Group: 1.6.0 - features Created: Fri Mar 21, 2014 09:52 AM UTC by Chris
      Bamford Last Updated: Mon Mar 24, 2014 09:43 PM UTC Owner: nobody

      Hi

      I would like to extract embedded files from RTF docs. Loking through the codebase I have found
      an example on how to extract text, but nothing more. Please can you point me in the right
      direction?

      Thanks!

      • Chris

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/aperture/feature-requests/118/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

      -----BEGIN PGP SIGNATURE-----
      Version: GnuPG v2.0.19 (GNU/Linux)
      Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

      iEYEARECAAYFAlMy1PEACgkQ6EqMXq+WZg9t7wCdGWI2vaWA6tTH/XKgufP7oqFJ
      ocgAn0NMvO9UOAEWZkhgTeZ7Tf7y8yh4
      =u+d4
      -----END PGP SIGNATURE-----

       

      Related

      Feature Requests: #118