[htdig-dev] External parsers and MIME headers

[Rzepa, H. <h....@ic...>]

We have the following apparent failure which I would value any hints for.

Way back at the time of  ht://dig 3.1.14 we developed a set of external parsers,
invoked as usual via 
external_parsers: chemical/x-pdb "/usr/java/bin/java chemical.Htdigfront"
in the conf file (these parsers are used to extract only important tokens
from the files, and to derive metadata and heuristic perception
about the content; for chemists the molecular formula etc from a molecule coordinate file etc etc). 

The server we index has all the types set via the  Apache mime.types, and
-vvv  correctly shows these as mapping to those in the htdig.conf file. An example
of what (should) happen is at http://www.ch.ic.ac.uk/chemime/test6.html

Having compiled  3.1.16 (using IRIX, we missed  3.1.15 out), and with only very minor
changes, given it the same server, same conf file and same parsers, it refuses all the external types
(see again http://www.ch.ic.ac.uk/chemime/test6.html ). The parsers themselves
do appear to be working if given the four htdig arguments manually from the
relevant directory, it just appears that  htdig is not calling them at all. The MIME
types are all of the type  chemical/foo. Might it be that somewhere hardcoded into
htdig are the primary types, and  chemical is not one of them? 

Its clearly a MIME header issue, since if all the external types are  REMOVED
from the Apache mime.types file, the headers all come over as text/plain,
and  htdig 3.1.16 now correctly includes them all as being pure text types.
and the external parsers are now not invoked at all, htdig doing all
the parsing internally.  

Can I ask if anyone has tested the  external parsing calls, and if anyone
has any suggestions as to what else we might try? 
-- 

Henry Rzepa. 
+44 (0870) 132 3747 (eFax) +44 0778 6268 220 (Mobile)
 http://www.ch.ic.ac.uk/rzepa/ Dept. Chemistry, Imperial College, London, SW7  2AY, UK.