Re: [htdig-dev] Re: [413879] local files / external parsers

[Gilles D. <gr...@sc...>]

According to Lachlan Andrew:
> On Fri, May 10, 2002 at 06:19:11PM -0400, Geoff Hutchison wrote:
> > On Fri, 10 May 2002, Lachlan Andrew wrote:
> > > KDE help used to use  ht://Dig  to provide a search capability.
> > > They changed the format of their files from HTML to  docbook  (XML).
> > > For some reason,  ht://Dig  refuses to call the parser that one of
> > > the KDE developers wrote.  The response was that it was not a bug,
> > > but a calculated feature, because  ht://Dig  didn't know that no server
> > > parsing was necessary.
> > 
> > For 3.2, the best approach is to either:
> > a) Index using file:// URLs, which should use the appropriate mime.types
> > file: <http://www.htdig.org/dev/htdig-3.2/attrs.html#mime_types>
> > b) Code the RetrieveLocal method to produce temporary file:// URLs that
> > are retrieved using the htnet/HtFile methods. (which again should use the
> > appropriate mime.types file)
> 
> I don't understand why there needs to be a temporary  file://  URL.
> I've attached a patch (against the latest beta, b3) in which
> RetrieveLocal  explicitly calls the method from  HtFile  which checks
> the MIME type.
> 
> Please let me know if this patch is unsuitable, and if so how I can
> fix it.  If it is OK, I'll go ahead and implement  bad_local_ext  etc.

I think your approach is a simple and elegant solution to this problem.
I'd recommend two changes:

1) Grab the most recent 3.2.0b4 snapshot from
http://www.htdig.org/files/snapshots/ and adapt your code to that version.
Some of the mime.types handling code in HtFile::Request() has changed
subtly between 3.2.0b3 and now, so it would be better to work with the
current code base.  Also, your patch to htsearch/Display.cc isn't needed
with 3.2.0b4.

2) The HtFile::Request() and Document::RetrieveLocal() methods both
have some hardcoded extensions, which should probably be kept in the
new HtFile::Ext2Mime() method.  HtFile::Request() currently falls back
on these when it can't open mime.types.

Thanks!

-- 
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)