[htdig-dev] Problem indexing file:// urls

[Rajendra I. <in...@be...>]

Hello,

I ran into indexing file:// urls with htdig ( htdig-3.20b3).
The same file is indexed properly with http:// urls with
local_urls.

With file:// all lines are appended together with no line breaks between
consecutive lines, resulting in incorrect word sequences. With certain 
files,
the word becomes so long that it takes forever to index the file. When it is
done, I suspect the resulting index would not be correct.

The same file is read correctly using http:// urls with local_urls. I 
have attached files
which illustrate the problem.

/usr/local/htdig/htdig/bin/htdig -v -v -v -v -a -m urllist \
    -c /usr/local/htdig/htdig/conf/htdig.conf  > htdig.log

This should be easy to fix if it is not fixed already.

/Raj Inamdar