htdig-dev

[htdig-dev] Problems with fuzzy algorithms

[Lachlan A. <lh...@us...>]

Greetings all,

I have been testing the fuzzy algorithms, and can't get "speling" to=20
work, and I don't understand "regex".

"Speling" generates permuted forms of the query term, and check each=20
to see if it is in the database, like so:

      // First transposes
      // (these are really common)
      initial =3D stripped;
      char=09temp =3D initial[pos];
      initial[pos] =3D initial[pos+1];
      initial[pos+1] =3D temp;
      if (!wordDB.Exists(initial))   // Seems weird, but this is=20
correct
=09words.Add(new String(initial));

      // Now let's do deletions


The problem is that, whether or not the permutation is in the=20
database,  wordDB.Exists(...)  always seems to return -1 ("true"),=20
which means that the 'if' always fails (despite the comment saying it=20
is correct :)


"Regex" has problems with the meta characters being stripped out=20
before getting to the algorithm.  Should the meta characters be part=20
of "extra_word_characters"?  If so, what happens if "regex" and=20
"exact" are both specified in  search_algorithms?  When I set=20
"extra_word_characters=3D.*[^]\\$", the query ".*vers[ia].*" expands to=20
all the *numbers* in the word database.

Has anyone got  speling  or  regex  to work, or is the person who=20
wrote either of them still here?

Thanks!
Lachlan

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)

[htdig-dev] Re: Problems with fuzzy algorithms

[Lachlan A. <lh...@us...>]

Greetings all,

It seems that the problem is that words in the word database have a=20
couple of numeric fields appended to them.  My guess is that these=20
are new since 3.1, but I don't know what they are for.

The solution seems to be to write a second compare function which only=20
compares the string component of the keys.

Comments, anyone?

BTW, my internet connection has been very flaky of late, so apologies=20
if I'm slow in replying to anything.

Cheers,
Lachlan

On Sat, 11 Oct 2003 12:33, Lachlan Andrew wrote:

> "Speling" generates permuted forms of the query term, and check
> each to see if it is in the database, like so:
>   if (!wordDB.Exists(initial))   // Seems weird, but this is correct
>
> The problem is that, whether or not the permutation is in the
> database,  wordDB.Exists(...)  always seems to return -1 ("true"),
> which means that the 'if' always fails (despite the comment saying
> it is correct :)

--=20
lh...@us...
ht://Dig developer DownUnder  (http://www.htdig.org)