OK. I will pull this back --lucky it never was uploaded-- on the grounds that the current implementation is in fact faster than DAWG. I would've never thought so, but it is.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anthony,
I have done some work on the internals of Tokens.pas and added a DWAG (Directed-acyclic word graph) to it so that searches get to be faster.
You'll find an entry [RQ ID: 1339343] under the Patches
It is expected to be a drop-in replacement for Tokens.pas.
Cheers,
Adem
Sounds good. I'm away this weekend and the next, but I will look at it as time permits.
Thanks
Anthony
Hiya. I don't see the file. It should be attached to the patch.
The question to ask is - what speedup does it give, for what added complexity?
The file seems to be missing somehow.
Speed: I did not do a numerical test, but I did notice an improvement.
Complexity. Should not add any more complexity as the unit that handles DAWG is an independent one.
DAWG unit will be most useful elsewhere later when used for Capitalisation and similar stuff.
OK. I will pull this back --lucky it never was uploaded-- on the grounds that the current implementation is in fact faster than DAWG. I would've never thought so, but it is.