a tokenizer that is general purpose, and multilanguage. it will provide consistent usage across the languages, and be tied into that language's unique common usage types.
Be the first to post a text review of fms string tokenizer. Rate and review a project by clicking thumbs up or thumbs down in the right column.
this is our first release, it is no where near complete, and it has some minor bugs. however, most of it works at this time. I currently believe that the benefits of working with a consistent string tokenizer across multiple languages out weigh the drawbacks of working with this project as it is at this time. In worst case, it will at least provide a building block upon which your own system may work.
Be the first person to add a text review.
Copyright © 2009 Geeknet, Inc. All rights reserved. Terms of Use
Thanks for your rating!
Would you also like to write a review?