Tokenizers Files

What's Changed

Okay mostly doing the release for these PR: * Update deserialize of added tokens by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1891 * update stub for typing by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1896 * bump PyO3 to 0.26 by @davidhewitt in https://github.com/huggingface/tokenizers/pull/1901

Basically good typing with at least ty, and a lot fast (from 4 to 8x faster) loading vocab with a lot of added tokens and GIL free !?

• ci: add support for building Win-ARM64 wheels by @MugundanMCW in https://github.com/huggingface/tokenizers/pull/1869
• Add cargo-semver-checks to Rust CI workflow by @haixuanTao in https://github.com/huggingface/tokenizers/pull/1875
• Update indicatif dependency by @gordonmessmer in https://github.com/huggingface/tokenizers/pull/1867
• Bump node-forge from 1.3.1 to 1.3.2 in /tokenizers/examples/unstable_wasm/www by @dependabot[bot] in https://github.com/huggingface/tokenizers/pull/1889

• Bump js-yaml from 3.14.1 to 3.14.2 in /bindings/node by @dependabot[bot] in https://github.com/huggingface/tokenizers/pull/1892

• fix: used normalize_str in BaseTokenizer.normalize by @ishitab02 in https://github.com/huggingface/tokenizers/pull/1884

• [MINOR:TYPO] Update mod.rs by @cakiki in https://github.com/huggingface/tokenizers/pull/1883
• Remove runtime stderr warning from Python bindings by @Copilot in https://github.com/huggingface/tokenizers/pull/1898
• Mark immutable pyclasses as frozen by @ngoldbaum in https://github.com/huggingface/tokenizers/pull/1861

• DOCS: add add_prefix_space to processors.ByteLevel by @CloseChoice in https://github.com/huggingface/tokenizers/pull/1878

• Bump express from 4.21.2 to 4.22.1 in /tokenizers/examples/unstable_wasm/www by @dependabot[bot] in https://github.com/huggingface/tokenizers/pull/1903

New Contributors

• @MugundanMCW made their first contribution in https://github.com/huggingface/tokenizers/pull/1869
• @haixuanTao made their first contribution in https://github.com/huggingface/tokenizers/pull/1875
• @gordonmessmer made their first contribution in https://github.com/huggingface/tokenizers/pull/1867
• @ishitab02 made their first contribution in https://github.com/huggingface/tokenizers/pull/1884
• @Copilot made their first contribution in https://github.com/huggingface/tokenizers/pull/1898
• @ngoldbaum made their first contribution in https://github.com/huggingface/tokenizers/pull/1861
• @CloseChoice made their first contribution in https://github.com/huggingface/tokenizers/pull/1878

Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.22.1...v0.22.2