Highlights

• Now you can see a progress bar when chunking a lot of texts with batch chunking

  :::python from chonkie import RecursiveChunker

  chunker = RecursiveChunker()

  chunks = chunker([...], show_progress_bar=True) # progress bar is enabled by default

  🦛 choooooooooooooooooooonk 100% • 200/200 docs chunked [00:00<00:00, 229.65doc/s] 🌱

What's Changed

• Add CONTRIBUTING.md, update issue templates, CI, Codecov and more... by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/119
• [FEAT] Add TQDM to default installs + CONTRIBUTING.md + other minor updates by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/120
• [fix] CI: reports were not being uploaded to Codecov by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/121
• Update CONTRIBUTING.md with first issue hyperlink by @shreyashnigam in https://github.com/chonkie-ai/chonkie/pull/122
• [FIX] Support class methods as token_counter objects for CustomEmbeddings (#92) by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/127
• [Fix] Add fix for [#92]: Support class.method as a Tokenizer for CustomEmbedding +. minor changes by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/128
• [FIX] [#116]: Incorrectstart_index when chunk_overlap is not 0 by @Udayk02 in https://github.com/chonkie-ai/chonkie/pull/126
• [FIX] start_index incorrect when chunk_overlap is not 0 (#116) by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/132
• [FIX] Remove tests for Py3.8 — Incompatible for support by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/134
• [fix] High chunk_overlap causes last chunk to be entirely redundant by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/136
• [FIX] Handle edge case for RecursiveChunker (#131) by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/137
• [DOCS] Update readme intro to match docs. by @shreyashnigam in https://github.com/chonkie-ai/chonkie/pull/135
• [FEAT] Add TQDM progress bars for chunk_batch + Update README.md by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/138
• Replace dead discord link with infinite lifetime by @shreyashnigam in https://github.com/chonkie-ai/chonkie/pull/140
• [FIX] Minor fixes + Stylistic enhancements for TQDM and Multiprocessing by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/141
• [chore] Bump up the package version to v0.4.1 by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/143

New Contributors

• @shreyashnigam made their first contribution in https://github.com/chonkie-ai/chonkie/pull/122
• @Udayk02 made their first contribution in https://github.com/chonkie-ai/chonkie/pull/126

Full Changelog: https://github.com/chonkie-ai/chonkie/compare/v0.4.0...v0.4.1