Patch for local models
Many of you use Crush to use LLM models running in your local machines. In this release, our favorite contributor @Kartik33 worked on two small fixes to make Crush more compatible with local models:
- Some apps like LM Studio rejects
max_tokens = 0, so we'll omit the field when unnset instead of sending zero to make them happy. - When context window was unset (
context_window = 0), Crush would immediately try to summarize a brand new session. We'll skip summarization when the context windows is unset or unknown.
If you're using Crush for local models, please let us know how we can make it better for you in Slack or Discord!
Charm ✨
Changelog
Fixed
- 741c18a21121fde99dbcadc7eadafca037ba790e: fix: support local models with unknown max_tokens and context window (#2554) (@Kartik33)
Verifying the artifacts
First, download the [`checksums.txt` file](https://github.com/charmbracelet/crush/releases/download/v0.55.1/checksums.txt) and the [`checksums.txt.sigstore.json` file](https://github.com/charmbracelet/crush/releases/download/v0.55.1/checksums.txt.sigstore.json) files, for example, with `wget`: :::bash wget 'https://github.com/charmbracelet/crush/releases/download/v0.55.1/checksums.txt' wget 'https://github.com/charmbracelet/crush/releases/download/v0.55.1/checksums.txt.sigstore.json' Then, verify it using [`cosign`](https://github.com/sigstore/cosign): :::bash cosign verify-blob \ --certificate-identity 'https://github.com/charmbracelet/meta/.github/workflows/goreleaser.yml@refs/heads/main' \ --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \ --bundle 'checksums.txt.sigstore.json' \ ./checksums.txt If the output is `Verified OK`, you can safely use it to verify the checksums of other artifacts you downloaded from the release using `sha256sum`: :::bash sha256sum --ignore-missing -c checksums.txt Done! You artifacts are now verified!Thoughts? Questions? We love hearing from you. Feel free to reach out on X, Discord, Slack, The Fediverse, Bluesky.
