Download
LongWriter is an open-source framework and set of large language models designed to enable ultra-long text generation that can exceed 10,000 words while maintaining coherence and structure. Traditional large language models can process large inputs but often struggle to generate long outputs due to limitations in training data and alignment strategies. LongWriter addresses this challenge by introducing a specialized dataset and training approach that encourages models to produce longer responses. The system uses an agent-based pipeline called AgentWrite that decomposes large writing tasks into smaller subtasks, allowing the model to produce long documents section by section. Researchers also created the LongWriter-6k dataset containing thousands of examples with outputs ranging from a few thousand to tens of thousands of words.
Features
- Large language models optimized for generating ultra-long text outputs
- AgentWrite pipeline that divides long writing tasks into smaller subtasks
- Training datasets designed specifically for long-form output generation
- Support for producing documents exceeding 10,000 words
- Benchmark tools for evaluating long-form generation performance
- Models derived from architectures such as GLM and LLaMA
Project Samples
