#34 An Atomic Workflow for Context-Limited Local LLMs

[Anonymous]

Originally created by: sachin-detrax

Here's a cleaner version with your additional idea integrated naturally:

One of the biggest limitations of using local LLMs for coding and bug fixing is the context window. Even though models like Qwen 3.6 35B are among the best coding models that can run on consumer GPUs, my system can only handle around a 90k token context. Large codebases quickly exceed that limit.

Instead of trying to fit the entire repository into the model's context, we could build a tool that works in small, iterative steps.

The idea is to first analyze the repository and create a graph of the codebase, showing how files, functions, classes, modules, and their relationships are connected. That information, along with intermediate findings, would be stored locally in a SQLite database, effectively acting as long-term memory for the model.

When given a feature request or bug to fix, the model wouldn't immediately start reading files. Instead, it would first read the code graph and the task description, analyze what parts of the codebase are likely relevant, and retrieve only those files into its context.

From there, the workflow would be completely iterative and atomic:

1. Scan the repository and build a code graph.
2. Save the graph and metadata to SQLite.
3. Clear the model's context.
4. Read the graph and the current task (feature or bug).
5. Identify and load only the relevant files.
6. Analyze them, summarize findings, identify possible root causes or implementation steps, and update the graph and database with everything learned.
7. Clear the context again.
8. Reload only the updated memory from SQLite and continue from exactly where it left off.
9. Repeat this cycle until the feature is implemented or the bug is fully resolved.

Each iteration is independent. The model loads only the information needed for the current step, performs a small piece of work, saves everything it learned back to the database, clears its context, and starts the next iteration from the saved state.

Instead of relying on one massive context window, the database becomes the model's persistent memory. The code graph continuously evolves as the model discovers new relationships, dependencies, implementation details, and decisions. Every iteration updates that shared knowledge before clearing the context and resuming work.

This allows relatively small local models to tackle codebases far larger than their context window would normally allow. Rather than trying to remember the entire repository at once, the model progressively builds an understanding of the project, retrieves only what's necessary for each step, and continues working until the task is complete. It's essentially giving local LLMs an external memory system and an atomic execution loop instead of forcing everything into a single prompt. Humans keep inventing bigger context windows. This approach simply admits memory exists and uses it. A strangely efficient idea.

[Anonymous]

Originally posted by: sosidudku1

Thank you for writing this up properly, this is one of the strongest ideas in the tracker. It matches where we want the runtime to go: structured external memory instead of ever-bigger context windows. The nearest concrete pain this maps to is the agent struggling on large files, so a first experiment could target exactly that: repo graph plus a SQLite findings store, loading only what the current step needs. If you feel like prototyping it behind an experimental flag, we will review it. Keeping this open as a design discussion.