Five days of autonomous local AI development: 77% to 99%
Five days of an experimental self-improving development environment around a frozen local model: quality, speed, cost and what stays open.
Between 31 October and 5 November 2026, an experimental development environment for Qwen3.8-Flash-Next IQ3_S raised the share of perfectly solved tasks from 77% to 99% on a fixed set of 178 tasks (invoices, spreadsheets, diagnostics, coding and filtering). Model weights were not fine-tuned.
Inference moved from Strata to Basalt. Decode rose from about 80 to 130–220 tok/s; prefill roughly doubled; a full benchmark run fell from 87 minutes to 6.9 minutes. The development environment went through 15 releases with contract checks.
Across the machines used for benchmark runs there were about 34,000 model calls and about 37 million tokens processed (about 4 million useful answer tokens). Calendar time was 120 hours; a large share of capacity stayed idle because of hardware changes, restarts and waiting for human decisions.
Estimated cost for the same token volume through commercial APIs (assuming 33M input and 4M output, standard rates, no cache): Qwen Flash about $3.25; DeepSeek Flash $7–15; Gemini 2.5 Flash about $20; GPT-5.4 Mini about $43; Grok 4.3 about $51; Claude Sonnet 5.5 about $106; GPT-5.4 about $143. Local electricity for five days is estimated at €7–10 at €0.25/kWh, including idle. This is a scale check, not a claim that local AI is always cheaper or that models are equal.
Against a compute-matched raw-model baseline (blind best-of-2), the development environment scored +13.5 percentage points. Holdout runs are underway to check whether gains hold on unseen tasks. Releases, method and benchmarks are on GitHub.