News
Strata releases on our fleet: the agent updates, measures and optimizes
Our agent detects new Strata releases by coldniko, updates the machines and re-runs the benchmarks: decode up to 2.4× faster at 16K on a Tesla P100 system, +46% expert slots on a Radeon system, the Mac Studio confirmed optimal, and DeepSeek V4.1-Flash running locally.
Strata is a local AI inference engine developed by coldniko. Since mid-October we have been following its releases on several of our own machines. Every computer runs the same model, Qwen3.8-Flash-Next IQ3_S, so the results are comparable. The machines are named after their hardware level:
- P1005: Intel Xeon E5-2690 v4, NVIDIA Tesla P100 16 GB HBM2, 32 GB DDR4, NVMe about 2.5 GB/s, CUDA 12.
- P2002: Intel Core Ultra 7 270K Plus, AMD Radeon RDNA4 (gfx1201) 16 GB, 64 GB DDR5, NVMe RAID 0, ROCm / HIP.
- Mac Studio: Apple M3 Ultra, Metal.
The agent itself detects a new Strata release, downloads it, runs the measurements and, on request, optimizes the configuration. All optimization runs locally, using the same Qwen3.8-Flash-Next IQ3_S model on the latest Strata version.
P1005 (Tesla P100): from data-starved to compute-bound
The model experts total 46.8 GiB, too big for RAM, and the 16 GB card was so full that the engine warned that decode would fall to a few tokens per second. Following the Strata repository documentation, the agent changed four things:
--vram-reserve-mib 960 (was 700)
--max-context 65536 (was 262144)
STRATA_Q8_SM60=0
--resident-experts
STRATA_DENSE_F32=1Result: GPU utilization went from 27% to 92% (155–170 W). Prefill became up to 4.4× faster; decode +40% at 8K and 2.4× at 16K.
| Context | Before (prompt / decode, tok/s) | After (prompt / decode, tok/s) |
|---|---|---|
| 512 | 47.7 / 13.4 | 54.5 / 14.3 |
| 2K | 62.6 / 10.3 | 74.5 / 13.9 |
| 8K | 46.4 / 9.9 | 178.1 / 14.3 |
| 16K | 43.8 / 6.2 | 193.9 / 15.1 |
| 32K | 44.1 / 6.0 | 213.8 / 12.6 |
Values are prompt / decode tokens per second. The cost: 24 GiB of RAM is now page-locked for experts, so nothing else large should run on this machine, and the context cap is 65K instead of 262K. Everything persists across reboots.
P2002 (Radeon RDNA4): live tuning, then a short restart
First, live tuning without restarting the engine. The CPU governor set to performance and a longer page cache for experts were kept; pinned GPU clocks reduced prefill by about 10% and were rejected; larger read-ahead had no effect. Decode at 2K and 8K improved by 10–18%.
Then, with a restart of about 40 seconds (the fallback chain carried the session, with no visible interruption), the agent raised the VRAM reserve from 700 to 960 MiB and enabled --kv-grow, keeping the full 262K context. Expert slots grew from 2518 to 3686 (+46%), the expert hit rate rose from 74% to 87% and the paging warnings disappeared. It also added a watchdog so the engine restarts on its own.
| Context | Before (prompt / decode, tok/s) | After (prompt / decode, tok/s) |
|---|---|---|
| 512 | 547 / 63.6 | 602 / 57.5–64 |
| 2K | 1228 / 60–71 | 1275–1303 / 66–74 |
| 8K | 2085 / 59–69 | 1723–2144 / 79–82 |
| 16K | 2180 / 72.7 | 1988–2169 / 76–78 |
| 32K | 2056 / 72.9 | 2023–2096 / 76–77 |
Standard suite: chat decode 74.3 tok/s (was 71.9), first token 0.39 s (was 0.46), agent decode 66.7 tok/s (was 59.2). The agent also noted honestly that the first numbers right after a restart are about 25% low because the expert cache is still cold.
Mac Studio (M3 Ultra): the defaults were already optimal
The agent restarted the engine six times for clean A/B runs and proved empirically that the default configuration was already the best one. KV options had no effect on unified memory; a large wired expert cache was harmful (decode collapsed to about 32 tok/s) and was reverted. Prefill of about 790 tok/s is the bandwidth limit of the hardware, not of the configuration. Final suite: chat decode 69.1 tok/s, first token 0.48 s. All dead ends were documented so nobody repeats them.
New releases, measured
- Strata 0.2.0 with a user interface: Strata and local AI can now be installed without the command line, which makes them accessible to a much wider audience.
- v0.2.2: three computers updated from v0.2.1.
- v0.2.3 → v0.2.3.1 on the Mac Studio: decode 61.3 → 66.4 tok/s (+8%), prefill 729.9 → 738.9 tok/s (+1%), no regression and no re-downloads of model files.
- v0.2.6 and v0.2.7: every release so far has brought another performance gain. For v0.2.7 the agent prepared a summary of the release notes and installed nothing without approval.
DeepSeek V4.1-Flash running locally
DeepSeek V4.1-Flash, a large MoE model, now runs locally on Strata: the hot experts stay in RAM and the rest is streamed from the SSD. We measured 14 tok/s decode. On our current hardware this is not enough for the main brain of the agent, but it works well as a second-opinion model and overnight auditor.
What this means for a Box customer
- Engine updates are detected, applied and measured by the agent, with approval before changes.
- Each hardware level gets its own measured configuration, and failed options are documented.
- The same model on different hardware gives a clear basis for choosing a configuration.