News
First Strata runs and a lab for repeatable local AI configurations
31 tok/s with a ~47 GB model using only about 3 GB of system RAM, 45–48 tok/s after a RAM upgrade, 32 GB of RAM confirmed as enough, the first autonomous overnight workflow, and identical machines for repeatable configurations.
The first result: 31 tokens per second
- Strata v0.1.41
- Model: Qwen3.8-Flash-Next IQ3_XXS
- CPU: Intel Core Ultra 7 270K Plus
- GPU: AMD Radeon RX 9070 OC, 16 GB GDDR6
- RAM: 32 GB DDR5-6000
- SSD: Lexar NM1090 Pro 2 TB, NVMe PCIe 5.0
- Context: 65,536 tokens
The model generated 12,136 tokens in about 6.5 minutes: 31 tokens per second. With a model of about 47 GB, VRAM was almost full while system RAM usage was only about 3 GB: Strata reads part of the model directly from the SSD. The GPU drew about 130–150 W, stayed at 45 °C and the fans were barely audible. This opens the possibility of significantly cheaper local AI systems, without huge amounts of RAM or expensive server platforms.
A systematic programme on different hardware
These results started a more systematic programme: Strata on 11 computers with very different configurations, with NVIDIA, AMD and Intel GPUs, DDR4 and DDR5, and NVMe storage from PCIe 3.0 to PCIe 5.0, including a PCIe 5.0 RAID 0 array of about 35 GB/s. First compatibility, performance and stability; then practical everyday and business tasks.
The opposite case was checked too: on a mini PC with no discrete GPU and 8 GB of RAM, Strata could be built but the model did not launch, even after adding RAM, because of dynamically allocated video memory. Conclusion: Strata does not launch on this undocumented configuration.
256K context and the first practical lessons
On the RX 9070 system with a 256K context, output was 15–30 tokens per second. In one session (9:44–12:00): 18 requests, about 108K prompt tokens (114 tok/s), about 856K reused, about 100K written (22.6 tok/s). Out of about 25 requests, a few broke: a prompt with five questions at once (it worked when split) and a request for a 60–70 thousand token answer. Other outputs reached 14–22 thousand tokens, and long generations held up in quality.
Strata did not detect the GPU without a connected monitor on this ASUS Pro Q870M-C-CSM board; an HDMI dummy plug solved it.
32 GB of RAM is enough
With 32 GB of RAM, Strata can run without loading the model into system RAM: memory stays free for other tasks and the CPU for parallel workloads, at 15–20 tok/s. With 64 GB on the same machine (Strata v0.1.43, Qwen3.8-Flash-Next IQ3_S), generation reached 45–48 tok/s, but with much less memory left for other work. Later, on two different configurations including DDR4, Strata barely used RAM at 15–25 tok/s while about half of the CPU stayed free for Hermes and supporting components. Performance drops as context length grows, so it depends on the tasks.
The first autonomous workflow
An overnight run on unsupported hardware with no discrete GPU gave about 1 token per second, because much of the model ended up in swap. Speed was not the goal: the task was assigned in the evening and the result was ready in the morning. Hermes then moved to a machine with a usable speed and began everyday work. One morning check already uses Home Assistant sensors and monitors backups, disk health and temperatures across connected computers; it promptly reported that the server room was getting too warm with six computers running.
A lab of identical machines
To turn individual results into a repeatable setup, we bought 10 identical motherboards and CPUs for 10 matching systems: an ASUS platform with 8 RAM slots, affordable DDR4 and BIOS-level remote management, so agents can even reinstall the OS remotely.
We had already set up 20–30 computers almost from scratch and managed them by voice from a phone through cloud agents, including an accounting workflow from ERP installation to entering invoices into the Italian tax system. The aim now is to reproduce such workflows with local AI. Out of the box, tasks that take seconds or a minute in the cloud took 5–10 minutes locally; optimization came next. Tasks come in from many sources, and the agents file them in OpenProject and assign them to the best suited agent or person.
What this means for a Box customer
- Large models can run on affordable hardware with modest RAM.
- Configurations are chosen from measurements on real machines.
- Identical machines make results repeatable and remote management possible.