Published: 2026-06-30
Summary

Ornith 9B Local Coding Test: 9B vs 35B Agentic Model on a Mac Mini

Chapters / key moments (click to jump — plays here on the page)

Bart Slodyczka runs the new open-source Ornith 1.0 9B dense model locally on a 16GB M4 Mac mini via LM Studio and the PyAgent harness, tasking it to build a tower-defense game in a single HTML file. The 9B model produced broken, non-functional code, while the 35B variant one-shotted a playable game — a clear, honest demonstration of where small local coding models still fall short.

Source video

"New Agentic Coding Model Ornith 9B — Is It Worth Running Locally?" by Bart Slodyczka — Watch on YouTube →

Key Takeaways

The model family. Ornith 1.0 (by Deep Reinforce, San Francisco) is an agentic-coding family in 9B dense, 31B dense, 35B MoE and 397B MoE sizes. The 9B, 35B and 397B are open-weight — downloadable and runnable today; the 31B isn't released yet. It's built on pretrained Gemma 4 and Qwen 3.5.
Memory & context on 16GB. On the Mac mini, the 9B GGUF used ~6GB for weights and reached ~185K context at full KV cache (~15GB total system usage). Enabling Flash Attention and quantizing the KV cache to Q8 lets you push the full context window while consuming only ~10GB.
Throughput. ~16 tokens/sec for the 9B on the constrained Mac mini, versus ~100 tokens/sec for the 35B running on a larger Mac Studio. On Apple Silicon the MLX format is faster than GGUF, especially for prefill on longer prompts.
The 9B failed the build. Asked to build a tower-defense game in one HTML file, the 9B produced ~750 lines that didn't work — undeclared or empty functions, broken tool calls, and it looped endlessly when asked to self-debug. The 35B one-shotted a comparable, playable game.
Bottom line. At 9B parameters the accuracy for from-scratch builds isn't there — you have to reduce scope or expectations. Small local models struggle to "zip the app together"; the 35B has the capacity for real work but needs more hardware.

Tools & Commands Mentioned

LM Studio            # free desktop app to download + run local models; search "Ornith", pick 9B (GGUF or MLX)
                     # enable Flash Attention + Q8 KV cache to fit more context in limited RAM
pyagent              # lightweight harness used to drive the local model (faster than Claude Code for small models)
/model               # inside PyAgent — pick the loaded Ornith model (e.g. Ornith 1.0 9B)

What you can actually set up from this

Extracted from the video's own transcript — the specifics the original summary left out.

Configuration shown

Test machine and memory budget

M4 Mac mini, 16 GB RAM. 4 GB left free for the operating system, roughly 12 GB for the model. Runtime: LM Studio (free desktop app). Harness: Pi (auto-captioned as "pie"), not Claude Code, because the speaker found the wait on small local models too long in Claude Code (first request about 3 to 4 minutes).

Ornith 1.0 variants (Deep Reinforce)

9B dense, 31B dense, 35B MoE, 397B MoE. The speaker believes the 9B, 35B and 397B are open weight and that the 31B was not released yet. Built on pre-trained Gemma 4 and Qwen 3.5 per the release paper.

LM Studio settings used for the 9B (GGUF)

Quantization file 5.63 GB. 6.2 GB for weights at 8,000 tokens of context. Maximum context 260,000 tokens, which needs about 14.45 GB. Staying near 12 GB gives about 186,000 tokens (the speaker used 185,000 with a full-precision KV cache). With flash attention on and the KV cache quantized to Q8 (both K and V), the full context window fits in 10.29 GB.

Template shown in the video. Adapt to your project — do not copy verbatim without reviewing each line.

Reproducible steps

Pick a format
LM Studio offers GGUF and MLX builds of the 9B. MLX is slightly smaller and is the one to use on Apple silicon: prefill is faster once the prompt is thousands of tokens long, though for a tiny message the tokens per second are about the same.
Measure speed
9B on the Mac mini: about 16 tokens per second in this recording (about 18 the day before; the speaker blames memory pressure from other open apps). The 35B 4-bit MLX build on a Mac Studio, pulled across with LM Link, ran about 100 tokens per second, roughly five times faster, on a bigger-bandwidth machine.
Run the same one-shot build on both
Prompt used: create a new HTML file (9b-demo / an MLX demo for the 35B) replicating a simple tower-defense game in a single HTML file. Both outputs were about 750 lines.
Compare results
The 35B game ran after the speaker sent it one fix request (towers were not shooting). The 9B output did not look right and nothing happened in play. Letting the 9B debug its own code looped and made it worse; the speaker handed the 9B code to the 35B model, which fixed it over four or five iterations into a playable but weaker game.

Gotchas

The speaker's own verdict: at 9B, reduce the expectation or reduce the scope. The model can write plausible code but misses on precision.
Failure patterns seen over several days with the 9B: functions called but never declared, empty functions, misspelt names, functions defined but never used, and tool calls that were broken so the harness could not tell what the model wanted.
The 9B produced a lot of thinking tokens when debugging.
Benchmark charts (Terminal-Bench 2.1, SWE-bench Verified and Pro, a Claw eval) test working in an existing harness or fixing a bug in a codebase, not building from scratch. The speaker is sceptical of vendor charts and says to test on your own hardware and use case.
One run per model, one prompt, one machine: an anecdote, not a benchmark.