R&D case study

MoEVM — a 58 GiB model on a 48 GB Mac.

Mixture-of-Experts models only use a few experts per token. MoEVM keeps the right ones on the GPU and pages the rest from the SSD, so models beyond the GPU’s memory budget run on Apple Silicon — with output identical to llama.cpp without MoEVM.

  • 8.72 tok/s gpt-oss-120b (58 GiB) on the GPU, median of 5
  • 3.1× Qwen3-Coder-Next vs llama.cpp op-offload, same prompt
  • 0.0 max logit difference — 1,024 rows × 151,936 tokens
  • 95.8 % expert-cache hits with LRU (optimal: 97.5%)
  1. 1 Pack

    Routed experts packed into a sidecar file — 4,608 records with a SHA-256 index.

  2. 2 Stage

    A RAM arena with LRU eviction; 4 threads prefetch the experts the router is likely to pick.

  3. 3 Route

    Read the router’s choice once per layer, plan GPU slots, swap expert ids atomically.

  4. 4 Compute

    Experts run from a persistent Metal pool; the rest of the model stays on the GPU.

  5. 5 Protect

    Epochs: a slot is never evicted before the GPU is done with it. A memory-pressure listener trims RAM.

  6. 6 Verify

    Every change is gated by logit parity against llama.cpp without MoEVM.

The performance ladder
  1. llama.cpp op-offload 4.24
  2. Sidecar + Metal pool 13.08
  3. Hardened, dynamic-K 19.31
  4. 4k context 21.88
  5. Graph-build, no copy hook 21.90

Qwen3-Coder-Next Q4, tokens/s, runs with routing traces on. Clean head-to-head with tracing off: 6.41 → 19.90 tok/s (3.1×).

How I kept it honest

  • Parity row by row: max difference, argmax and token over the full vocabulary.
  • Jitter test: 500 µs injected delays and forced syncs — still 0.0.
  • Launch gates written before the results; median of 5 runs, never the best one; warm-cache numbers refused.

What didn’t work — and the limits

  • Callback-based expert loading: 2.07 tok/s. Dropped.
  • When a model fits in memory, plain llama.cpp wins (49.69 tok/s). MoEVM is for the ones that don’t.
  • On CPU, plain llama.cpp runs gpt-oss-120b at 10.9–14.4 tok/s on short prompts. MoEVM’s edge is the GPU path with bounded memory.
  • Speculative decoding brought no real gain on the final path.
  • +4,755 lines of C++ in the llama.cpp fork
  • 5.8k lines of Python tooling
  • 1,123 benchmark run files
  • ~18 h of active work, with AI coding agents

Draft pull request #25309 on llama.cpp (sparse mmap ranges). An outside user reported it helped them load a model on a 128 GB M5 Max. See the pull request ↗ · MoEVM repository on GitHub ↗

More case studies

Own product · in development

Where?

344 designed flow steps

Plan outings alone or in groups, then reveal the map by actually going. Product, UX, iOS app and backend — all mine, built with a team of AI agents.

  • Expo
  • iOS → Android
  • TypeScript
  • Fastify
  • PostGIS
  • Mapbox
Read the case study

Founder & CEO

Selvren

11,901 tests in the merge gate

AI domain experts grounded in a company’s private documents — every claim checked against its sources before it’s shown. Dashboard and enterprise API.

  • Fastify
  • PostgreSQL + pgvector
  • Mistral
  • React
  • Clerk
Read the case study

Let’s build something.

I take freelance projects: I design, build and ship AI tools wired into the software your team already uses. Also open to joining a team or a startup that needs someone who builds — remote-first today, regular stays in Seoul, and a move to Korea ahead.

Email me GitHub ↗ LinkedIn ↗

Or reach me directly

  • WhatsApp +33 6 19 32 62 26 Open ↗
  • KakaoTalk ID Maxence.foulon

Download my resume

Pick a language — both versions tell the same story, 3 pages each.

Maxence Foulon © 2026 Designed and built by Maxence Foulon.