Adding web search to a local model meant writing a textbook egress guard. Search then returned empty results, reliably — the guard was refusing Wikipedia and the search API, and its judgement was entirely correct. The cause was synthetic addresses from a transparent proxy landing in an RFC 2544 reserved range. That check-versus-use gap turned up three times in one file.
A lookup table cannot cover variant × format × context × card count, so you model it. This records that modelling: what each of the three terms accounts for, how to measure the global-attention-layer share you cannot look up by reading a slope, how the forgotten concurrency multiplier makes inference two orders of magnitude slower, and two hard constraints beyond VRAM — tensor-parallel divisibility and why recommendations cannot be ranked by size.
Symptom, cause and fix for four real errors: shipping the .wasm without its .mjs glue module, nginx 1.24 having no .mjs entry so browsers refuse the module script, config arriving 155 bytes after the dependency that reads it, and why the cross-origin isolation that WASM threads require is a bad trade for a content site.
A debugging log from a browser-side matting model. int8 quantisation measured IoU 0.976–0.984 on my test set; on a real image where subject and background share a palette it scored 0.296. Four hypotheses were killed by experiment before the cause turned out to be that I had only validated on samples the model was confident about.
Architecture, official benchmarks, real checkpoint sizes, and measured VRAM for E2B, E4B, 12B Unified, 26B A4B, and 31B. What PLE effective parameters, the encoder-free Unified design, and MoE active parameters each solve, and why E4B quantized is larger than 12B.
Two complete routes for going public: a Cloudflare tunnel needing no inbound ports, and direct port forwarding with your own domain and certificate. Covers the nginx layer, DNS-01 issuance, DDNS caveats, and fixing a correct port forward that still cannot complete a handshake.
Making the model useful: disable thinking mode to cut token usage by 90%, add a shim for tools that cannot customize the request body, declare multimodal capabilities correctly, plus a symptom-indexed troubleshooting table.
Build RAG on the same machine: run embeddings on CPU using no VRAM, chunk by heading then by size, combine sqlite-vec vector search with FTS5 lexical search via RRF, and wrap it as an OpenAI-compatible endpoint that returns citations.
Start from nvidia-smi, look up real checkpoint sizes instead of estimating, choose vLLM or llama.cpp by VRAM tier, install the engine, download weights, run it as a systemd service, and verify real throughput with curl.
A field report from building a local Gemma 4 inference gateway on 8GB of VRAM: why QAT checkpoints are 2x larger than parameter counts suggest, how one flag changes prompt throughput 40x, a port-forward that fails because replies leave by the wrong gateway, and four traps in RAG chunking and CJK full-text search.
