A 235B model in a browser window

The last piece showed a language model with no home. You cut it into layers, run each layer in a different person's browser tab, and let them pass the work down the line one token at a time. What crosses between tabs is tiny, a whisper. Keeping it that small is the whole reason it works. That is the part Codec makes cheap. Given enough tabs, a model too big for any one of them runs across all of them, assembling itself out of whoever happens to be online.

That works for ordinary models. The largest ones, the kind a hyperscaler runs on a rack of specialized chips, strain it. Their layers are enormous. A single layer of the model in this piece runs to a gigabyte and a half. It still fits a browser. It is the first layer big enough that loading it the old way, all at once, stops scaling. A size beyond it, the layer stops fitting at all. So you split again, underneath the first cut. Not the model into layers, but a layer into its experts, streamed through a browser a piece at a time instead of staged whole. That second cut runs today at legion.codecai.net.

A mixture of experts is a different animal

A dense transformer runs every weight for every token. A mixture-of-experts model does not. Each layer holds a bank of parallel sub-networks, its experts. A small router picks a handful to run for each token. Qwen3-235B has 128 experts in a layer and runs 8 of them per token. That is the trick that lets the model advertise 235 billion parameters while only doing the work of about 22 billion per token. Most of the weights sit idle on any given step.

Idle is not the same as absent. To host a layer, a peer has to hold all 128 experts. It cannot know in advance which eight the router will call. And the experts are almost the entire weight of the layer. Each expert is a small gated feed-forward network built from three matrices. Two of them, gate and up, blow the incoming vector up into a much wider working space; the third, down, brings the result back. The gate matrix drives a switch that decides how much of the up matrix's output is let through. That is where the name comes from. One expert comes to about twelve megabytes.

A layer has 128 of them. The format does not store them as 128 separate things. It stacks them by role, with all 128 gate matrices in one tensor, all 128 up matrices in a second, all 128 down matrices in a third. That is the shape a GPU wants. It runs the handful of chosen experts as a single batched multiply over the stack rather than 128 small ones. Those three stacked tensors run to 432, 432, and 630 mebibytes, each holding all 128 experts. With the attention weights around them the layer comes to about 1.5 gibibytes, against roughly fifty megabytes for a dense 8B layer. One MoE layer weighs as much as an entire small dense model. Pin the 630-mebibyte figure down now. It is not one expert. It is the down matrices of all 128 experts, stacked into one block.

A gigabyte and a half still fits. WebGPU, the browser's path to the GPU, caps a single storage buffer at two gibibytes. A 1.5-gibibyte layer slots under that with room left; a browser can hold several of these layers at once. That is how the 235B runs across the mesh, with one host carrying a handful of its layers rather than one. The strain is in how a layer gets in. The old loader read the whole file into the WebAssembly heap, then handed it to the GPU in one piece. At the peak the layer sat in memory twice, once in the heap and once in VRAM. That heap is capped near four gigabytes. Spending a gigabyte and a half of it to stage one layer, in full, before the layer can serve a token, is fine once and cramped by the third. And for the model a size up, whose layer will not fit the two-gibibyte buffer at all, staging it whole stops being wasteful and becomes impossible.

Stream the layer, one tensor at a time

A layer is a short list of tensors, the three expert blocks, the attention weights, a few small norms. Each has a name, a byte offset, and a length. Nothing requires them to be resident together. You can allocate the destinations on the GPU up front, then, one tensor at a time, fetch only that tensor's bytes and place them into their slot. When you move to the next one, the previous bytes are gone. Peak memory stops being the whole layer and becomes the single largest tensor, the 630-mebibyte down block. It still carries all 128 experts; reaching inside it to pull out a single one is a further cut, one this piece comes back to.

legion-stage-runtime, the WebAssembly-plus-WebGPU engine the whole project rests on, holds the machinery.

An offset index ships alongside the layer, a small table listing every tensor's name, its byte offset in the file, and its length. An HTTP range request per tensor asks the content host for exactly those bytes and no more. Hugging Face honours it. Each request comes back 206 Partial Content with just that tensor's bytes. Fetching the 630-mebibyte down block therefore pulls 630 mebibytes over the wire, not the gigabyte and a half around it. A placement primitive in the runtime takes one tensor's bytes, writes them into their pre-allocated GPU buffer by name, then forgets them.

The runtime has to build the correctly-shaped, correctly-typed GPU buffers before any weight arrives, or it has nowhere to place the first tensor. So the load begins with a metadata index, a six-megabyte header-only file that carries the model's shapes and the name and type of every tensor but none of the weights. The runtime reads it and allocates every buffer the layer will need. Only then does it start streaming bytes into them. Begin, stream, finish.

Staging the whole layer holds all three expert matrices plus attention at once, about 1.5 gibibytes resident. Streaming keeps only the single largest tensor in flight, the 630-mebibyte down block, and drops each one before fetching the next. The peak more than halves. Residency is a separate question from that peak. A layer a peer is not running this instant does not have to sit in VRAM. It can wait in the WebAssembly heap or on the browser's own file system and come up when the router calls it, or the range can be divided so three peers each carry part of it.

What is proven, and what is not

v0.1.5 ships a browser tab that allocates a layer's GPU buffers up front and streams its expert tensors into them one at a time. Peak weight memory is 630 mebibytes against 1.5 gibibytes for staging the layer whole, observed on live loads. A layer that used to crowd both the heap and VRAM now arrives tensor by tensor under a fixed ceiling.

It is not fast. I have no tokens-per-second figure and will not invent one. What governs that number is coverage and download volume.

The full model is 94 of these layers, roughly 141 gibibytes of weights. An end-to-end chat needs every one of them online across the mesh at the same time. A missing range stalls the pipeline. Every token then walks all 94 stages with a network hop between each, so latency is the sum of 94 hops and nothing about a bigger tab shortens it.

Download volume is the other half. A host pulls all 128 experts of its layer, the full 1.5 gibibytes, though the router fires only 8 of them per token. That is fifteen times more weight than the architecture needs at any step, paid on every join.

The expert-dimension slice closes that gap by addressing a single expert inside the stacked blocks. Bandwidth. Fetch only the eight that fire, roughly 100 megabytes a layer against 1,500. Residency. With experts addressable one at a time, a host keeps hot ones in VRAM, spills warm ones to system memory, and leaves cold ones on disk or in a local model folder. A peer then carries far more of the model than its GPU alone could hold. Parallelism. Split one layer's 128 experts across several browsers and run them at once. mesh-llm slices a model only by layer. colibrì leans on one heavy peer. Neither can put half a layer on one machine and half on another. Legion's cut can.

The expert-dimension slice is a working prototype. Loading a layer's weights from a local folder is live. Hot-cold placement is a design. I will put a tokens-per-second number here when a mesh exists that is large enough to measure one.

A layer holds 128 experts; the router runs 8 of them for a given token. Streaming, as shipped, downloads all 128 but keeps only one resident at a time. That bounds memory. The frontier is to fetch only the handful that fire and evict the rest. That bounds bandwidth too. The gap between these two bars is the sparse fraction a mixture of experts exists to exploit.

No single place has to hold it

The envelope tax argued the wire between machines should carry the model's native numbers, not sentences dressed up for a reader who is another machine. The peer-to-peer piece spread one model across browsers by handing a token's hidden state between slices. This is what happens when a slice grows too big to load. The lever is distribution. Compression does almost nothing here, since trained dictionaries barely move already-quantized weights, and I measured that before saying it.

A Declaration of Independence landed on one finding. Whether AI is the ultimate dependency or the ultimate instrument of independence is a property of who owns the machine it runs on. The largest models are the ground the data center held most confidently. A 235-billion-parameter model streaming its experts through an ordinary browser tab is a small flag planted on that ground. Once the wire is cheap and the weights move in slivers, no single owner gets to decide whether you keep it.