Unstable Legion: the peer-to-peer ChatGPT
Unstable Legion is a peer-to-peer ChatGPT: one language model cut into layer-stages that run in ordinary web browsers and hand work to each other over the Codec wire, with no data center in the loop. Open a tab, chat, and optionally donate your GPU to host a few layers. The pipeline assembles itself across whoever is online and keeps generating as peers come and go. It is early and experimental, and it runs today at legion.codecai.net. Here is how it works, what is proven, and where it is headed.
Open a browser tab, point it at legion.codecai.net, and start typing. You get a chat window and a language model answers you. Nothing unusual so far. The unusual part is where the model is: it is not in a data center. It is not even entirely on your machine. It is spread across a handful of other browser tabs, on other people's computers, each one holding a slice of the model and passing its work to the next. You are talking to a model that has no home.
This is Unstable Legion, and it is the thing the envelope tax was quietly pointing at. That piece argued a small, specific point: AI ships the model's native numbers wrapped in text and JSON to a reader that is almost always another machine, which tears the packaging off and throws it away. The fix was to speak the model's native format on the wire, a machine-to-machine transport called Codec. That article stopped at the fix in principle. This one is the shipped proof, and the next move.
The model speaks in numbers
A model does not emit words. It emits tokens: integer IDs from a fixed vocabulary. Internally it produces and consumes those numbers natively; the text is a costume put on at the very end for the one human who might be reading. But most of the time nobody is. The caller is another agent, another server, another stage in a pipeline. Codec's premise is to stop dressing the numbers up for an audience of zero and ship them as they are: a small, binary, length-prefixed frame instead of a text-and-JSON envelope. Once you do that, something else opens up. If the thing crossing the wire is already small and native, the machine on the other end no longer has to be a whole model in a rented data center. It can be a slice of one, in someone's browser.
What Codec actually is
Codec is not a compression trick, although it compresses. It is a transport for computer-to-computer AI traffic. Token IDs go on the wire as bytes. A ToolWatcher catches a tool call by comparing one number against another, with no detokenizing and no text-scanning. A Translator hands a stream from one model's vocabulary to another's without ever passing through English in the middle. It is source-available, at version 0.5, with client libraries in TypeScript, Python, Rust, Java, .NET, and a C library small enough for a microcontroller.
The property that the rest of this story rests on is subtle: a Codec frame is self-describing about its precision. Each hop declares whether it is carrying 32-bit floats, 16-bit floats, or 8-bit integers. That sounds like a detail. It is the whole reason the next part works, because it means two machines running at different precisions can hand work back and forth and the differences are transport, never corruption.
Splitting the workload
A transformer is a stack of layers. Text goes in the bottom, gets tokenized and embedded, flows up through the stack, and a set of numbers called the hidden state passes from each layer to the next. Only the very top of the stack turns those numbers back into a token. Nothing says one machine has to own the whole stack.
The technique for splitting it is not mine. It comes from mesh-llm, the server-side cousin of a project called Petals, which pools GPUs across machines and serves one model from many boxes. Its slicing scheme, which the project calls Skippy, cuts a model into contiguous layer stages: the first stage owns the tokenizer and the embedding, the last stage owns the output head and does the sampling, and each stage in between just runs its own range of layers and keeps its own attention cache. What crosses the boundary between two stages is the hidden state for one token: on the order of a few kilobytes. Eight at 32-bit, four at 16-bit, and two at 8-bit, which is the leanest wire and the one Legion runs by default. That is the number that makes the whole idea feasible. The weights are large and get downloaded once. The thing that moves per token, between the slices, is a whisper.
Because only a whisper crosses the boundary, splitting turns a model that will not fit in one browser tab into several shares that will. An 8-billion-parameter model, quantized, is roughly five gigabytes of weights. No single browser tab is going to hold that comfortably. Cut into three contiguous stages, it is under two gigabytes per peer, which a modern laptop's GPU can carry.
The runtime: legion-stage-runtime
The slicing scheme is portable. The problem was that nothing could run a slice in a browser. Every browser inference engine runs a whole model: text in, text out. None of them could start at layer 12, accept a hidden state injected from another machine, run to layer 24, and hand the result on. That gap, running an arbitrary range of layers with activations passed in and out, was the load-bearing missing piece, and it is what most of the engineering went into.
The result is legion-stage-runtime: mesh-llm's Skippy stage patches applied to llama.cpp, compiled to WebAssembly with a WebGPU backend, so a browser tab can execute one contiguous layer range as a first-class operation. Activation in, activation out. That is the genuinely new artifact here, and it is open under Apache 2.0. Everything else, the mesh, the chat, the peer discovery, is plumbing around that one capability.
The receipts
I am wary of demos that work once on the machine that built them, so the milestones are dated and each was checked against the boring baseline of a single model running alone.
The first gate, on 14 July, was correctness. A small model, Qwen3-0.6B, was split across two WebGPU workers and run against the identical model running whole. At 32-bit precision the split produced token-for-token identical output, across five prompts and sixty-four tokens each, at about 33 tokens per second, verified by an automated browser test. As far as I can tell, that was the first time a language model had ever been run pipeline-parallel across browser workers at all.
The second gate was two real machines over a peer-to-peer WebRTC connection, and it surfaced the kind of finding that tells you the thing is real. Split across two different GPUs, the output eventually diverged from the single-machine run, deterministically, at one specific token. The cause was not a bug in the wire; the wire was bit-for-bit faithful. It was that floating-point addition is not associative, so two different GPUs summing the same numbers in a different order can land on opposite sides of a near-tie and pick a different next word. Token-for-token identity is a same-hardware property. The transport is exact regardless. That is a distinction worth stating plainly, because the honest version of a claim is more useful than the impressive one.
The third gate is the thing you can open right now. Legion runs a chat product on Qwen3-8B, split across whoever is hosting layers at the time. The pipeline assembles itself with no coordinator, and it survives churn: peers drop, new ones join, the mesh re-plans around the gap and keeps generating. The multi-host relay, the part that carries a token's hidden state through three or more machines in sequence, is landed and produces coherent output. You can watch it happen.
Why it is called Unstable Legion
The name is not modesty for its own sake. It is version 0.0.1, it is experimental, and the honest picture has a proven half and a frontier half.
What is proven today: dense models in the Qwen3 family, at 8, 14, and 32 billion parameters, slice cleanly and run in browsers. The sweet spot for latency is peers that are close together, a local network or a small trusted group, where each hop costs a millisecond or two. Across the open internet a single conversation pays real per-hop latency, tens of milliseconds a time. But a mesh is not one deep chain serving one person. It is many machines running many pipelines at once, so its capacity comes from parallelism, from lots of conversations in flight, not from one stream sprinting through a single relay. Judging it by the speed of one lonely chain would be measuring the wrong thing.
What is frontier, being built right now and not yet a claim I will put a number on: browsers are not the ceiling. A native peer can hold far more than a tab can, and through the same mesh. The current work is bringing genuinely large models into the network as native heavy peers, up to a 744-billion-parameter model, by giving each peer a slice and letting it stream its slice's experts from memory and disk the way a specialized single-machine engine already does. Aggregate the memory of several such peers and you escape the wall that any one machine hits, without a data center's interconnect. That is the direction. It is under active development, and I am not going to pretend it is finished or fast yet, because it is neither.
There is one genuine dead end worth naming, so the frontier does not sound like magic. The newest hybrid architectures, the ones that blend attention with a recurrent state-space memory, do not slice for a mesh: the state they would have to move between machines is tens to hundreds of times larger than the tidy hidden state a plain transformer hands across a boundary, and it swamps the wire. Ordinary dense transformers pass a whisper. Those pass a flood. For now, that class stays out.
And the real lever, the unglamorous one, is not compression of the weights. Trained dictionaries do essentially nothing to already-quantized model weights; I measured it. The lever is distribution: download each layer once, cache it locally, serve it from a content network, and slice finely enough that no peer has to hold more than its share.
How this ties in
Two of the longer arguments on this site are this same shape.
The envelope tax is the direct parent: it made the case that the wire between machines should carry numbers, not sentences. Legion is what that wire is for. Once the thing crossing the network is a small, self-describing, native frame, you are no longer limited to moving finished answers between whole models. You can move a single token's worth of half-finished thought between two halves of one model, and the model can live anywhere.
The case that the cloud lock-in is eroding is the same move, taken to its limit. That piece tracked inference drifting from rented cloud compute toward hardware people already own. Legion is the far end of that drift: the model does not just run closer to you, it leaves the data center entirely and reassembles out of whatever machines happen to be online. And the stenographic mediator is the pattern underneath both, machines talking to machines, where the text in the middle was always for nobody.
The token-native wire was pitched as an efficiency fix, a way to stop shipping envelopes. That was the modest version. The real payload is architectural. When the wire is cheap enough, the model stops having to live at any one address.