~1-bit llm

A language model where every single weight is −1, 0 or +1 — nothing in between. The whole thing is 2.31 MB and it runs entirely in this tab. No server, no API key, nothing leaves your machine.

—
tokens per second
— on disk — parameters — of them ternary

fetching weights…

It forgets who its characters are after a few sentences. Names change mid-story, a rabbit may fly. That is not the 1-bit part — the full-precision version does it too. It read about eleven times less text than it needed, deliberately, because the point was measuring what ternary weights cost, not building something good.

tweak it

Switch the engine to JavaScript (reference) to feel what it was like before the inner loops moved into WebAssembly — roughly ten times slower, same output.

look at the weights

Three possible values means a weight matrix prints as a picture. - is −1, · is zero (no connection), + is +1. Not many models let you read their parameters with your eyes. About 31% come out as zero, which falls out of the maths rather than being chosen. 


    

  


  

    how does 1-bit even work
    

      

        A model is a giant pile of numbers. Normally each is stored as a precise
        decimal like 0.8371942, and producing one word means
        multiplying your input by millions of them and adding up the results.
      

      

        Multiplying is the expensive part. Try 47 × 89 in your
        head, then 47 + 89 — chips feel the same way, and a
        multiplier circuit costs ten to twenty times what an adder does.
      

      

        So: what if every weight could only be −1, 0 or +1? Then multiply
        by +1 means add it, by −1 means subtract it, and
        by 0 means skip it. The multiplication disappears. Every hard
        operation becomes an easy one. Three values is really about 1.58 bits, but
        the name stuck.
      

      

        Training such a thing has an obvious problem: you cannot nudge a number
        that is only allowed to be one of three values. The trick is two copies.
        Behind every −1/0/+1 sits a hidden precise number that nothing ever
        computes with — all the tiny training nudges go into that, and the
        visible weight is whichever of the three it is nearest. Most nudges change
        nothing. Then one tips it over a boundary and the weight flips. A light
        switch with a dimmer behind it.
      

      

        This one has 11,159,360 parameters. It read twenty million tokens of
        children's stories — about fifteen million words — and trained in
        roughly fifteen minutes on a free graphics card.