Back to explore

Building an LLM from First Principles with Jev: A Classifier-Driven Autoregressive Loop

The author built an LLM from first principles with Jev: 29 yes/no questions per character predict whether the next key is a–z, space, comma, or period; the highest probability is appended and fed back into the loop, forming an autoregressive process made from a classifier.

i made an llm from first principles with Jev 29 yes/no questions per character: should the next key be a–z, space, comma, or period? highest probability gets append to it, then fed the updated text back in & repeat an autoregressive loop made out of a classifier

· 884 likesOpen on X