Ask HN: Build Your Own LLM?
The best way to really understand how something works is to build it yourself. So I am wondering if there are any good tutorials on building your own LLM from scratch. I.e. implementing tokenisation, embeddings, attention and so on. I am not suggesting one could replicate chatGPT, but more a toy model that implements the core features but based on a much smaller corpus and training data.
How about this?
https://mathstodon.xyz/@empty/115088095028020763
thanks
Andrej Karpathy: Let's build GPT: from scratch, in code, spelled out. https://www.youtube.com/watch?v=kCc8FmEb1nY
Andrej Karpathy's Nano GPT is reasonably accessible and easy to run.
https://github.com/karpathy/nanoGPT
https://www.amazon.com/Build-Large-Language-Model-Scratch/dp...
I'd get it straight from Manning and save a few bucks and take out the middle man: https://www.manning.com/books/build-a-large-language-model-f...
thanks. looks potential