TL;DR
This study guide replaces a list of 200 resources with five stages for learning large language models, from next-token prediction to how chat models are trained and used. Each stage pairs one or two trusted resources with something to build. Good news for anyone with a browser full of course tabs: you can close most of them.
Pick up the path here:
Say you are a developer who calls a large language model API every day. You can write a prompt and parse the response, but when a colleague asks why the model miscounts letters in a word or forgets the start of a long document, you do not have an answer. You search for “how to learn LLMs” and find lists of 200 courses, papers and videos with no order to them.
You do not need 200 resources. To learn large language models properly, work through five stages in order: the basics of how a model predicts the next word, tokens and embeddings, neural networks and training, the transformer architecture, and finally how modern chat models are trained and used in applications. Each stage below lists what to learn, one or two trusted resources, and something to build so the idea sticks.
Key Takeaways
- A large language model (LLM) is a neural network trained to predict the next token in a sequence of text. Almost everything else builds on that one idea.
- Learn in order: next-token prediction, tokens and embeddings, neural network training, transformers and attention, then post-training and applications.
- Build something at every stage. Writing a bigram model, a tokenizer and a tiny GPT teaches more than reading about them.
- A few free resources cover most of the path: Andrej Karpathy’s Neural Networks: Zero to Hero, the Hugging Face LLM Course, and the original transformer paper.
- You need basic Python and some comfort with matrices and probability. You do not need a PhD.
What You Need Before You Start
Three things make the rest of this guide much easier:
- Python. Enough to write functions, loops and classes, and to read code that uses NumPy or PyTorch.
- Linear algebra basics. Vectors, matrix multiplication and the dot product. Attention, the core of a transformer, is mostly matrix multiplication.
- Probability basics. What a probability distribution is, and what it means to sample from one. A language model outputs a distribution over the next token.
If calculus is rusty, you can pick up what you need (derivatives and the chain rule) when you reach backpropagation in stage 3.
Stage 1: Understand Next-Token Prediction
Start with the simplest possible language model, one that predicts the next word from the current word by counting. This is called a bigram model, and it holds the core idea of every LLM: learn the probability of what comes next, then generate text by repeatedly picking a next word.
from collections import Counter, defaultdict
text = "the cat sat on the mat . the cat ate . the dog sat on the cat ."
words = text.split()
# Count which word follows which: P(next | current)
counts = defaultdict(Counter)
for current, nxt in zip(words, words[1:]):
counts[current][nxt] += 1
print(counts["the"].most_common()) # what usually follows "the"?
# Generate by always picking the most likely next word
word, output = "the", ["the"]
for _ in range(5):
word = counts[word].most_common(1)[0][0]
output.append(word)
print(" ".join(output)) Output:
[('cat', 3), ('mat', 1), ('dog', 1)]
the cat sat on the cat The model has learned that “cat” usually follows “the”. It has also shown its main weakness: it only sees one word back, so it loops. Every later stage is about giving the model more context and a better way to use it.
Build: extend the example to characters instead of words, then sample from the probabilities instead of always taking the top choice.
Stage 2: Learn Tokens and Embeddings
Real LLMs do not read words. They read tokens, which are chunks of text such as a whole word, part of a word or a single symbol. A tokenizer turns text into a list of token IDs before the model sees it. This explains many odd behaviors, from miscounted letters to number comparisons; our article on why LLMs say 9.11 is bigger than 9.9 walks through one example, and our guide to using fewer LLM tokens shows real token counts.
Each token ID is then mapped to an embedding, a vector of numbers the model learns during training. Tokens used in similar ways end up with similar vectors. The idea goes back to word2vec (Mikolov and colleagues at Google, 2013), which showed that word vectors could capture relationships, such as “king” relating to “queen” the way “man” relates to “woman”.
Build: install a tokenizer library such as tiktoken, encode a few sentences, and look at how words, numbers and code get split.
Stage 3: Learn How Neural Networks Train
Counting stops working once the context is longer than a word or two, because most combinations never appear in the data. Neural networks solve this by learning a function that predicts the next token from the context, adjusted step by step with gradient descent and backpropagation.
The best free resource for this stage is Andrej Karpathy’s Neural Networks: Zero to Hero, a series of video lectures with notebooks. It starts by building micrograd, a tiny automatic differentiation engine, then builds makemore, a character-level language model, step by step through a multilayer perceptron and the training details that make deep networks work.
Build: follow the first two lectures and write micrograd and the bigram makemore model yourself, without copying.
Stage 4: Learn Attention and the Transformer
The transformer is the architecture behind nearly every modern LLM. It was introduced in Attention Is All You Need (Vaswani and colleagues, 2017). Its key part, self-attention, lets every token in the context look at every earlier token and decide how much each one matters for predicting what comes next.
Read the paper once to see the overall design, then build it. Lecture 7 of Zero to Hero, “Let’s build GPT: from scratch, in code, spelled out”, writes a working GPT in PyTorch, and lecture 8 builds a GPT tokenizer. Karpathy’s nanoGPT repository was the classic follow-up for training a GPT-2 sized model; its README now marks it as deprecated and points to his newer nanochat project, which covers tokenization, pretraining, fine-tuning, evaluation and inference in one codebase. If you prefer another framework, our walkthrough of reproducing nanoGPT in JAX covers the same ideas.
Build: a character-level GPT trained on a public-domain text you like. Watch the generated text go from noise to something that looks like the source.
Stage 5: Learn How Chat Models Are Made and Used
A model trained only to predict the next token is a base model. It continues text but does not follow instructions well. Chat models go through post-training: fine-tuning on examples of good answers, then learning from human or AI feedback on which answers are better. Two papers explain the shift: “Language Models are Few-Shot Learners” (Brown and colleagues, 2020), which introduced GPT-3 and showed that a large base model can do new tasks from a few examples in the prompt, and “Training Language Models to Follow Instructions with Human Feedback” (Ouyang and colleagues, 2022), which describes InstructGPT.
Then learn how LLMs are used in real applications: prompting, retrieval-augmented generation (RAG), tool use and evaluation. The free Hugging Face LLM Course is a practical way in. It uses the Transformers, Datasets and Tokenizers libraries to load models, fine-tune them and share them on the Hugging Face Hub.
Build: fine-tune a pretrained open model on a narrow task, such as classifying support tickets, and compare it with prompting a general model on the same test set.
A 12-Week Study Plan
At a pace of 6 to 8 hours a week, a working developer can get through the core path in about three months. Adjust it to your background.
| Weeks | Focus | Main resource | Build |
|---|---|---|---|
| 1 to 2 | Python, vectors, probability, next-token prediction | Any linear algebra refresher, the bigram example above | Word and character bigram models |
| 3 to 5 | Neural networks and training | Zero to Hero, lectures 1 to 5 | micrograd and makemore |
| 6 to 8 | Tokenizers, attention, transformers | Zero to Hero, lectures 7 and 8; Attention Is All You Need | A character-level GPT and a byte-pair tokenizer |
| 9 to 10 | Pretrained models and fine-tuning | Hugging Face LLM Course | Fine-tune an open model on one task |
| 11 to 12 | Applications: prompting, RAG, evaluation | Model provider documentation, your own test set | A RAG app with a measured test set |
Books and Courses for Going Further
- Build a Large Language Model (From Scratch) by Sebastian Raschka (Manning). It codes a GPT-like model in PyTorch, then pretrains and fine-tunes it. The code is free in the book’s official GitHub repository.
- Speech and Language Processing by Dan Jurafsky and James H. Martin. The standard NLP textbook; drafts of the third edition are free on Jurafsky’s Stanford site.
- Stanford CS224N: Natural Language Processing with Deep Learning. Lecture videos from past years are public on YouTube.
For more reading, see our list of books on large language models.
Back to the Developer
The developer from the opening does not need the list of 200 resources. After stage 2 they can explain why the model miscounts letters: it sees tokens, not letters. After stage 4 they know why long documents are hard: attention over a long context is expensive, and models are trained on limited context lengths. Twelve weeks, a handful of resources and a project at each stage are enough to go from calling an API to understanding what happens behind it.
Frequently Asked Questions
How Long Does It Take to Learn Large Language Models?
For a developer who already knows Python, about three months at 6 to 8 hours a week covers the core ideas and a few working projects. Going deeper into research topics, such as training at scale or new architectures, takes much longer and is ongoing for everyone in the field.
Do I Need to Know Math to Learn LLMs?
You need basic linear algebra (vectors and matrix multiplication), basic probability, and the idea of a derivative for backpropagation. You can learn most of it alongside the material. Building the models in code is often the fastest way to make the math concrete.
Should I Still Learn Classic NLP Like NLTK?
It helps but is not required. Classic tools teach tokenization, stemming and text classification, which give useful context. If your goal is LLMs specifically, spend most of your time on neural networks, tokenizers and transformers.
Can I Learn LLMs Without a GPU?
Yes, for most of the path. Bigram models, micrograd, tokenizers and a tiny character-level GPT run on a laptop CPU. For fine-tuning larger models, free notebook services with GPUs or a few hours of rented cloud GPU time are usually enough for learning projects.




