Understanding AI in a Month
Thirty days, one idea a day, from nothing to following the argument.
A thirty-day course in what these systems actually are. Each day is a written article with its figures and its sources, an audio edition, and a printable copy. Days are published as they are made.
Day 12026-09-1012 min listen8 figures
The Model Is Not the Product
In July a benchmark score nearly tripled without anything inside the model changing. The interesting question is not how the software was improved.
Day 22026-09-1011 min listen5 figures
How Text Becomes Tokens
The best-known failure of large language models is that they cannot count the letters in a word, and the best-known explanation for it is that the word arrives broken into pieces.
Day 32026-09-1012 min listen5 figures
One Token at a Time
What "predict the next token" actually means — and the half of the sentence that almost every popular account leaves out.
Day 42026-09-0712 min listen4 figures
How Words Affect Other Words
The operation that lets a word at the end of a sentence change what a pronoun in the middle refers to is nine years old, was named after something it does not do, and is now a minority of the layers in the models whose internals can be read.
Day 52026-09-1012 min listen6 figures
How an Answer Unfolds
A machine asked the same question a thousand times, with greedy decoding requested, returned eighty distinct completions. What decides which word comes next, and how much of the past the decision may consult, are two settings — and both are somebody's decision rather than a fact about the machine.
Day 62026-09-1012 min listen7 figures
What the Application Adds
A trained model reads text and writes text. Everything a product appears to do besides that — searching, opening files, running commands, remembering a previous conversation — is ordinary software deciding what text to place in front of it and what to do with the text it returns. That division decides a great deal about what a system costs to run. It decides less than the industry's own marketing suggests about whether the system is right.
6 of 30 days published; 24 still to come. Each day is a written article with its own figures, an audio edition, and a printable copy.