We ran Stanford's new 'whole draft first' AI model ourselves. It aces Sudoku and loses numbers
Next-Concept Modeling writes a rough version of the whole text first, then sharpens it level by level. We rented a $0.34-an-hour graphics card and reran the published models. The paper's results held up. So did its warning: names and numbers get lost.

Key points
- Next-Concept Modeling (NCM), from Stanford's Hazy Research group, generates text coarse to fine: a short rough sketch of the whole answer first, then more detail at each level, instead of one word at a time.
- We reran the released models on a rented RTX 4090. Sudoku, translation and summaries matched the paper almost exactly: 86.25% of the hardest Sudoku puzzles solved (paper: 86.1%), translation BLEU 22.6 (paper: 22.6).
- The paper's headline speed-ups (up to 31× on summaries) are against diffusion language models, another experimental approach, not against ChatGPT-style models.
- In translation, numbers survived intact in 88% of sentences that contained them and names in 55%, against 99.6% and 92% for a standard open translator we ran on the same sentences.
- It is a research prototype of about 100 million parameters. Interesting for fast drafts and planning tasks; not something to use for work documents yet.
Every chatbot you have used writes the same way. One word, then the next, then the next. It cannot go back. If the third word was a bad choice, every word after it has to live with it.
A paper published on 9 October by Stanford’s Hazy Research group tries another way. Its method, Next-Concept Modeling, or NCM, first sketches the whole answer roughly, then adds detail in a handful of passes. Like a photo that loads blurry and sharpens.
The paper reports big speed-ups and an impressive result on Sudoku. We wanted to see it work, so we ran it ourselves.
Watch it write
This is our own run of the released translation model, turning a German news sentence into English. Move the slider, or press play.
GermanSie stehen keine 100 Meter voneinander entfernt: Am Dienstag ist in Gutach die neue B 33-Fußgängerampel am Dorfparkplatz in Betrieb genommen worden - in Sichtweite der älteren Rathausampel.
Human translationThey are not even 100 metres apart: On Tuesday, the new B 33 pedestrian lights in Dorfparkplatz in Gutach became operational - within view of the existing Town Hall traffic lights.
German"Laut aktuellen Messungen durchfahren auf der B 33 täglich etwa 12 000 Fahrzeuge die Gemeinde Gutach, davon sind etwa zehn Prozent Schwerlastverkehr", betont Arnold.
Human translation"According to current measurements, around 12,000 vehicles travel through the town of Gutach on the B33 on a daily basis, of which heavy goods traffic accounts for around ten per cent," emphasised Arnold.
GermanDies bestätigt auch Peter Arnold vom Landratsamt Offenburg.
Human translationThis was also confirmed by Peter Arnold from the Offenburg District Office.
They are not 100 meters away from another another: On Tuesday in the new B 33 pedestrian traffic was moved into operation in the village premises - in sight of the older town hall lights.
At level 2 there is almost nothing: a comma and “the”. By level 5 the shape is there, with repeats. By level 11 it is a sentence. Look at the second and third examples: the model gets the structure right and still stumbles on “12,000” and on “Peter Arnold”.
How it works, in plain words
NCM has two parts. A tokenizer learns to squeeze a piece of text into a stack of codes: a few rough ones at the top, more and finer ones below. A generator, a transformer like the ones inside ChatGPT, then produces that stack one level at a time, filling in every code of a level at once.
Our translation model used 11 levels, growing from 1 code to 64. Writing word by word, the same sentence would need one step per word.
The “concepts” are not ideas a person could read. They are learned codes. The name is a metaphor.
What we tested
We rented one NVIDIA RTX 4090 graphics card on RunPod for about $0.34 an hour, downloaded the published models and ran the authors’ own evaluation on three tasks. The whole test took about an hour and cost us $1.15.
| Task | Paper | Our run | Time on our card |
|---|---|---|---|
| Sudoku, easy / medium / hard (puzzles solved) | 99.5% / 96.5% / 86.1% | 99.40% / 96.45% / 86.25% | about 14 seconds for 2,000 puzzles |
| German to English translation (BLEU, 3,003 sentences) | 22.6 | 22.62 | about 40 seconds, including loading the model |
| Article summaries, XSum (ROUGE-L, 11,334 articles) | 28.0 | 27.93 | about 7 minutes |
The results held up. Anyone with a dollar and an hour can check them.
The Sudoku result is the interesting one
On the hardest puzzles, NCM solved 86% of boards. In the paper, a word-by-word model of the same size solved 1%.
That gap makes sense. In Sudoku every number depends on many others. A model that writes left to right has to commit to the first cell before it knows the rest. NCM works on the whole board at once and sharpens it level by level.
It is the same kind of problem that tripped up two AI slide makers in our test last week: a table where every cell has to agree with the rest. Whether this approach would fix that in a real product, nobody knows yet. But it is the right kind of question to ask of it.
The speed claims, and what they compare
The paper’s headline numbers, 21 times faster on one benchmark and 31 times faster on summaries, are measured against diffusion language models. Those are another experimental way of writing text in parallel. They are not the models behind ChatGPT, Claude or Gemini.
The paper’s own word-by-word baseline tells a fuller story. On summaries, NCM scored better than it. On translation, the word-by-word model of similar size scored higher: 25.2 BLEU against NCM’s 22.6. So NCM is not simply better than how chatbots write today. It is a different trade-off.
Names and numbers: our own check
The authors write that their tokenizer occasionally swaps names and numbers. We wanted to know how often that matters, so we checked every translated sentence.
For comparison we ran a standard open translator, Helsinki-NLP’s opus-mt, on the same 3,003 sentences and the same card. This is not a fair contest of methods. opus-mt was trained on far more text than the NCM model, which saw only this one dataset. It shows what a working tool looks like, not what NCM could become.
Did the numbers and names survive translation?
Share of test sentences in which every number (537 sentences) or every capitalised name that appears in both the German and the human English translation (1,204 sentences) also appears in the machine translation. Strict check: '30' written as 'thirty' counts as lost. Our run, 23 October 2026.
Source: AI Beat test of NCM (WMT14 De–En) and Helsinki-NLP opus-mt-de-en
Some of NCM’s slips were harmless: “thirty years” for “30 years”. Others would not be:
- “first documented together in the year 794” became “In 1794 church and church calls were first mentioned”.
- “the 1,250th anniversary … from 18 to 20 July” became “from 188th to 20th of July 1250”.
- “Peter Arnold” became “Peter Peter”.
opus-mt was also faster in our run: 19 seconds for all 3,003 sentences with careful settings, 14 seconds with the quickest. Our NCM timing includes loading two large model files, so the two numbers are not exactly comparable, but there was no speed-up over a standard translator on this task.
Why it isn’t in your chatbot
- Size. The main models have about 100 million parameters; the largest in the paper, 725 million. Today’s chatbots are hundreds or thousands of times bigger. Nobody yet knows how NCM behaves at that scale.
- One model per task. Each dataset gets its own tokenizer and generator. There is no general NCM assistant.
- Fixed shape. Every answer passes through the same number of levels, even a short one.
- Exact details. Names and numbers, as we measured.
- Compute. Scaling a method like this to chatbot size takes the kind of computing power academic labs rarely have.
What it could mean for you
Not much this year. In the longer run, three things:
- Faster drafts. A model that sketches the whole answer at once could produce a first draft very quickly, for a bigger model or a person to check.
- Planning and structure. Schedules, tables, puzzles: tasks where everything has to fit together. This is where the paper’s strongest result is.
- Lower latency. Answers that appear almost at once, which matters for voice assistants and live tools.
Until then, the lesson is the one from our slide test: whatever writes your text, check the numbers and the names.
How we tested: NCM code from the authors’ public repository (commit c702742) and the published checkpoints, run with the authors’ evaluation script on one RTX 4090 (24 GB) on 23 October 2026. Baseline: Helsinki-NLP/opus-mt-de-en, half precision, batch 64, greedy and 4-beam search. BLEU with sacreBLEU against the WMT14 reference. Our scripts and raw results are available on request.
Sources
- Kumbong, Khare et al.: Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation, arXiv 2610.14162 (9 October 2026)
- HazyResearch/ncm: code and evaluation (MIT licence)
- Hugging Face: hazyresearch/ncm-checkpoints
- Helsinki-NLP/opus-mt-de-en (the standard translator we used for comparison)
- RunPod: GPU pricing
- AI Beat: We gave four AI slide makers the same document
Comments
What do you think? No account needed. Please be kind; comments that attack people are removed.
No comments yet. Be the first.