Moonshot describes Kimi K3.1 Pro as a post-training release on the same architecture as Kimi K3 — same 2.8T-parameter mixture of experts, same 104B active parameters, same 1,048,576-token context, with substantially larger post-training compute [VENDOR]. On Artificial Analysis’s version of that comparison, the composite index moved 43.59 to 55.09. And one row moved 12.63% to 67.17% on Terminal-Bench 4.0 [INDEPENDENT, index v4.3.2, 26 October]. OrcaRouter‘s Kimi K3.1 card carries both Terminal-Bench figures beside the $2.00/$10.00 rate, and the size of the gap is the most interesting engineering question in the release.
A 54.5-point jump on an agentic benchmark, from a model whose weights were not rebuilt, is not impossible. It is also not the normal shape of a post-training gain, and the difference matters because it determines whether the improvement will hold when the harness changes.
The five rows that moved
| Evaluation | Kimi K3 (Max) | Kimi K3.1 Pro | Change |
| Terminal-Bench 4.0 | 12.63% | 67.17% | +54.54 |
| GDPval-AA | 1,534.1 | 1,845.7 | +311.6 Elo |
| AutomationBench-AA | 58% | 73% | +15 to +24 |
| CritPt | 3.14% | 24.57% | +21.43 |
| Humanity’s Last Exam | 46.90% | 52.46% | +5.56 |
| SciCode | 59.49% | 60.76% | +1.27 |
| AA-Omniscience | 19.70 | 29.52 | +9.82 |
| Long-context recall | 88.67% | 86.33% | −2.34 |
The shape is not “everything went up by ten”. It is “the agentic and professional-deliverable rows went up enormously, the academic rows barely moved, and recall went slightly down”. That is a signature, and it points at what the post-training run was optimised for.
Three explanations, in order of plausibility
One: the training changed behaviour, not knowledge. RL over “hundreds of downstream domains” [VENDOR] rewards finishing tasks. A model trained to finish learns to use tools, retry on error, and stop hedging — which is precisely what Terminal-Bench scores. SciCode moving 1.27 points while Terminal-Bench moves 54.5 is consistent: science coding is a single-shot skill the base model already had; terminal work is a behaviour the post-training installed.
Two: the measurement harness was more favourable. Terminal-Bench results move by tens of points with scaffolding — how tool output is formatted, whether retries are allowed, how much context carries between steps. If the two Kimi rows were produced with different scaffolds or tool budgets, part of the 54.5 is a harness difference wearing a model’s name. The evaluator publishes one harness, which argues against this, but the size of the jump is a reason to ask.
Three: the predecessor was under-measured. Kimi K3 at 12.63% is a low number for a model that scored 43.59 on the composite and 59.49% on SciCode. A model that can write science code but fails twelve out of thirteen terminal tasks is describing a model that does not know how to operate a shell — plausible for a July release whose harness behaviour was not tuned, and consistent with the fact that Moonshot’s own launch table for K3 did not feature Terminal-Bench 4.0 at all.
The honest position is that all three contribute, and that the split matters for exactly one decision: whether to trust the number. If mechanism one dominates, the improvement is in the weights and will survive any reasonable harness. If mechanism two dominates,you will see something closer to K3 on your own scaffolding. Both hypotheses predict a big improvement over the predecessor; they differ on the absolute level.
How to find out in an afternoon
The test that separates them is cheap, because the predecessor is still callable and the difference is not in the model’s price.
1. Run five terminal tasks you actually care about through both models with the same scaffold, the same tool budget, the same retry policy and the same effort setting.
2. Score on pass/fail, not on time or token count.
3. Repeat each task three times — agentic benchmarks are noisy per item.
If K3.1 Pro wins most of them, the improvement is real for your harness and the absolute level is your number, not AA’s 67. If it wins by a little, expect a partial transfer. If it does not win, your scaffold is doing the work the harness did in the benchmark, and the ranking tells you nothing about your stack.
That experiment costs a few dollars and is worth more than any table in this article.
What not to conclude
Do not conclude the model is 54 points better at engineering. Terminal-Bench is one task family with a specific scaffold. It is a good predictor of agentic terminal work and a poor proxy for “engineering ability”.
Do not conclude the predecessor is bad at agents. It scored 88.67% on long-context recall and 59.49% on SciCode, both strong. It was bad at *operating a terminal under an evaluator’s harness*.
Do not port an agent configuration across generations without re-testing. The prompt that squeezed a pass out of K3 may be over-constrained for a model that now takes more steps, and the tool-call budget that suited a hesitant model may now be too tight. Re-baseline the configuration as well as the model — the 54-point jump says the behaviour changed more than the capability did.
One sanity check costs nothing: if a vendor’s own table omits the benchmark that moved most, ask why. Moonshot’s launch material did not feature Terminal-Bench 4.0 for the predecessor at all, which is consistent with a model that had not been tuned for it rather than one that had been measured and found wanting
The reading that survives
Whatever the mechanism, the direction is not in doubt: on the same independent harness, in the same index version, the new model scores five times the old one on the most agentic row in the table, and its composite is 11.5 points higher. A release that lifts GDPval by 312 Elo, CritPt by 21 points and Terminal-Bench by 54 is doing real work — most likely work aimed at the delivery of finished tasks rather than at raw problem solving.
For a buyer, the practical translation is simple. If you tried Kimi K3 for agentic work and it did not hold a plan together, that experience does not carry over. If you are choosing between K3.1 Pro and a frontier model on Terminal-Bench numbers, remember that the 67% and the 59.6% behind it were produced by the same harness and the same evaluator, which is the only comparison that is clean; and the 65.7% in the vendor’s table is a different measurement entirely.
