Which models can actually do CAD?
2 models, 29 tasks, 174 attempts. Each attempt is graded on whether it built, on measured geometry, and on a checklist an AI judge answers from the renders. Click any number to open the attempt behind it.
Read this gap with care
This run limited Kimi K3.1 Pro to 32,000 output tokens per response. On the articulated figure task, two of its three attempts reached that limit in their first response and built nothing. That one task is most of the gap between the two models: without it, both score about 92.
We then raised Kimi's limit to 60,000 and ran that task six more times on its own. All six passed the gates and none reached the limit. The task's mean score rose from 29 to 89, against 98 for Sonnet 5.5. The attempts also cost more and took longer: $1.30 and about 12.7 minutes each, up from $0.54 and about 5 minutes.
Those six attempts are not in the numbers on this page. Using their score for that task would put Kimi at about 91.6 against Sonnet's 92.4, but that is an estimate, not a measured run.
All models
| # | Model | Quality | Cost and time | Overall | Strict | Gate pass |
|---|---|---|---|---|---|---|
| 1 | Sonnet 5.5Anthropic | 92 | 56% | 97% | ||
| 2 | Kimi K3.1 ProModal | 89 | 53% | 93% |
Overall gives partial credit at every layer, so the top of the board bunches. Strict counts only the attempts that got everything right. Gate pass is the share whose part was watertight, hole-correct and overlap-free. Hover any bar for its number. Ties break by cost, then time.
Browse the tasks
Hard
608 bearing seat
Articulated figure
Bottle neck adapter
Cable clip
Drawer divider
Extrusion 2020 end cap
Laser box printed hinges
Mesh add tabs
Mesh holder holes
Mesh rebuild blender handle
Mesh rebuild freecad bracket
Mesh rebuild freecad knob
Mesh rebuild scan
Mesh socket mount
Nema17 face mount
Photo cluttered real
Photo knob
Photo knob oblique
Photo lid
Photo motor bell
Photo motor bell unstated
Photo resistor
Rpi4 tray
Bracket second mount
Enclosure build up
Knob then widen
Lid fit and finger hole
Photo lid then centred
Posable figure then hinges
How the score is made
Built 30%, measured geometry 50%, AI judge 20%. A failed gate zeroes geometry for that attempt. The AI judges are Opus 5.5 and GPT-6 Sol. Read the methodology.
This is an older run. See the current results.
Run kimi31-sonnet-hard2026-11-05commit 58201d93spend $50.99 + $14.79 AI judgingOther runs: 2026-11-05T05-50-12-kimi31-vs-sonnet, 2026-10-16T18-49-10-hard-compact-2026-10-16, 2026-09-23T06-56-16-hard-baseline-2026-09-23, 2026-09-10T19-44-23-baseline-2026-09, 2026-09-04T20-02-10-stage2-hard-cases, 2026-09-04T19-48-31-screen-14-fixed