partforge
Browse forgesSign in

Which models can actually do CAD?

7 models, 26 tasks, 502 attempts. Each attempt is graded on whether it built, on measured geometry, and on a checklist an AI judge answers from the renders. Click any number to open the attempt behind it.

Quality
against
0%25%50%75%100%$0.00$0.50$1.00$1.50$2.00$2.500s3m 20s6m 40s10m 0sOpus 5.5Gemini 3.8 FlashGPT-6 SolGPT-6 LunaDeepSeek V4.1 FlashKimi K3 FastKimi K3 (Moonshot)
Gemini 3.8 Flash leads on human rank. GPT-6 Luna is the cheapest to run. No model is both. 100% is first on every task, 0% is last, 50% is tied with the field.

Where the score and the reviewer disagree

Overall gives partial credit at every layer. This run's five spread over 8.6 points. Where the two orders below cross, the score is missing something the reviewer saw.

Overall scoreHuman rankingOpus 5.5: 1 by overall score, 2 by the reviewer's ranking, 1 place lowerGemini 3.8 Flash: 2 by overall score, 1 by the reviewer's ranking, 1 place higherGPT-6 Sol: 3 on bothGPT-6 Luna: 4 by overall score, 5 by the reviewer's ranking, 1 place lowerDeepSeek V4.1 Flash: 5 by overall score, 4 by the reviewer's ranking, 1 place higherKimi K3 Fast: 6 on bothKimi K3 (Moonshot): 7 on both1Opus 5.52Gemini 3.8 Flash3GPT-6 Sol4GPT-6 Luna5DeepSeek V4.1 Flash6Kimi K3 Fast7Kimi K3 (Moonshot)Gemini 3.8 Flash1Opus 5.52GPT-6 Sol3DeepSeek V4.1 Flash4GPT-6 Luna5Kimi K3 Fast6Kimi K3 (Moonshot)7

All models

Rank by
#ModelQualityCost and timeOverallStrictHuman rankGate pass
1Opus 5.5Anthropic
9358%69%95%
2Gemini 3.8 FlashGoogle
9164%69%96%
3GPT-6 SolOpenAI
9154%64%92%
4GPT-6 LunaOpenAI
8653%49%92%
5DeepSeek V4.1 Flash
8453%56%90%
—Kimi K3 FastFireworks
12 of 26 tasks
4128%29%
12 of 26
39%
—Kimi K3 (Moonshot)
24 of 26 tasks
71%3%8%

Overall gives partial credit at every layer, so the top of the board bunches. Strict counts only the attempts that got everything right. Gate pass is the share whose part was watertight, hole-correct and overlap-free. Hover any bar for its number. Ties break by cost, then time. In Quality, the bar marked with a person is one reviewer's ranking of the cases they ordered by hand. First place is 100%, last is 0%, ties share, and the tick marks 50%. That reviewer marked some first attempts unacceptable: DeepSeek V4.1 Flash 96%, Kimi K3 Fast 50%, Kimi K3 (Moonshot) 15%. Neither affects Overall or Strict.

Browse the tasks

Hard

How the score is made

Built 30%, measured geometry 50%, AI judge 20%. A failed gate zeroes geometry for that attempt. The AI judges are Opus 4.8 and GPT-5.6 Sol. Read the methodology.

This is an older run. See the current results.

Run hard-baseline-2026-09-232026-09-23commit 6edf19e0spend $212.67 + $43.09 AI judging, 15 unpricedstopped early (interrupted)Other runs: 2026-10-16T18-49-10-hard-compact-2026-10-16, 2026-09-10T19-44-23-baseline-2026-09, 2026-09-04T20-02-10-stage2-hard-cases, 2026-09-04T19-48-31-screen-14-fixed