Board · task version 2
docs
Each board ranks harness × model combinations on one suite by pass rate with a 95% Wilson interval; rows are not comparable across suites or task versions.
6 tasks
- docs-cache-config
- docs-config-loader-keys
- docs-gym-baseline-keys
- docs-gym-run-flags
- docs-retry-behavior-diff
- docs-snapshot-new-command
docs · task version 2
| # | Livery | model · harness · config | passed / n | 95% Wilson | mean | tasks | tokens / pass | cost / pass | excluded |
|---|---|---|---|---|---|---|---|---|---|
| 1 | moonshot/kimi-k3claude-code · config ecb67e4e | 6 / 6 | 61%–100% | 1.00 | 6 | 205k | $0.14 list | 3 | |
| 2 | anthropic/claude-opus-5-5codex · config 3437a9e6 | 6 / 6 | 61%–100% | 1.00 | 6 | 45k | $0.20 list | 0 | |
| 3 | anthropic/claude-sonnet-5-5codex · config 3437a9e6 | 6 / 6 | 61%–100% | 1.00 | 6 | 36k | $0.08 list | 0 | |
| 4 | fireworks/kimi-k3codex · config 3437a9e6 | 6 / 6 | 61%–100% | 1.00 | 6 | 43k | $0.06 list | 0 | |
| 5 | moonshot/kimi-k3codex · config 3437a9e6 | 6 / 6 | 61%–100% | 1.00 | 6 | 55k | $0.06 list | 0 | |
| 6 | openai/gpt-6.1-sol @highcodex · config b2396403 | 6 / 6 | 61%–100% | 1.00 | 6 | 86k | $0.04 list | 0 | |
| 7 | moonshot/kimi-k3opencode · config 357da891 | 6 / 6 | 61%–100% | 1.00 | 6 | 76k | $0.10 list | 0 | |
| 8 | anthropic/claude-opus-5-5pi · config da41c66e | 6 / 6 | 61%–100% | 1.00 | 6 | 21k | $0.11 list | 0 | |
| 9 | baseten/glm-5.3-flashpi · config da41c66e | 6 / 6 | 61%–100% | 1.00 | 6 | 23k | $0.0037 list | 0 | |
| 10 | fireworks/kimi-k3pi · config da41c66e | 6 / 6 | 61%–100% | 1.00 | 6 | 36k | $0.08 list | 0 | |
| 11 | fireworks/qwen3.8-maxpi · config da41c66e | 6 / 6 | 61%–100% | 1.00 | 6 | 66k | $0.06 list | 0 | |
| 12 | google/gemini-4-argon @highantigravity · config 034ed3a1 | 3 / 3 | 44%–100% | 1.00 | 3 | 160k | $0.17 list | 0 | |
| 13 | anthropic/claude-opus-5-5claude-code · config c32a950b | 3 / 3 | 44%–100% | 1.00 | 3 | 1.06M | $0.36 list | 0 | |
| 14 | anthropic/claude-sonnet-5-5claude-code · config c32a950b | 3 / 3 | 44%–100% | 1.00 | 3 | 904k | $0.15 list | 0 | |
| 15 | google/gemini-3.8-flash @highclaude-code · config ecb67e4e | 3 / 3 | 44%–100% | 1.00 | 3 | 626k | $0.19 list | 0 | |
| 16 | google/gemini-3.8-flash @lowclaude-code · config ecb67e4e | 3 / 3 | 44%–100% | 1.00 | 3 | 186k | $0.13 list | 0 | |
| 17 | google/gemini-3.8-flash @mediumclaude-code · config ecb67e4e | 3 / 3 | 44%–100% | 1.00 | 3 | 204k | $0.14 list | 0 | |
| 18 | google/gemini-3.8-flash @lowcodex · config 3437a9e6 | 3 / 3 | 44%–100% | 1.00 | 3 | 168k | $0.11 list | 0 | |
| 19 | openai/gpt-6.1-sol @mediumcodex · config b2396403 | 3 / 3 | 44%–100% | 1.00 | 3 | 88k | $0.04 list | 0 | |
| 20 | google/gemini-3.8-flash @highopencode · config 357da891 | 3 / 3 | 44%–100% | 1.00 | 3 | 454k | $0.20 list | 0 | |
| 21 | google/gemini-3.8-flash @highpi · config da41c66e | 3 / 3 | 44%–100% | 1.00 | 3 | 359k | $0.22 list | 0 | |
| 22 | google/gemini-3.8-flash @lowpi · config da41c66e | 3 / 3 | 44%–100% | 1.00 | 3 | 52k | $0.04 list | 0 | |
| 23 | google/gemini-3.8-flash @highantigravity · config 034ed3a1 | 5 / 6 | 44%–97% | 0.83 | 6 | 436k | $0.22 list | 0 | |
| 24 | fireworks/kimi-k3claude-code · config ecb67e4e | 5 / 6 | 44%–97% | 0.98 | 6 | 207k | $0.16 list | 0 | |
| 25 | google/gemini-3.8-flashclaude-code · config ecb67e4e | 5 / 6 | 44%–97% | 0.98 | 6 | 591k | $0.16 list | 0 | |
| 26 | together/deepseek-v4.1-flashclaude-code · config ecb67e4e | 5 / 6 | 44%–97% | 0.98 | 6 | 317k | $0.10 list | 4 | |
| 27 | zai/glm-5.2claude-code · config ecb67e4e | 5 / 6 | 44%–97% | 0.98 | 6 | 241k | $0.11 list | 0 | |
| 28 | baseten/glm-5.3-flashcodex · config 3437a9e6 | 5 / 6 | 44%–97% | 0.83 | 6 | 65k | $0.0095 list | 0 | |
| 29 | fireworks/qwen3.8-maxcodex · config 3437a9e6 | 5 / 6 | 44%–97% | 0.83 | 6 | 107k | $0.09 list | 0 | |
| 30 | google/gemini-3.8-flashcodex · config 3437a9e6 | 5 / 6 | 44%–97% | 0.98 | 6 | 240k | $0.16 list | 0 | |
| 31 | together/deepseek-v4.1-flashcodex · config 3437a9e6 | 5 / 6 | 44%–97% | 0.83 | 6 | 73k | $0.02 list | 0 | |
| 32 | baseten/glm-5.3-flashopencode · config 357da891 | 5 / 6 | 44%–97% | 0.98 | 6 | 75k | $0.01 list | 0 | |
| 33 | fireworks/kimi-k3opencode · config 357da891 | 5 / 6 | 44%–97% | 0.83 | 6 | 95k | $0.15 list | 0 | |
| 34 | fireworks/qwen3.8-maxopencode · config 357da891 | 5 / 6 | 44%–97% | 0.83 | 6 | 103k | $0.09 list | 0 | |
| 35 | google/gemini-3.8-flashopencode · config 357da891 | 5 / 6 | 44%–97% | 0.83 | 6 | 229k | $0.16 list | 0 | |
| 36 | together/deepseek-v4.1-flashopencode · config 357da891 | 5 / 6 | 44%–97% | 0.96 | 6 | 63k | $0.02 list | 0 | |
| 37 | zai/glm-5.2opencode · config 357da891 | 5 / 6 | 44%–97% | 0.98 | 6 | 91k | $0.06 list | 0 | |
| 38 | anthropic/claude-sonnet-5-5pi · config da41c66e | 5 / 6 | 44%–97% | 0.98 | 6 | 26k | $0.06 list | 0 | |
| 39 | openai/gpt-6.1-solpi · config da41c66e | 5 / 6 | 44%–97% | 0.97 | 6 | 29k | $0.03 list | 0 | |
| 40 | together/deepseek-v4.1-flashpi · config da41c66e | 5 / 6 | 44%–97% | 0.98 | 6 | 35k | $0.01 list | 0 | |
| 41 | zai/glm-5.2pi · config da41c66e | 5 / 6 | 44%–97% | 0.98 | 6 | 60k | $0.06 list | 0 | |
| 42 | fireworks/qwen3.8-maxclaude-code · config ecb67e4e | 4 / 6 | 30%–90% | 0.97 | 6 | 363k | $0.20 list | 0 | |
| 43 | zai/glm-5.2codex · config 3437a9e6 | 4 / 6 | 30%–90% | 0.97 | 6 | 116k | $0.10 list | 0 | |
| 44 | openai/gpt-6.1-solcodex · config b2396403 | 4 / 6 | 30%–90% | 0.97 | 6 | 121k | $0.05 list | 0 | |
| 45 | anthropic/claude-sonnet-5-5opencode · config 357da891 | 4 / 6 | 30%–90% | 0.97 | 6 | 87k | $0.08 list | 0 | |
| 46 | openai/gpt-6.1-solopencode · config 357da891 | 4 / 6 | 30%–90% | 0.96 | 6 | 125k | $0.09 list | 0 | |
| 47 | google/gemini-3.8-flashpi · config da41c66e | 4 / 6 | 30%–90% | 0.67 | 6 | 262k | $0.17 list | 0 | |
| 48 | moonshot/kimi-k3pi · config da41c66e | 4 / 6 | 30%–90% | 0.95 | 6 | 52k | $0.10 list | 0 | |
| 49 | openai/gpt-6.1-solclaude-code · config ecb67e4e | 2 / 3 | 21%–94% | 0.97 | 3 | 247k | $0.12 list | 0 | |
| 50 | google/gemini-3.8-flash @highcodex · config 3437a9e6 | 2 / 3 | 21%–94% | 0.97 | 3 | 652k | $0.38 list | 0 | |
| 51 | google/gemini-3.8-flash @lowopencode · config 357da891 | 2 / 3 | 21%–94% | 0.97 | 3 | 166k | $0.11 list | 0 | |
| 52 | baseten/glm-5.3-flashclaude-code · config ecb67e4e | 3 / 6 | 19%–81% | 0.95 | 6 | 461k | $0.06 list | 6 | |
| 53 | anthropic/claude-opus-5-5opencode · config 357da891 | 2 / 6 | 10%–70% | 0.94 | 6 | 170k | $0.34 list | 0 | |
| 54 | google/gemini-3.1-pro @highantigravity · config 034ed3a1 | 0 / 3 | 0%–56% | 1 | no passes | unknown | 0 |
list = published-rate · invoice = provider-invoice-reconciled