Board · task version 1
review
Each board ranks harness × model combinations on one suite by pass rate with a 95% Wilson interval; rows are not comparable across suites or task versions.
6 tasks
- review-clean-retention-plan
- review-corrupt-refresh-checkpoint
- review-publish-guard-order
- review-rescue-local-tip
- review-retry-empty-page
- review-stale-lock-recovery
review · task version 1
| # | Livery | model · harness · config | passed / n | 95% Wilson | mean | tasks | tokens / pass | cost / pass | excluded |
|---|---|---|---|---|---|---|---|---|---|
| 1 | google/gemini-3.8-flash @highantigravity · config 034ed3a1 | 9 / 9 | 70%–100% | 0.91 | 3 | 350k | $0.18 list | 0 | |
| 2 | together/deepseek-v4.1-flashopencode · config 357da891 | 8 / 8 | 68%–100% | 0.93 | 5 | 47k | $0.02 list | 4 | |
| 3 | google/gemini-3.1-pro @highantigravity · config 034ed3a1 | 6 / 6 | 61%–100% | 0.93 | 6 | not measured (0/6) | unknown | 12 | |
| 4 | anthropic/claude-opus-5-5claude-code · config c32a950b | 6 / 6 | 61%–100% | 0.93 | 6 | not measured (0/6) | unknown | 0 | |
| 5 | anthropic/claude-sonnet-5-5claude-code · config c32a950b | 6 / 6 | 61%–100% | 0.97 | 6 | not measured (0/6) | unknown | 0 | |
| 6 | together/deepseek-v4.1-flashclaude-code · config ecb67e4e | 6 / 6 | 61%–100% | 0.97 | 6 | 217k | $0.08 list | 0 | |
| 7 | openai/gpt-6.1-solcodex · config b2396403 | 6 / 6 | 61%–100% | 0.97 | 6 | not measured (0/6) | unknown | 0 | |
| 8 | google/gemini-4-argon @highantigravity · config 034ed3a1 | 3 / 3 | 44%–100% | 0.93 | 3 | 104k | $0.10 list | 0 | |
| 9 | baseten/glm-5.3-flashclaude-code · config ecb67e4e | 3 / 3 | 44%–100% | 0.93 | 3 | 234k | $0.03 list | 0 | |
| 10 | fireworks/kimi-k3claude-code · config ecb67e4e | 3 / 3 | 44%–100% | 0.93 | 3 | 136k | $0.06 list | 0 | |
| 11 | google/gemini-3.8-flashclaude-code · config ecb67e4e | 3 / 3 | 44%–100% | 0.93 | 3 | 417k | $0.13 list | 0 | |
| 12 | moonshot/kimi-k3claude-code · config ecb67e4e | 3 / 3 | 44%–100% | 0.93 | 3 | 176k | $0.10 list | 0 | |
| 13 | openai/gpt-6.1-solclaude-code · config ecb67e4e | 3 / 3 | 44%–100% | 0.87 | 3 | 145k | $0.07 list | 0 | |
| 14 | anthropic/claude-opus-5-5codex · config 3437a9e6 | 3 / 3 | 44%–100% | 0.87 | 3 | 51k | $0.21 list | 0 | |
| 15 | anthropic/claude-sonnet-5-5codex · config 3437a9e6 | 3 / 3 | 44%–100% | 0.93 | 3 | 44k | $0.09 list | 0 | |
| 16 | fireworks/kimi-k3codex · config 3437a9e6 | 3 / 3 | 44%–100% | 0.93 | 3 | 31k | $0.05 list | 0 | |
| 17 | fireworks/qwen3.8-maxcodex · config 3437a9e6 | 3 / 3 | 44%–100% | 0.87 | 3 | 56k | $0.05 list | 3 | |
| 18 | google/gemini-3.8-flashcodex · config 3437a9e6 | 3 / 3 | 44%–100% | 0.93 | 3 | 346k | $0.19 list | 0 | |
| 19 | moonshot/kimi-k3codex · config 3437a9e6 | 3 / 3 | 44%–100% | 0.93 | 3 | 36k | $0.04 list | 0 | |
| 20 | openai/gpt-6.1-sol @highcodex · config b2396403 | 3 / 3 | 44%–100% | 0.87 | 3 | 76k | $0.04 list | 2 | |
| 21 | anthropic/claude-opus-5-5opencode · config 357da891 | 3 / 3 | 44%–100% | 0.87 | 3 | 56k | $0.09 list | 0 | |
| 22 | anthropic/claude-sonnet-5-5opencode · config 357da891 | 3 / 3 | 44%–100% | 0.93 | 3 | 57k | $0.04 list | 0 | |
| 23 | baseten/glm-5.3-flashopencode · config 357da891 | 3 / 3 | 44%–100% | 0.93 | 3 | 48k | $0.0061 list | 0 | |
| 24 | fireworks/kimi-k3opencode · config 357da891 | 3 / 3 | 44%–100% | 0.93 | 3 | 59k | $0.10 list | 0 | |
| 25 | fireworks/qwen3.8-maxopencode · config 357da891 | 3 / 3 | 44%–100% | 0.87 | 3 | 69k | $0.06 list | 0 | |
| 26 | google/gemini-3.8-flashopencode · config 357da891 | 3 / 3 | 44%–100% | 0.87 | 3 | 117k | $0.09 list | 0 | |
| 27 | moonshot/kimi-k3opencode · config 357da891 | 3 / 3 | 44%–100% | 0.93 | 3 | 59k | $0.08 list | 0 | |
| 28 | openai/gpt-6.1-solopencode · config 357da891 | 3 / 3 | 44%–100% | 0.87 | 3 | 62k | $0.05 list | 0 | |
| 29 | anthropic/claude-opus-5-5pi · config da41c66e | 3 / 3 | 44%–100% | 0.87 | 3 | 22k | $0.10 list | 0 | |
| 30 | anthropic/claude-sonnet-5-5pi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 18k | $0.04 list | 0 | |
| 31 | baseten/glm-5.3-flashpi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 17k | $0.0027 list | 0 | |
| 32 | fireworks/kimi-k3pi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 18k | $0.05 list | 0 | |
| 33 | fireworks/qwen3.8-maxpi · config da41c66e | 3 / 3 | 44%–100% | 0.87 | 3 | 41k | $0.04 list | 0 | |
| 34 | google/gemini-3.8-flashpi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 111k | $0.11 list | 0 | |
| 35 | google/gemini-3.8-flash @highpi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 188k | $0.15 list | 0 | |
| 36 | moonshot/kimi-k3pi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 23k | $0.05 list | 0 | |
| 37 | openai/gpt-6.1-solpi · config da41c66e | 3 / 3 | 44%–100% | 0.93 | 3 | 19k | $0.02 list | 0 | |
| 38 | fireworks/qwen3.8-maxclaude-code · config ecb67e4e | 5 / 6 | 44%–97% | 0.83 | 6 | not measured (0/6) | unknown | 6 | |
| 39 | zai/glm-5.2codex · config 3437a9e6 | 5 / 6 | 44%–97% | 0.90 | 6 | 315k | $0.39 list | 0 | |
| 40 | fireworks/qwen3.8-27bclaude-code · config ecb67e4e | 4 / 5 | 38%–96% | 0.84 | 5 | not measured (0/5) | unknown | 1 | |
| 41 | zai/glm-5.2claude-code · config ecb67e4e | 4 / 6 | 30%–90% | 0.80 | 6 | not measured (0/6) | unknown | 1 | |
| 42 | zai/glm-5.2pi · config da41c66e | 4 / 6 | 30%–90% | 0.80 | 6 | 236k | $0.28 list | 0 | |
| 43 | baseten/glm-5.3-flashcodex · config 3437a9e6 | 2 / 3 | 21%–94% | 0.85 | 3 | 60k | $0.0094 list | 0 | |
| 44 | zai/glm-5.2opencode · config 357da891 | 3 / 6 | 19%–81% | 0.63 | 6 | 243k | $0.23 list | 0 |
list = published-rate · invoice = provider-invoice-reconciled