Board · task version 1

docs

Each board ranks harness × model combinations on one suite by pass rate with a 95% Wilson interval; rows are not comparable across suites or task versions.

6 tasks

docs-cache-config
docs-config-loader-keys
docs-gym-baseline-keys
docs-gym-run-flags
docs-retry-behavior-diff
docs-snapshot-new-command

docs · task version 1

#	model · harness · config	passed / n	95% Wilson	mean	tasks	tokens / pass	cost / pass	excluded
1	google/gemini-3.8-flashpi · config `da41c66e`	4 / 6	30%–90%	0.94	6	262k	$0.17 list	0
2	google/gemini-3.8-flash @highantigravity · config `034ed3a1`	3 / 6	19%–81%	0.94	6	726k	$0.37 list	0
3	anthropic/claude-opus-5-5opencode · config `357da891`	3 / 6	19%–81%	0.91	6	107k	$0.22 list	1
4	zai/glm-5.2pi · config `da41c66e`	3 / 6	19%–81%	0.91	6	74k	$0.09 list	0
5	fireworks/qwen3.8-maxclaude-code · config `ecb67e4e`	7 / 18	20%–61%	0.93	6	not measured (0/18)	unknown	12
6	anthropic/claude-sonnet-5-5claude-code · config `c32a950b`	4 / 12	14%–61%	0.92	6	not measured (0/12)	unknown	6
7	fireworks/kimi-k3claude-code · config `ecb67e4e`	2 / 6	10%–70%	0.89	6	497k	$0.21 list	0
8	anthropic/claude-opus-5-5codex · config `3437a9e6`	2 / 6	10%–70%	0.92	6	135k	$0.60 list	0
9	fireworks/kimi-k3codex · config `3437a9e6`	2 / 6	10%–70%	0.91	6	129k	$0.19 list	0
10	fireworks/qwen3.8-maxopencode · config `357da891`	2 / 6	10%–70%	0.93	6	257k	$0.21 list	0
11	google/gemini-3.8-flashopencode · config `357da891`	2 / 6	10%–70%	0.91	6	572k	$0.39 list	0
12	anthropic/claude-opus-5-5pi · config `da41c66e`	2 / 6	10%–70%	0.89	6	57k	$0.29 list	0
13	anthropic/claude-sonnet-5-5pi · config `da41c66e`	2 / 6	10%–70%	0.89	6	60k	$0.15 list	0
14	anthropic/claude-opus-5-5claude-code · config `c32a950b`	3 / 13	8%–50%	0.88	6	not measured (0/13)	unknown	16
15	openai/gpt-6.1-solcodex · config `b2396403`	3 / 15	7%–45%	0.90	6	not measured (0/15)	unknown	12
16	google/gemini-3.1-pro @highantigravity · config `034ed3a1`	3 / 18	6%–39%	0.86	6	not measured (0/18)	unknown	0
17	zai/glm-5.2claude-code · config `ecb67e4e`	2 / 12	5%–45%	0.86	6	not measured (0/12)	unknown	0
18	together/deepseek-v4.1-flashclaude-code · config `ecb67e4e`	1 / 6	3%–56%	0.87	6	1.06M	$0.33 list	0
19	anthropic/claude-sonnet-5-5codex · config `3437a9e6`	1 / 6	3%–56%	0.83	6	216k	$0.47 list	0
20	moonshot/kimi-k3codex · config `3437a9e6`	1 / 6	3%–56%	0.89	6	332k	$0.35 list	0
21	zai/glm-5.2codex · config `3437a9e6`	1 / 6	3%–56%	0.85	6	430k	$0.33 list	0
22	anthropic/claude-sonnet-5-5opencode · config `357da891`	1 / 6	3%–56%	0.83	6	380k	$0.34 list	0
23	baseten/glm-5.3-flashopencode · config `357da891`	1 / 6	3%–56%	0.89	6	361k	$0.04 list	0
24	moonshot/kimi-k3opencode · config `357da891`	1 / 6	3%–56%	0.89	6	423k	$0.51 list	0
25	zai/glm-5.2opencode · config `357da891`	1 / 6	3%–56%	0.87	6	479k	$0.27 list	0
26	fireworks/kimi-k3pi · config `da41c66e`	1 / 6	3%–56%	0.89	6	217k	$0.47 list	0
27	moonshot/kimi-k3pi · config `da41c66e`	1 / 6	3%–56%	0.89	6	174k	$0.38 list	0
28	together/deepseek-v4.1-flashopencode · config `357da891`	1 / 12	1%–35%	0.83	6	607k	$0.19 list	0
29	anthropic/claude-sonnet-5claude-code · config `c32a950b`	0 / 12	0%–24%	0.84	6	no passes	unknown	6
30	anthropic/claude-haiku-4-5claude-code · config `c32a950b`	0 / 6	0%–39%	0.78	6	no passes	unknown	6
31	baseten/glm-5.3-flashclaude-code · config `ecb67e4e`	0 / 6	0%–39%	0.87	6	no passes	unknown	0
32	baseten/glm-5.3-flashcodex · config `3437a9e6`	0 / 6	0%–39%	0.82	6	no passes	unknown	0
33	fireworks/qwen3.8-maxcodex · config `3437a9e6`	0 / 6	0%–39%	0.80	6	no passes	unknown	6
34	fireworks/kimi-k3opencode · config `357da891`	0 / 6	0%–39%	0.83	6	no passes	unknown	0
35	baseten/glm-5.3-flashpi · config `da41c66e`	0 / 6	0%–39%	0.87	6	no passes	unknown	0
36	fireworks/qwen3.8-maxpi · config `da41c66e`	0 / 6	0%–39%	0.84	6	no passes	unknown	0
37	moonshot/kimi-k3claude-code · config `ecb67e4e`	0 / 2	0%–66%	0.83	2	no passes	unknown	0

list = published-rate · invoice = provider-invoice-reconciled