Board · task version 2

hard-review

Each board ranks harness × model combinations on one suite by pass rate with a 95% Wilson interval; rows are not comparable across suites or task versions.

9 tasks
  • hard-review-clean-long
  • hard-review-cross-file
  • hard-review-cross-file-v2
  • hard-review-decoy-decision-replies
  • hard-review-decoy-job-leases
  • hard-review-multi-defect-outbox
  • hard-review-multi-defect-outbox-v2
  • hard-review-multi-defect-tool-calls
  • hard-review-multi-defect-tool-calls-v2
hard-review · task version 2
#Liverymodel · harness · configpassed / n95% Wilsonmeantaskstokens / passcost / passexcluded
1anthropic/claude-sonnet-5-5 @maxcodex · config 3437a9e64 / 4 51%–100%1.003892k$3.11 list11
2anthropic/claude-haiku-5-5claude-code · config c32a950b2 / 2 34%–100%0.902788k$0.03 list1
3anthropic/claude-opus-5-5 @maxopencode · config 357da8911 / 1 21%–100%1.0011.02M$4.58 list1
4anthropic/claude-opus-5-5 @maxpi · config da41c66e1 / 1 21%–100%1.001427k$3.22 list0
5together/deepseek-v4.1-flashpi · config da41c66e31 / 32 84%–99%0.909216k$0.08 list2
6zai/glm-5.2codex · config 3437a9e620 / 22 72%–97%0.879482k$0.52 list0
7together/deepseek-v4.1-flashopencode · config 357da89110 / 11 62%–98%0.859142k$0.05 list0
8zai/glm-5.2claude-code · config ecb67e4e9 / 10 60%–98%0.8591.37M$0.64 list9
9together/deepseek-v4.1-flashcodex · config 3437a9e69 / 10 60%–98%0.899216k$0.09 list0
10fireworks/kimi-k3opencode · config 357da8919 / 10 60%–98%0.879213k$0.36 list0
11zai/glm-5.2opencode · config 357da8919 / 10 60%–98%0.849268k$0.24 list0
12fireworks/kimi-k3pi · config da41c66e9 / 10 60%–98%0.839142k$0.29 list0
13moonshot/kimi-k3pi · config da41c66e9 / 10 60%–98%0.829148k$0.29 list0
14google/gemini-3.8-flash @highantigravity · config 034ed3a124 / 27 72%–96%0.9291.67M$0.58 list0
15anthropic/claude-opus-5-5 @maxclaude-code · config c32a950b8 / 9 57%–98%0.9486.10M$4.53 list0
16anthropic/claude-opus-5-5 @highcodex · config 3437a9e68 / 9 57%–98%0.968130k$0.59 list0
17google/gemini-3.8-flash @mediumantigravity · config 034ed3a17 / 8 53%–98%0.9481.50M$0.48 list0
18google/gemini-4-argon @mediumantigravity · config 034ed3a17 / 8 53%–98%0.898144k$0.18 list0
19anthropic/claude-sonnet-5-5 @maxclaude-code · config c32a950b7 / 8 53%–98%0.94812.51M$3.98 list0
20anthropic/claude-sonnet-5-5 @xhighclaude-code · config c32a950b7 / 8 53%–98%0.8881.91M$0.60 list0
21google/gemini-3.8-flash @mediumclaude-code · config ecb67e4e7 / 8 53%–98%0.928938k$0.38 list0
22openai/gpt-5.6-terra @lowclaude-code · config ecb67e4e7 / 8 53%–98%0.858197k$0.14 list2
23openai/gpt-6-luna @highclaude-code · config ecb67e4e7 / 8 53%–98%0.848332k$0.01 list0
24openai/gpt-6.1-sol @lowclaude-code · config ecb67e4e7 / 8 53%–98%0.988263k$0.11 list8
25google/gemini-3.8-flash @highcodex · config 3437a9e67 / 8 53%–98%1.0081.72M$0.59 list1
26openai/gpt-6-luna @xhighcodex · config b23964037 / 8 53%–98%0.908242k$0.0093 list0
27openai/gpt-6.1-sol @lowcodex · config b23964037 / 8 53%–98%0.968157k$0.07 list0
28anthropic/claude-opus-5-5 @xhighopencode · config 357da8917 / 8 53%–98%0.938378k$0.89 list7
29anthropic/claude-sonnet-5-5 @xhighopencode · config 357da8917 / 8 53%–98%0.898279k$0.35 list1
30google/gemini-3.8-flash @highopencode · config 357da8917 / 8 53%–98%0.9481.88M$0.55 list0
31google/gemini-3.8-flash @mediumopencode · config 357da8917 / 8 53%–98%0.9481.12M$0.47 list2
32openai/gpt-6-luna @lowopencode · config 357da8917 / 8 53%–98%0.86897k$0.0039 list0
33anthropic/claude-opus-5-5 @xhighpi · config da41c66e7 / 8 53%–98%0.918157k$0.81 list0
34together/deepseek-v4.1-flashclaude-code · config ecb67e4e19 / 22 67%–95%0.889705k$0.25 list0
35baseten/glm-5.3-flashopencode · config 357da89119 / 22 67%–95%0.879235k$0.03 list1
36anthropic/claude-opus-5-5 @maxcodex · config 3437a9e66 / 7 49%–97%0.906886k$5.72 list6
37google/gemini-3.8-flashpi · config da41c66e17 / 20 64%–95%0.9391.12M$0.46 list7
38google/gemini-3.8-flash @highpi · config da41c66e10 / 12 55%–95%0.9291.67M$0.62 list3
39openai/gpt-6.1-sol @mediumcodex · config b239640356 / 68 72%–90%0.929170k$0.09 list1
40baseten/glm-5.3-flashpi · config da41c66e18 / 22 61%–93%0.869189k$0.03 list2
41fireworks/qwen3.8-maxpi · config da41c66e17 / 21 60%–92%0.879331k$0.28 list0
42fireworks/kimi-k3claude-code · config ecb67e4e8 / 10 49%–94%0.869290k$0.24 list0
43google/gemini-3.8-flashcodex · config 3437a9e68 / 10 49%–94%0.9191.11M$0.44 list1
44anthropic/claude-sonnet-5-5opencode · config 357da8918 / 10 49%–94%0.819151k$0.13 list1
45moonshot/kimi-k3opencode · config 357da8918 / 10 49%–94%0.809229k$0.37 list0
46openai/gpt-6.1-solpi · config da41c66e8 / 10 49%–94%0.93986k$0.08 list1
47zai/glm-5.2pi · config da41c66e8 / 10 49%–94%0.819325k$0.30 list0
48anthropic/claude-opus-5-5 @xhighclaude-code · config c32a950b22 / 28 60%–90%0.9182.26M$1.43 list3
49google/gemini-4-argon @highantigravity · config 034ed3a17 / 9 45%–94%0.869310k$0.50 list0
50anthropic/claude-opus-5-5 @xhighcodex · config 3437a9e67 / 9 45%–94%0.908269k$1.36 list0
51anthropic/claude-sonnet-5-5 @xhighcodex · config 3437a9e67 / 9 45%–94%0.868227k$0.56 list0
52fireworks/qwen3.8-maxcodex · config 3437a9e67 / 9 45%–94%0.919369k$0.39 list1
53fireworks/qwen3.8-maxopencode · config 357da8917 / 9 45%–94%0.929359k$0.32 list0
54openai/gpt-6-luna @xhighopencode · config 357da8917 / 9 45%–94%0.888441k$0.02 list0
55openai/gpt-6.1-sol @lowopencode · config 357da8917 / 9 45%–94%0.948121k$0.09 list0
56openai/gpt-6.1-sol @lowpi · config da41c66e7 / 9 45%–94%0.93863k$0.05 list0
57google/gemini-4-argon @lowantigravity · config 034ed3a16 / 8 41%–93%0.878153k$0.12 list0
58openai/gpt-5.6-terra @highclaude-code · config ecb67e4e6 / 8 41%–93%0.868296k$0.27 list0
59google/gemini-3.8-flash @mediumcodex · config 3437a9e66 / 8 41%–93%0.8981.26M$0.47 list0
60openai/gpt-5.6-terra @highcodex · config b23964036 / 8 41%–93%0.888343k$0.20 list1
61openai/gpt-5.6-terra @mediumcodex · config b23964036 / 8 41%–93%0.838258k$0.15 list0
62openai/gpt-5.6-terra @xhighcodex · config b23964036 / 8 41%–93%0.868366k$0.27 list0
63openai/gpt-6-luna @highcodex · config b23964036 / 8 41%–93%0.828226k$0.0063 list0
64openai/gpt-6-luna @maxcodex · config b23964036 / 8 41%–93%0.918264k$0.01 list0
65anthropic/claude-haiku-5-5 @mediumopencode · config 357da8916 / 8 41%–93%0.828183k$0.0080 list1
66openai/gpt-5.6-terra @lowopencode · config 357da8916 / 8 41%–93%0.878194k$0.13 list0
67openai/gpt-6-luna @highopencode · config 357da8916 / 8 41%–93%0.898282k$0.01 list0
68anthropic/claude-haiku-5-5 @highpi · config da41c66e6 / 8 41%–93%0.798151k$0.02 list0
69anthropic/claude-sonnet-5-5 @highpi · config da41c66e6 / 8 41%–93%0.83870k$0.17 list0
70anthropic/claude-sonnet-5-5 @xhighpi · config da41c66e6 / 8 41%–93%0.81893k$0.26 list0
71google/gemini-3.8-flash @mediumpi · config da41c66e6 / 8 41%–93%0.9081.35M$0.44 list0
72openai/gpt-6-luna @highpi · config da41c66e6 / 8 41%–93%0.848106k$0.0076 list0
73openai/gpt-6.1-solcodex · config b239640323 / 32 55%–84%0.919196k$0.11 list0
74google/gemini-3.8-flash @lowpi · config da41c66e5 / 7 36%–92%0.827434k$0.17 list1
75fireworks/qwen3.8-maxclaude-code · config ecb67e4e17 / 24 51%–85%0.889430k$0.35 list5
76baseten/glm-5.3-flashclaude-code · config ecb67e4e7 / 10 40%–89%0.8691.19M$0.09 list0
77moonshot/kimi-k3claude-code · config ecb67e4e7 / 10 40%–89%0.799485k$0.34 list0
78openai/gpt-6.1-solclaude-code · config ecb67e4e7 / 10 40%–89%0.929387k$0.21 list0
79google/gemini-3.8-flashopencode · config 357da8917 / 10 40%–89%0.8591.30M$0.52 list1
80fireworks/kimi-k3codex · config 3437a9e615 / 22 47%–84%0.799175k$0.25 list0
81openai/gpt-6.1-sol @highcodex · config b239640339 / 58 54%–78%0.909230k$0.13 list9
82openai/gpt-6.1-sol @highclaude-code · config ecb67e4e6 / 9 35%–88%0.948352k$0.21 list0
83anthropic/claude-haiku-5-5 @xhighcodex · config 3437a9e66 / 9 35%–88%0.778498k$0.07 list0
84anthropic/claude-opus-5-5codex · config 3437a9e66 / 9 35%–88%0.929136k$0.61 list0
85anthropic/claude-opus-5-5 @mediumcodex · config 3437a9e66 / 9 35%–88%0.908150k$0.66 list0
86anthropic/claude-opus-5-5opencode · config 357da8916 / 9 35%–88%0.909144k$0.29 list0
87openai/gpt-6.1-sol @xhighcodex · config b239640321 / 33 47%–78%0.919286k$0.18 list0
88google/gemini-3.8-flash @lowantigravity · config 034ed3a15 / 8 31%–86%0.858881k$0.22 list1
89anthropic/claude-haiku-5-5 @highclaude-code · config c32a950b5 / 8 31%–86%0.7781.31M$0.05 list0
90anthropic/claude-haiku-5-5 @lowclaude-code · config c32a950b5 / 8 31%–86%0.818766k$0.02 list0
91anthropic/claude-haiku-5-5 @mediumclaude-code · config c32a950b5 / 8 31%–86%0.7981.03M$0.03 list1
92anthropic/claude-haiku-5-5 @xhighclaude-code · config c32a950b5 / 8 31%–86%0.8182.62M$0.12 list0
93google/gemini-3.8-flash @highclaude-code · config ecb67e4e5 / 8 31%–86%0.9082.24M$0.67 list0
94google/gemini-3.8-flash @lowclaude-code · config ecb67e4e5 / 8 31%–86%0.818542k$0.20 list0
95openai/gpt-6-luna @lowclaude-code · config ecb67e4e5 / 8 31%–86%0.868412k$0.0099 list0
96openai/gpt-6-luna @xhighclaude-code · config ecb67e4e5 / 8 31%–86%0.848310k$0.02 list0
97anthropic/claude-haiku-5-5 @lowcodex · config 3437a9e65 / 8 31%–86%0.808174k$0.02 list0
98anthropic/claude-sonnet-5-5 @highcodex · config 3437a9e65 / 8 31%–86%0.848160k$0.35 list3
99google/gemini-3.8-flash @lowcodex · config 3437a9e65 / 8 31%–86%0.878784k$0.28 list0
100openai/gpt-5.6-terra @lowcodex · config b23964035 / 8 31%–86%0.798322k$0.18 list0
101openai/gpt-5.6-terra @maxcodex · config b23964035 / 8 31%–86%0.828676k$0.51 list0
102openai/gpt-6.1-sol @maxcodex · config b23964035 / 8 31%–86%0.918350k$0.20 list0
103anthropic/claude-haiku-5-5 @highopencode · config 357da8915 / 8 31%–86%0.768246k$0.01 list0
104anthropic/claude-haiku-5-5 @lowopencode · config 357da8915 / 8 31%–86%0.788175k$0.0087 list0
105openai/gpt-5.6-terra @highopencode · config 357da8915 / 8 31%–86%0.818334k$0.25 list0
106anthropic/claude-haiku-5-5 @lowpi · config da41c66e5 / 8 31%–86%0.898110k$0.01 list0
107anthropic/claude-haiku-5-5 @mediumpi · config da41c66e5 / 8 31%–86%0.758131k$0.02 list0
108anthropic/claude-opus-5-5 @mediumpi · config da41c66e5 / 8 31%–86%0.84885k$0.41 list0
109openai/gpt-5.6-terra @highpi · config da41c66e5 / 8 31%–86%0.81895k$0.15 list0
110openai/gpt-5.6-terra @lowpi · config da41c66e5 / 8 31%–86%0.77872k$0.08 list0
111openai/gpt-6-luna @maxpi · config da41c66e5 / 8 31%–86%0.828281k$0.05 list3
112anthropic/claude-sonnet-5-5claude-code · config c32a950b6 / 10 31%–83%0.8191.49M$0.33 list0
113google/gemini-3.8-flashclaude-code · config ecb67e4e6 / 10 31%–83%0.9392.42M$0.72 list0
114baseten/glm-5.3-flashcodex · config 3437a9e66 / 10 31%–83%0.809271k$0.04 list0
115moonshot/kimi-k3codex · config 3437a9e66 / 10 31%–83%0.759243k$0.35 list0
116anthropic/claude-sonnet-5-5pi · config da41c66e6 / 10 31%–83%0.79994k$0.23 list1
117anthropic/claude-haiku-5-5 @mediumcodex · config 3437a9e65 / 9 27%–81%0.728225k$0.02 list0
118openai/gpt-6.1-sol @ultracodex · config b23964035 / 9 27%–81%0.928504k$0.20 list0
119anthropic/claude-opus-5-5 @highopencode · config 357da8915 / 9 27%–81%0.828270k$0.54 list0
120openai/gpt-6.1-sol @highopencode · config 357da8915 / 9 27%–81%0.928406k$0.23 list0
121openai/gpt-6.1-sol @maxopencode · config 357da8915 / 9 27%–81%0.908562k$0.59 list0
122anthropic/claude-opus-5-5pi · config da41c66e5 / 9 27%–81%0.909105k$0.52 list0
123anthropic/claude-opus-5-5 @highpi · config da41c66e5 / 9 27%–81%0.888110k$0.54 list0
124openai/gpt-6.1-sol @highpi · config da41c66e5 / 9 27%–81%0.908138k$0.12 list4
125openai/gpt-6.1-sol @maxpi · config da41c66e5 / 9 27%–81%0.928161k$0.26 list1
126openai/gpt-6.1-solopencode · config 357da89112 / 22 35%–73%0.899399k$0.25 list0
127anthropic/claude-opus-5-5claude-code · config c32a950b17 / 34 34%–66%0.8792.27M$0.95 list1
128anthropic/claude-sonnet-5-5codex · config 3437a9e65 / 10 24%–76%0.749161k$0.35 list0
129anthropic/claude-haiku-5-5 @maxclaude-code · config c32a950b4 / 8 22%–78%0.8288.31M$0.39 list0
130anthropic/claude-opus-5-5 @highclaude-code · config c32a950b4 / 8 22%–78%0.8981.91M$0.83 list2
131anthropic/claude-opus-5-5 @mediumclaude-code · config c32a950b4 / 8 22%–78%0.8981.77M$0.66 list1
132anthropic/claude-sonnet-5-5 @highclaude-code · config c32a950b4 / 8 22%–78%0.8081.87M$0.40 list0
133openai/gpt-6-luna @maxclaude-code · config ecb67e4e4 / 8 22%–78%0.858401k$0.04 list0
134openai/gpt-6.1-sol @maxclaude-code · config ecb67e4e4 / 8 22%–78%0.918612k$0.54 list5
135openai/gpt-6-luna @mediumcodex · config b23964034 / 8 22%–78%0.808268k$0.0069 list0
136anthropic/claude-haiku-5-5 @xhighopencode · config 357da8914 / 8 22%–78%0.758795k$0.08 list0
137anthropic/claude-opus-5-5 @mediumopencode · config 357da8914 / 8 22%–78%0.848207k$0.46 list0
138anthropic/claude-sonnet-5-5 @highopencode · config 357da8914 / 8 22%–78%0.778232k$0.22 list0
139google/gemini-3.8-flash @lowopencode · config 357da8914 / 8 22%–78%0.818422k$0.13 list0
140anthropic/claude-sonnet-5-5 @mediumpi · config da41c66e4 / 8 22%–78%0.83894k$0.21 list0
141openai/gpt-6-luna @lowpi · config da41c66e4 / 8 22%–78%0.85877k$0.0038 list0
142openai/gpt-6-luna @xhighpi · config da41c66e4 / 8 22%–78%0.828198k$0.02 list1
143google/gemini-3.1-pro @highantigravity · config 034ed3a14 / 9 19%–73%0.709862kunknown0
144openai/gpt-6-luna @maxopencode · config 357da8914 / 9 19%–73%0.798759k$0.08 list0
145anthropic/claude-sonnet-5-5 @lowclaude-code · config c32a950b3 / 8 14%–69%0.7882.29M$0.40 list0
146anthropic/claude-sonnet-5-5 @mediumclaude-code · config c32a950b3 / 8 14%–69%0.8282.14M$0.42 list4
147anthropic/claude-haiku-5-5 @highcodex · config 3437a9e63 / 8 14%–69%0.648604k$0.07 list0
148openai/gpt-5.6-terra @ultracodex · config b23964033 / 8 14%–69%0.7282.05M$1.01 list0
149anthropic/claude-haiku-5-5 @maxopencode · config 357da8913 / 8 14%–69%0.8183.49M$0.50 list0
150anthropic/claude-haiku-5-5 @maxpi · config da41c66e3 / 8 14%–69%0.7082.15M$0.42 list0
151anthropic/claude-haiku-5-5 @xhighpi · config da41c66e3 / 8 14%–69%0.728900k$0.13 list0
152anthropic/claude-haiku-5-5 @maxcodex · config 3437a9e63 / 9 12%–65%0.7482.28M$0.43 list9
153anthropic/claude-opus-5-5 @lowcodex · config 3437a9e63 / 9 12%–65%0.818244k$1.04 list1
154anthropic/claude-opus-5-5 @lowopencode · config 357da8913 / 9 12%–65%0.868281k$0.47 list0
155anthropic/claude-opus-5-5 @lowclaude-code · config c32a950b2 / 8 7%–59%0.8283.25M$1.09 list3
156anthropic/claude-sonnet-5-5 @lowcodex · config 3437a9e62 / 8 7%–59%0.748353k$0.75 list0
157anthropic/claude-sonnet-5-5 @mediumopencode · config 357da8912 / 8 7%–59%0.758461k$0.36 list2
158anthropic/claude-sonnet-5-5 @lowpi · config da41c66e2 / 8 7%–59%0.778172k$0.39 list0
159anthropic/claude-sonnet-5-5 @mediumcodex · config 3437a9e62 / 9 6%–55%0.778450k$0.96 list0
160anthropic/claude-opus-5-5 @lowpi · config da41c66e2 / 9 6%–55%0.848206k$0.93 list0
161openai/gpt-6-luna @lowcodex · config b23964031 / 8 2%–47%0.7381.00M$0.03 list0
162anthropic/claude-sonnet-5-5 @lowopencode · config 357da8911 / 8 2%–47%0.688921k$0.67 list0
list = published-rate · invoice = provider-invoice-reconciled