507 execution attempts across two waves. Each one links to the commands the agent actually ran, the output the platform actually returned, and the verifier's decision. Nothing here is summarised — the summaries are on the issue page, and this is what they were made from.
Boundary-axis attempts (d1_edge_fn, d2_oauth) cannot pass by construction and are marked.
| Model | Passed | Attempts |
|---|---|---|
| openai/gpt-5.6-sol | 33 | 39 |
| openai/gpt-5.6-luna | 33 | 39 |
| z-ai/glm-5.2 | 32 | 39 |
| claude-opus-4-8 | 32 | 39 |
| claude-sonnet-5 | 31 | 39 |
| google/gemini-3.1-pro-preview | 31 | 39 |
| moonshotai/kimi-k3 | 31 | 39 |
| claude-opus-5 | 31 | 39 |
| tencent/hy3 | 31 | 39 |
| xiaomi/mimo-v2.5 | 30 | 39 |
| deepseek/deepseek-v4-pro | 29 | 39 |
| minimax/minimax-m3 | 29 | 39 |
| qwen/qwen3-coder-plus | 22 | 39 |
| Model | Task | Rep | Verdict | Steps | Cost | Evidence |
|---|---|---|---|---|---|---|
| claude-opus-4-8 | d1_auth_email | 1 | pass | 3 | $0.090 | open → |
| claude-opus-4-8 | d1_auth_email | 2 | pass | 3 | $0.083 | open → |
| claude-opus-4-8 | d1_auth_email | 3 | pass | 3 | $0.087 | open → |
| claude-opus-5 | d1_auth_email | 1 | pass | 10 | $0.354 | open → |
| claude-opus-5 | d1_auth_email | 2 | pass | 8 | $0.263 | open → |
| claude-opus-5 | d1_auth_email | 3 | pass | 8 | $0.263 | open → |
| claude-sonnet-5 | d1_auth_email | 1 | pass | 3 | $0.075 | open → |
| claude-sonnet-5 | d1_auth_email | 2 | pass | 8 | $0.164 | open → |
| claude-sonnet-5 | d1_auth_email | 3 | pass | 5 | $0.121 | open → |
| deepseek/deepseek-v4-pro | d1_auth_email | 1 | pass | 7 | $0.007 | open → |
| deepseek/deepseek-v4-pro | d1_auth_email | 2 | pass | 7 | $0.006 | open → |
| deepseek/deepseek-v4-pro | d1_auth_email | 3 | pass | 5 | $0.004 | open → |
| google/gemini-3.1-pro-preview | d1_auth_email | 1 | pass | 5 | $0.080 | open → |
| google/gemini-3.1-pro-preview | d1_auth_email | 2 | pass | 5 | $0.080 | open → |
| google/gemini-3.1-pro-preview | d1_auth_email | 3 | pass | 5 | $0.080 | open → |
| minimax/minimax-m3 | d1_auth_email | 1 | pass | 5 | $0.008 | open → |
| minimax/minimax-m3 | d1_auth_email | 2 | pass | 8 | $0.014 | open → |
| minimax/minimax-m3 | d1_auth_email | 3 | pass | 7 | $0.012 | open → |
| moonshotai/kimi-k3 | d1_auth_email | 1 | pass | 4 | $0.056 | open → |
| moonshotai/kimi-k3 | d1_auth_email | 2 | pass | 5 | $0.072 | open → |
| moonshotai/kimi-k3 | d1_auth_email | 3 | pass | 7 | $0.059 | open → |
| openai/gpt-5.6-luna | d1_auth_email | 1 | pass | 3 | $0.008 | open → |
| openai/gpt-5.6-luna | d1_auth_email | 2 | pass | 3 | $0.011 | open → |
| openai/gpt-5.6-luna | d1_auth_email | 3 | pass | 3 | $0.014 | open → |
| openai/gpt-5.6-sol | d1_auth_email | 1 | pass | 1 | $0.040 | open → |
| openai/gpt-5.6-sol | d1_auth_email | 2 | pass | 1 | $0.037 | open → |
| openai/gpt-5.6-sol | d1_auth_email | 3 | pass | 1 | $0.049 | open → |
| qwen/qwen3-coder-plus | d1_auth_email | 1 | pass | 9 | $0.064 | open → |
| qwen/qwen3-coder-plus | d1_auth_email | 2 | pass | 11 | $0.105 | open → |
| qwen/qwen3-coder-plus | d1_auth_email | 3 | pass | 9 | $0.060 | open → |
| tencent/hy3 | d1_auth_email | 1 | pass | 4 | $0.003 | open → |
| tencent/hy3 | d1_auth_email | 2 | pass | 4 | $0.003 | open → |
| tencent/hy3 | d1_auth_email | 3 | pass | 3 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d1_auth_email | 1 | pass | 4 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d1_auth_email | 2 | pass | 8 | $0.005 | open → |
| xiaomi/mimo-v2.5 | d1_auth_email | 3 | pass | 6 | $0.003 | open → |
| z-ai/glm-5.2 | d1_auth_email | 1 | pass | 5 | $0.010 | open → |
| z-ai/glm-5.2 | d1_auth_email | 2 | pass | 8 | $0.022 | open → |
| z-ai/glm-5.2 | d1_auth_email | 3 | pass | 5 | $0.011 | open → |
| claude-opus-4-8 | d1_crud | 1 | pass | 7 | $0.255 | open → |
| claude-opus-4-8 | d1_crud | 2 | pass | 5 | $0.132 | open → |
| claude-opus-4-8 | d1_crud | 3 | pass | 5 | $0.174 | open → |
| claude-opus-5 | d1_crud | 1 | pass | 15 | $0.776 | open → |
| claude-opus-5 | d1_crud | 2 | pass | 11 | $0.392 | open → |
| claude-opus-5 | d1_crud | 3 | pass | 12 | $0.361 | open → |
| claude-sonnet-5 | d1_crud | 1 | pass | 5 | $0.084 | open → |
| claude-sonnet-5 | d1_crud | 2 | pass | 5 | $0.073 | open → |
| claude-sonnet-5 | d1_crud | 3 | pass | 6 | $0.109 | open → |
| deepseek/deepseek-v4-pro | d1_crud | 1 | pass | 6 | $0.005 | open → |
| deepseek/deepseek-v4-pro | d1_crud | 2 | pass | 7 | $0.007 | open → |
| deepseek/deepseek-v4-pro | d1_crud | 3 | pass | 6 | $0.005 | open → |
| google/gemini-3.1-pro-preview | d1_crud | 1 | pass | 4 | $0.044 | open → |
| google/gemini-3.1-pro-preview | d1_crud | 2 | pass | 3 | $0.034 | open → |
| google/gemini-3.1-pro-preview | d1_crud | 3 | pass | 3 | $0.023 | open → |
| minimax/minimax-m3 | d1_crud | 1 | pass | 6 | $0.007 | open → |
| minimax/minimax-m3 | d1_crud | 2 | pass | 4 | $0.005 | open → |
| minimax/minimax-m3 | d1_crud | 3 | pass | 5 | $0.007 | open → |
| moonshotai/kimi-k3 | d1_crud | 1 | pass | 6 | $0.118 | open → |
| moonshotai/kimi-k3 | d1_crud | 2 | pass | 3 | $0.047 | open → |
| moonshotai/kimi-k3 | d1_crud | 3 | pass | 3 | $0.044 | open → |
| openai/gpt-5.6-luna | d1_crud | 1 | pass | 3 | $0.011 | open → |
| openai/gpt-5.6-luna | d1_crud | 2 | pass | 2 | $0.009 | open → |
| openai/gpt-5.6-luna | d1_crud | 3 | pass | 2 | $0.010 | open → |
| openai/gpt-5.6-sol | d1_crud | 1 | pass | 4 | $0.080 | open → |
| openai/gpt-5.6-sol | d1_crud | 2 | pass | 4 | $0.079 | open → |
| openai/gpt-5.6-sol | d1_crud | 3 | pass | 5 | $0.122 | open → |
| qwen/qwen3-coder-plus | d1_crud | 1 | pass | 9 | $0.035 | open → |
| qwen/qwen3-coder-plus | d1_crud | 2 | pass | 5 | $0.010 | open → |
| qwen/qwen3-coder-plus | d1_crud | 3 | pass | 8 | $0.022 | open → |
| tencent/hy3 | d1_crud | 1 | pass | 3 | $0.002 | open → |
| tencent/hy3 | d1_crud | 2 | pass | 3 | $0.002 | open → |
| tencent/hy3 | d1_crud | 3 | pass | 4 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d1_crud | 1 | pass | 3 | $0.001 | open → |
| xiaomi/mimo-v2.5 | d1_crud | 2 | pass | 4 | $0.001 | open → |
| xiaomi/mimo-v2.5 | d1_crud | 3 | pass | 4 | $0.002 | open → |
| z-ai/glm-5.2 | d1_crud | 1 | pass | 4 | $0.011 | open → |
| z-ai/glm-5.2 | d1_crud | 2 | pass | 5 | $0.011 | open → |
| z-ai/glm-5.2 | d1_crud | 3 | pass | 5 | $0.010 | open → |
| claude-opus-4-8 | d1_edge_fn ⟂ | 1 | fail | 16 | $0.998 | open → |
| claude-opus-4-8 | d1_edge_fn ⟂ | 2 | fail | 17 | $0.965 | open → |
| claude-opus-4-8 | d1_edge_fn ⟂ | 3 | fail | 18 | $1.028 | open → |
| claude-opus-5 | d1_edge_fn ⟂ | 1 | fail | 20 | $1.021 | open → |
| claude-opus-5 | d1_edge_fn ⟂ | 2 | fail | 24 | $1.033 | open → |
| claude-opus-5 | d1_edge_fn ⟂ | 3 | fail | 12 | $0.277 | open → |
| claude-sonnet-5 | d1_edge_fn ⟂ | 1 | fail | 15 | $0.486 | open → |
| claude-sonnet-5 | d1_edge_fn ⟂ | 2 | fail | 14 | $0.522 | open → |
| claude-sonnet-5 | d1_edge_fn ⟂ | 3 | fail | 15 | $0.489 | open → |
| deepseek/deepseek-v4-pro | d1_edge_fn ⟂ | 1 | fail | 20 | $0.050 | open → |
| deepseek/deepseek-v4-pro | d1_edge_fn ⟂ | 2 | fail | 22 | $0.050 | open → |
| deepseek/deepseek-v4-pro | d1_edge_fn ⟂ | 3 | fail | 21 | $0.049 | open → |
| google/gemini-3.1-pro-preview | d1_edge_fn ⟂ | 1 | fail | 24 | $0.502 | open → |
| google/gemini-3.1-pro-preview | d1_edge_fn ⟂ | 2 | fail | 24 | $0.497 | open → |
| google/gemini-3.1-pro-preview | d1_edge_fn ⟂ | 3 | fail | 24 | $0.489 | open → |
| minimax/minimax-m3 | d1_edge_fn ⟂ | 1 | fail | 23 | $0.058 | open → |
| minimax/minimax-m3 | d1_edge_fn ⟂ | 2 | fail | 21 | $0.059 | open → |
| minimax/minimax-m3 | d1_edge_fn ⟂ | 3 | fail | 23 | $0.059 | open → |
| moonshotai/kimi-k3 | d1_edge_fn ⟂ | 1 | fail | 31 | $0.634 | open → |
| moonshotai/kimi-k3 | d1_edge_fn ⟂ | 2 | fail | 31 | $0.641 | open → |
| moonshotai/kimi-k3 | d1_edge_fn ⟂ | 3 | fail | 26 | $0.663 | open → |
| openai/gpt-5.6-luna | d1_edge_fn ⟂ | 1 | fail | 11 | $0.076 | open → |
| openai/gpt-5.6-luna | d1_edge_fn ⟂ | 2 | fail | 22 | $0.244 | open → |
| openai/gpt-5.6-luna | d1_edge_fn ⟂ | 3 | fail | 16 | $0.187 | open → |
| openai/gpt-5.6-sol | d1_edge_fn ⟂ | 1 | fail | 22 | $0.853 | open → |
| openai/gpt-5.6-sol | d1_edge_fn ⟂ | 2 | fail | 20 | $1.169 | open → |
| openai/gpt-5.6-sol | d1_edge_fn ⟂ | 3 | fail | 20 | $1.220 | open → |
| qwen/qwen3-coder-plus | d1_edge_fn ⟂ | 1 | fail | 19 | $0.139 | open → |
| qwen/qwen3-coder-plus | d1_edge_fn ⟂ | 2 | fail | 13 | $0.065 | open → |
| qwen/qwen3-coder-plus | d1_edge_fn ⟂ | 3 | fail | 20 | $0.138 | open → |
| tencent/hy3 | d1_edge_fn ⟂ | 1 | fail | 19 | $0.026 | open → |
| tencent/hy3 | d1_edge_fn ⟂ | 2 | fail | 17 | $0.025 | open → |
| tencent/hy3 | d1_edge_fn ⟂ | 3 | fail | 15 | $0.024 | open → |
| xiaomi/mimo-v2.5 | d1_edge_fn ⟂ | 1 | fail | 18 | $0.017 | open → |
| xiaomi/mimo-v2.5 | d1_edge_fn ⟂ | 2 | fail | 18 | $0.017 | open → |
| xiaomi/mimo-v2.5 | d1_edge_fn ⟂ | 3 | fail | 17 | $0.016 | open → |
| z-ai/glm-5.2 | d1_edge_fn ⟂ | 1 | fail | 21 | $0.124 | open → |
| z-ai/glm-5.2 | d1_edge_fn ⟂ | 2 | fail | 22 | $0.131 | open → |
| z-ai/glm-5.2 | d1_edge_fn ⟂ | 3 | fail | 28 | $0.127 | open → |
| claude-opus-4-8 | d1_rls_owner | 1 | pass | 6 | $0.319 | open → |
| claude-opus-4-8 | d1_rls_owner | 2 | pass | 10 | $0.534 | open → |
| claude-opus-4-8 | d1_rls_owner | 3 | pass | 10 | $0.539 | open → |
| claude-opus-5 | d1_rls_owner | 1 | pass | 18 | $0.890 | open → |
| claude-opus-5 | d1_rls_owner | 2 | pass | 19 | $0.971 | open → |
| claude-opus-5 | d1_rls_owner | 3 | pass | 15 | $0.830 | open → |
| claude-sonnet-5 | d1_rls_owner | 1 | fail | 11 | $0.558 | open → |
| claude-sonnet-5 | d1_rls_owner | 2 | pass | 10 | $0.351 | open → |
| claude-sonnet-5 | d1_rls_owner | 3 | fail | 8 | $0.292 | open → |
| deepseek/deepseek-v4-pro | d1_rls_owner | 1 | fail | 20 | $0.051 | open → |
| deepseek/deepseek-v4-pro | d1_rls_owner | 2 | fail | 4 | $0.003 | open → |
| deepseek/deepseek-v4-pro | d1_rls_owner | 3 | pass | 20 | $0.049 | open → |
| google/gemini-3.1-pro-preview | d1_rls_owner | 1 | pass | 15 | $0.202 | open → |
| google/gemini-3.1-pro-preview | d1_rls_owner | 2 | pass | 14 | $0.277 | open → |
| google/gemini-3.1-pro-preview | d1_rls_owner | 3 | pass | 20 | $0.528 | open → |
| minimax/minimax-m3 | d1_rls_owner | 1 | pass | 12 | $0.023 | open → |
| minimax/minimax-m3 | d1_rls_owner | 2 | pass | 8 | $0.016 | open → |
| minimax/minimax-m3 | d1_rls_owner | 3 | pass | 16 | $0.035 | open → |
| moonshotai/kimi-k3 | d1_rls_owner | 1 | pass | 9 | $0.153 | open → |
| moonshotai/kimi-k3 | d1_rls_owner | 2 | pass | 11 | $0.243 | open → |
| moonshotai/kimi-k3 | d1_rls_owner | 3 | pass | 11 | $0.230 | open → |
| openai/gpt-5.6-luna | d1_rls_owner | 1 | pass | 7 | $0.029 | open → |
| openai/gpt-5.6-luna | d1_rls_owner | 2 | pass | 7 | $0.030 | open → |
| openai/gpt-5.6-luna | d1_rls_owner | 3 | pass | 14 | $0.045 | open → |
| openai/gpt-5.6-sol | d1_rls_owner | 1 | pass | 5 | $0.148 | open → |
| openai/gpt-5.6-sol | d1_rls_owner | 2 | pass | 5 | $0.129 | open → |
| openai/gpt-5.6-sol | d1_rls_owner | 3 | pass | 4 | $0.093 | open → |
| qwen/qwen3-coder-plus | d1_rls_owner | 1 | pass | 10 | $0.033 | open → |
| qwen/qwen3-coder-plus | d1_rls_owner | 2 | pass | 12 | $0.045 | open → |
| qwen/qwen3-coder-plus | d1_rls_owner | 3 | pass | 9 | $0.036 | open → |
| tencent/hy3 | d1_rls_owner | 1 | pass | 8 | $0.011 | open → |
| tencent/hy3 | d1_rls_owner | 2 | pass | 11 | $0.013 | open → |
| tencent/hy3 | d1_rls_owner | 3 | pass | 16 | $0.017 | open → |
| xiaomi/mimo-v2.5 | d1_rls_owner | 1 | pass | 15 | $0.011 | open → |
| xiaomi/mimo-v2.5 | d1_rls_owner | 2 | pass | 19 | $0.014 | open → |
| xiaomi/mimo-v2.5 | d1_rls_owner | 3 | pass | 14 | $0.009 | open → |
| z-ai/glm-5.2 | d1_rls_owner | 1 | pass | 20 | $0.078 | open → |
| z-ai/glm-5.2 | d1_rls_owner | 2 | pass | 19 | $0.120 | open → |
| z-ai/glm-5.2 | d1_rls_owner | 3 | pass | 22 | $0.122 | open → |
| claude-opus-4-8 | d1_schema | 1 | pass | 6 | $0.218 | open → |
| claude-opus-4-8 | d1_schema | 2 | pass | 6 | $0.222 | open → |
| claude-opus-4-8 | d1_schema | 3 | pass | 6 | $0.213 | open → |
| claude-opus-5 | d1_schema | 1 | pass | 19 | $0.940 | open → |
| claude-opus-5 | d1_schema | 2 | pass | 15 | $0.644 | open → |
| claude-opus-5 | d1_schema | 3 | pass | 14 | $0.541 | open → |
| claude-sonnet-5 | d1_schema | 1 | pass | 4 | $0.056 | open → |
| claude-sonnet-5 | d1_schema | 2 | pass | 5 | $0.082 | open → |
| claude-sonnet-5 | d1_schema | 3 | pass | 5 | $0.101 | open → |
| deepseek/deepseek-v4-pro | d1_schema | 1 | pass | 6 | $0.009 | open → |
| deepseek/deepseek-v4-pro | d1_schema | 2 | pass | 6 | $0.008 | open → |
| deepseek/deepseek-v4-pro | d1_schema | 3 | pass | 4 | $0.005 | open → |
| google/gemini-3.1-pro-preview | d1_schema | 1 | pass | 4 | $0.058 | open → |
| google/gemini-3.1-pro-preview | d1_schema | 2 | pass | 3 | $0.043 | open → |
| google/gemini-3.1-pro-preview | d1_schema | 3 | pass | 3 | $0.040 | open → |
| minimax/minimax-m3 | d1_schema | 1 | pass | 12 | $0.023 | open → |
| minimax/minimax-m3 | d1_schema | 2 | pass | 8 | $0.020 | open → |
| minimax/minimax-m3 | d1_schema | 3 | pass | 21 | $0.060 | open → |
| moonshotai/kimi-k3 | d1_schema | 1 | pass | 7 | $0.137 | open → |
| moonshotai/kimi-k3 | d1_schema | 2 | pass | 7 | $0.156 | open → |
| moonshotai/kimi-k3 | d1_schema | 3 | pass | 5 | $0.054 | open → |
| openai/gpt-5.6-luna | d1_schema | 1 | pass | 4 | $0.014 | open → |
| openai/gpt-5.6-luna | d1_schema | 2 | pass | 4 | $0.016 | open → |
| openai/gpt-5.6-luna | d1_schema | 3 | pass | 4 | $0.013 | open → |
| openai/gpt-5.6-sol | d1_schema | 1 | pass | 4 | $0.095 | open → |
| openai/gpt-5.6-sol | d1_schema | 2 | pass | 4 | $0.082 | open → |
| openai/gpt-5.6-sol | d1_schema | 3 | pass | 4 | $0.079 | open → |
| qwen/qwen3-coder-plus | d1_schema | 1 | pass | 6 | $0.018 | open → |
| qwen/qwen3-coder-plus | d1_schema | 2 | pass | 8 | $0.031 | open → |
| qwen/qwen3-coder-plus | d1_schema | 3 | pass | 9 | $0.042 | open → |
| tencent/hy3 | d1_schema | 1 | pass | 5 | $0.003 | open → |
| tencent/hy3 | d1_schema | 2 | pass | 6 | $0.005 | open → |
| tencent/hy3 | d1_schema | 3 | pass | 6 | $0.005 | open → |
| xiaomi/mimo-v2.5 | d1_schema | 1 | pass | 7 | $0.003 | open → |
| xiaomi/mimo-v2.5 | d1_schema | 2 | pass | 6 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d1_schema | 3 | pass | 4 | $0.002 | open → |
| z-ai/glm-5.2 | d1_schema | 1 | pass | 8 | $0.023 | open → |
| z-ai/glm-5.2 | d1_schema | 2 | pass | 16 | $0.065 | open → |
| z-ai/glm-5.2 | d1_schema | 3 | pass | 8 | $0.021 | open → |
| claude-opus-4-8 | d1_storage | 1 | pass | 7 | $0.229 | open → |
| claude-opus-4-8 | d1_storage | 2 | pass | 7 | $0.231 | open → |
| claude-opus-4-8 | d1_storage | 3 | pass | 6 | $0.227 | open → |
| claude-opus-5 | d1_storage | 1 | pass | 12 | $0.364 | open → |
| claude-opus-5 | d1_storage | 2 | pass | 15 | $0.406 | open → |
| claude-opus-5 | d1_storage | 3 | pass | 17 | $0.502 | open → |
| claude-sonnet-5 | d1_storage | 1 | pass | 8 | $0.183 | open → |
| claude-sonnet-5 | d1_storage | 2 | pass | 6 | $0.163 | open → |
| claude-sonnet-5 | d1_storage | 3 | pass | 9 | $0.252 | open → |
| deepseek/deepseek-v4-pro | d1_storage | 1 | pass | 6 | $0.007 | open → |
| deepseek/deepseek-v4-pro | d1_storage | 2 | pass | 8 | $0.005 | open → |
| deepseek/deepseek-v4-pro | d1_storage | 3 | pass | 8 | $0.005 | open → |
| google/gemini-3.1-pro-preview | d1_storage | 1 | pass | 9 | $0.083 | open → |
| google/gemini-3.1-pro-preview | d1_storage | 2 | pass | 10 | $0.185 | open → |
| google/gemini-3.1-pro-preview | d1_storage | 3 | pass | 6 | $0.067 | open → |
| minimax/minimax-m3 | d1_storage | 1 | pass | 7 | $0.013 | open → |
| minimax/minimax-m3 | d1_storage | 2 | pass | 10 | $0.017 | open → |
| minimax/minimax-m3 | d1_storage | 3 | pass | 5 | $0.009 | open → |
| moonshotai/kimi-k3 | d1_storage | 1 | pass | 9 | $0.082 | open → |
| moonshotai/kimi-k3 | d1_storage | 2 | pass | 5 | $0.063 | open → |
| moonshotai/kimi-k3 | d1_storage | 3 | pass | 12 | $0.138 | open → |
| openai/gpt-5.6-luna | d1_storage | 1 | pass | 3 | $0.013 | open → |
| openai/gpt-5.6-luna | d1_storage | 2 | pass | 3 | $0.013 | open → |
| openai/gpt-5.6-luna | d1_storage | 3 | pass | 2 | $0.009 | open → |
| openai/gpt-5.6-sol | d1_storage | 1 | pass | 2 | $0.058 | open → |
| openai/gpt-5.6-sol | d1_storage | 2 | pass | 3 | $0.086 | open → |
| openai/gpt-5.6-sol | d1_storage | 3 | pass | 2 | $0.061 | open → |
| qwen/qwen3-coder-plus | d1_storage | 1 | pass | 11 | $0.066 | open → |
| qwen/qwen3-coder-plus | d1_storage | 2 | pass | 14 | $0.046 | open → |
| qwen/qwen3-coder-plus | d1_storage | 3 | pass | 17 | $0.136 | open → |
| tencent/hy3 | d1_storage | 1 | pass | 5 | $0.003 | open → |
| tencent/hy3 | d1_storage | 2 | pass | 5 | $0.004 | open → |
| tencent/hy3 | d1_storage | 3 | pass | 6 | $0.004 | open → |
| xiaomi/mimo-v2.5 | d1_storage | 1 | pass | 8 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d1_storage | 2 | pass | 9 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d1_storage | 3 | pass | 8 | $0.003 | open → |
| z-ai/glm-5.2 | d1_storage | 1 | pass | 9 | $0.021 | open → |
| z-ai/glm-5.2 | d1_storage | 2 | pass | 12 | $0.020 | open → |
| z-ai/glm-5.2 | d1_storage | 3 | pass | 13 | $0.037 | open → |
| claude-opus-4-8 | d2_backup_restore | 1 | pass | 10 | $0.468 | open → |
| claude-opus-4-8 | d2_backup_restore | 2 | pass | 11 | $0.446 | open → |
| claude-opus-4-8 | d2_backup_restore | 3 | pass | 11 | $0.572 | open → |
| claude-opus-5 | d2_backup_restore | 1 | pass | 25 | $1.022 | open → |
| claude-opus-5 | d2_backup_restore | 2 | pass | 19 | $0.973 | open → |
| claude-opus-5 | d2_backup_restore | 3 | pass | 22 | $1.022 | open → |
| claude-sonnet-5 | d2_backup_restore | 1 | pass | 14 | $0.479 | open → |
| claude-sonnet-5 | d2_backup_restore | 2 | pass | 9 | $0.352 | open → |
| claude-sonnet-5 | d2_backup_restore | 3 | pass | 13 | $0.304 | open → |
| deepseek/deepseek-v4-pro | d2_backup_restore | 1 | pass | 8 | $0.011 | open → |
| deepseek/deepseek-v4-pro | d2_backup_restore | 2 | pass | 14 | $0.028 | open → |
| deepseek/deepseek-v4-pro | d2_backup_restore | 3 | pass | 13 | $0.016 | open → |
| google/gemini-3.1-pro-preview | d2_backup_restore | 1 | pass | 12 | $0.128 | open → |
| google/gemini-3.1-pro-preview | d2_backup_restore | 2 | pass | 14 | $0.155 | open → |
| google/gemini-3.1-pro-preview | d2_backup_restore | 3 | pass | 15 | $0.159 | open → |
| minimax/minimax-m3 | d2_backup_restore | 1 | pass | 25 | $0.033 | open → |
| minimax/minimax-m3 | d2_backup_restore | 2 | pass | 8 | $0.014 | open → |
| minimax/minimax-m3 | d2_backup_restore | 3 | pass | 36 | $0.057 | open → |
| moonshotai/kimi-k3 | d2_backup_restore | 1 | pass | 8 | $0.151 | open → |
| moonshotai/kimi-k3 | d2_backup_restore | 2 | fail | 25 | $0.606 | open → |
| moonshotai/kimi-k3 | d2_backup_restore | 3 | pass | 8 | $0.159 | open → |
| openai/gpt-5.6-luna | d2_backup_restore | 1 | pass | 3 | $0.014 | open → |
| openai/gpt-5.6-luna | d2_backup_restore | 2 | pass | 7 | $0.039 | open → |
| openai/gpt-5.6-luna | d2_backup_restore | 3 | pass | 5 | $0.023 | open → |
| openai/gpt-5.6-sol | d2_backup_restore | 1 | pass | 5 | $0.163 | open → |
| openai/gpt-5.6-sol | d2_backup_restore | 2 | pass | 5 | $0.159 | open → |
| openai/gpt-5.6-sol | d2_backup_restore | 3 | pass | 6 | $0.252 | open → |
| qwen/qwen3-coder-plus | d2_backup_restore | 1 | fail | 15 | $0.145 | open → |
| qwen/qwen3-coder-plus | d2_backup_restore | 2 | pass | 17 | $0.139 | open → |
| qwen/qwen3-coder-plus | d2_backup_restore | 3 | fail | 19 | $0.132 | open → |
| tencent/hy3 | d2_backup_restore | 1 | pass | 10 | $0.011 | open → |
| tencent/hy3 | d2_backup_restore | 2 | pass | 9 | $0.008 | open → |
| tencent/hy3 | d2_backup_restore | 3 | pass | 9 | $0.009 | open → |
| xiaomi/mimo-v2.5 | d2_backup_restore | 1 | pass | 13 | $0.009 | open → |
| xiaomi/mimo-v2.5 | d2_backup_restore | 2 | pass | 10 | $0.005 | open → |
| xiaomi/mimo-v2.5 | d2_backup_restore | 3 | pass | 8 | $0.004 | open → |
| z-ai/glm-5.2 | d2_backup_restore | 1 | pass | 7 | $0.026 | open → |
| z-ai/glm-5.2 | d2_backup_restore | 2 | pass | 8 | $0.029 | open → |
| z-ai/glm-5.2 | d2_backup_restore | 3 | pass | 7 | $0.036 | open → |
| claude-opus-4-8 | d2_live_migration | 1 | pass | 12 | $0.514 | open → |
| claude-opus-4-8 | d2_live_migration | 2 | pass | 8 | $0.231 | open → |
| claude-opus-4-8 | d2_live_migration | 3 | pass | 9 | $0.320 | open → |
| claude-opus-5 | d2_live_migration | 1 | pass | 16 | $0.559 | open → |
| claude-opus-5 | d2_live_migration | 2 | pass | 12 | $0.483 | open → |
| claude-opus-5 | d2_live_migration | 3 | pass | 20 | $0.919 | open → |
| claude-sonnet-5 | d2_live_migration | 1 | pass | 7 | $0.153 | open → |
| claude-sonnet-5 | d2_live_migration | 2 | pass | 5 | $0.118 | open → |
| claude-sonnet-5 | d2_live_migration | 3 | pass | 6 | $0.161 | open → |
| deepseek/deepseek-v4-pro | d2_live_migration | 1 | pass | 8 | $0.013 | open → |
| deepseek/deepseek-v4-pro | d2_live_migration | 2 | pass | 13 | $0.021 | open → |
| deepseek/deepseek-v4-pro | d2_live_migration | 3 | pass | 8 | $0.013 | open → |
| google/gemini-3.1-pro-preview | d2_live_migration | 1 | pass | 7 | $0.059 | open → |
| google/gemini-3.1-pro-preview | d2_live_migration | 2 | pass | 6 | $0.055 | open → |
| google/gemini-3.1-pro-preview | d2_live_migration | 3 | pass | 7 | $0.074 | open → |
| minimax/minimax-m3 | d2_live_migration | 1 | pass | 9 | $0.017 | open → |
| minimax/minimax-m3 | d2_live_migration | 2 | pass | 11 | $0.015 | open → |
| minimax/minimax-m3 | d2_live_migration | 3 | pass | 6 | $0.012 | open → |
| moonshotai/kimi-k3 | d2_live_migration | 1 | pass | 11 | $0.225 | open → |
| moonshotai/kimi-k3 | d2_live_migration | 2 | pass | 7 | $0.114 | open → |
| moonshotai/kimi-k3 | d2_live_migration | 3 | pass | 8 | $0.139 | open → |
| openai/gpt-5.6-luna | d2_live_migration | 1 | pass | 9 | $0.022 | open → |
| openai/gpt-5.6-luna | d2_live_migration | 2 | pass | 3 | $0.016 | open → |
| openai/gpt-5.6-luna | d2_live_migration | 3 | pass | 3 | $0.016 | open → |
| openai/gpt-5.6-sol | d2_live_migration | 1 | pass | 3 | $0.068 | open → |
| openai/gpt-5.6-sol | d2_live_migration | 2 | pass | 4 | $0.082 | open → |
| openai/gpt-5.6-sol | d2_live_migration | 3 | pass | 6 | $0.120 | open → |
| qwen/qwen3-coder-plus | d2_live_migration | 1 | fail | 4 | $0.006 | open → |
| qwen/qwen3-coder-plus | d2_live_migration | 2 | pass | 15 | $0.059 | open → |
| qwen/qwen3-coder-plus | d2_live_migration | 3 | pass | 14 | $0.044 | open → |
| tencent/hy3 | d2_live_migration | 1 | pass | 7 | $0.006 | open → |
| tencent/hy3 | d2_live_migration | 2 | pass | 5 | $0.004 | open → |
| tencent/hy3 | d2_live_migration | 3 | pass | 7 | $0.005 | open → |
| xiaomi/mimo-v2.5 | d2_live_migration | 1 | pass | 5 | $0.002 | open → |
| xiaomi/mimo-v2.5 | d2_live_migration | 2 | pass | 15 | $0.009 | open → |
| xiaomi/mimo-v2.5 | d2_live_migration | 3 | pass | 12 | $0.003 | open → |
| z-ai/glm-5.2 | d2_live_migration | 1 | pass | 7 | $0.021 | open → |
| z-ai/glm-5.2 | d2_live_migration | 2 | pass | 5 | $0.013 | open → |
| z-ai/glm-5.2 | d2_live_migration | 3 | pass | 7 | $0.021 | open → |
| claude-opus-4-8 | d2_oauth ⟂ | 1 | fail | 18 | $0.970 | open → |
| claude-opus-4-8 | d2_oauth ⟂ | 2 | fail | 15 | $0.964 | open → |
| claude-opus-4-8 | d2_oauth ⟂ | 3 | fail | 15 | $1.024 | open → |
| claude-opus-5 | d2_oauth ⟂ | 1 | fail | 8 | $0.149 | open → |
| claude-opus-5 | d2_oauth ⟂ | 2 | fail | 8 | $0.190 | open → |
| claude-opus-5 | d2_oauth ⟂ | 3 | fail | 12 | $0.361 | open → |
| claude-sonnet-5 | d2_oauth ⟂ | 1 | fail | 13 | $0.498 | open → |
| claude-sonnet-5 | d2_oauth ⟂ | 2 | fail | 15 | $0.502 | open → |
| claude-sonnet-5 | d2_oauth ⟂ | 3 | fail | 16 | $0.501 | open → |
| deepseek/deepseek-v4-pro | d2_oauth ⟂ | 1 | fail | 21 | $0.052 | open → |
| deepseek/deepseek-v4-pro | d2_oauth ⟂ | 2 | fail | 20 | $0.048 | open → |
| deepseek/deepseek-v4-pro | d2_oauth ⟂ | 3 | fail | 24 | $0.050 | open → |
| google/gemini-3.1-pro-preview | d2_oauth ⟂ | 1 | fail | 22 | $0.503 | open → |
| google/gemini-3.1-pro-preview | d2_oauth ⟂ | 2 | fail | 21 | $0.491 | open → |
| google/gemini-3.1-pro-preview | d2_oauth ⟂ | 3 | fail | 21 | $0.497 | open → |
| minimax/minimax-m3 | d2_oauth ⟂ | 1 | fail | 18 | $0.058 | open → |
| minimax/minimax-m3 | d2_oauth ⟂ | 2 | fail | 22 | $0.057 | open → |
| minimax/minimax-m3 | d2_oauth ⟂ | 3 | fail | 19 | $0.059 | open → |
| moonshotai/kimi-k3 | d2_oauth ⟂ | 1 | fail | 27 | $0.602 | open → |
| moonshotai/kimi-k3 | d2_oauth ⟂ | 2 | fail | 25 | $0.642 | open → |
| moonshotai/kimi-k3 | d2_oauth ⟂ | 3 | fail | 25 | $0.602 | open → |
| openai/gpt-5.6-luna | d2_oauth ⟂ | 1 | fail | 27 | $0.239 | open → |
| openai/gpt-5.6-luna | d2_oauth ⟂ | 2 | fail | 21 | $0.243 | open → |
| openai/gpt-5.6-luna | d2_oauth ⟂ | 3 | fail | 19 | $0.240 | open → |
| openai/gpt-5.6-sol | d2_oauth ⟂ | 1 | fail | 26 | $1.181 | open → |
| openai/gpt-5.6-sol | d2_oauth ⟂ | 2 | fail | 30 | $1.159 | open → |
| openai/gpt-5.6-sol | d2_oauth ⟂ | 3 | fail | 37 | $1.253 | open → |
| qwen/qwen3-coder-plus | d2_oauth ⟂ | 1 | fail | 19 | $0.134 | open → |
| qwen/qwen3-coder-plus | d2_oauth ⟂ | 2 | fail | 25 | $0.137 | open → |
| qwen/qwen3-coder-plus | d2_oauth ⟂ | 3 | fail | 22 | $0.139 | open → |
| tencent/hy3 | d2_oauth ⟂ | 1 | fail | 26 | $0.026 | open → |
| tencent/hy3 | d2_oauth ⟂ | 2 | fail | 16 | $0.025 | open → |
| tencent/hy3 | d2_oauth ⟂ | 3 | fail | 14 | $0.025 | open → |
| xiaomi/mimo-v2.5 | d2_oauth ⟂ | 1 | fail | 17 | $0.017 | open → |
| xiaomi/mimo-v2.5 | d2_oauth ⟂ | 2 | fail | 20 | $0.016 | open → |
| xiaomi/mimo-v2.5 | d2_oauth ⟂ | 3 | fail | 23 | $0.018 | open → |
| z-ai/glm-5.2 | d2_oauth ⟂ | 1 | fail | 25 | $0.122 | open → |
| z-ai/glm-5.2 | d2_oauth ⟂ | 2 | fail | 36 | $0.131 | open → |
| z-ai/glm-5.2 | d2_oauth ⟂ | 3 | fail | 29 | $0.127 | open → |
| claude-opus-4-8 | d2_pgvector | 1 | pass | 6 | $0.207 | open → |
| claude-opus-4-8 | d2_pgvector | 2 | pass | 8 | $0.293 | open → |
| claude-opus-4-8 | d2_pgvector | 3 | pass | 5 | $0.159 | open → |
| claude-opus-5 | d2_pgvector | 1 | pass | 19 | $0.871 | open → |
| claude-opus-5 | d2_pgvector | 2 | pass | 20 | $0.885 | open → |
| claude-opus-5 | d2_pgvector | 3 | pass | 20 | $0.922 | open → |
| claude-sonnet-5 | d2_pgvector | 1 | pass | 9 | $0.202 | open → |
| claude-sonnet-5 | d2_pgvector | 2 | pass | 9 | $0.216 | open → |
| claude-sonnet-5 | d2_pgvector | 3 | pass | 7 | $0.166 | open → |
| deepseek/deepseek-v4-pro | d2_pgvector | 1 | pass | 12 | $0.013 | open → |
| deepseek/deepseek-v4-pro | d2_pgvector | 2 | pass | 10 | $0.011 | open → |
| deepseek/deepseek-v4-pro | d2_pgvector | 3 | pass | 11 | $0.013 | open → |
| google/gemini-3.1-pro-preview | d2_pgvector | 1 | pass | 5 | $0.054 | open → |
| google/gemini-3.1-pro-preview | d2_pgvector | 2 | pass | 7 | $0.078 | open → |
| google/gemini-3.1-pro-preview | d2_pgvector | 3 | pass | 5 | $0.055 | open → |
| minimax/minimax-m3 | d2_pgvector | 1 | pass | 9 | $0.014 | open → |
| minimax/minimax-m3 | d2_pgvector | 2 | pass | 18 | $0.021 | open → |
| minimax/minimax-m3 | d2_pgvector | 3 | pass | 6 | $0.008 | open → |
| moonshotai/kimi-k3 | d2_pgvector | 1 | pass | 9 | $0.185 | open → |
| moonshotai/kimi-k3 | d2_pgvector | 2 | pass | 6 | $0.110 | open → |
| moonshotai/kimi-k3 | d2_pgvector | 3 | pass | 8 | $0.154 | open → |
| openai/gpt-5.6-luna | d2_pgvector | 1 | pass | 4 | $0.016 | open → |
| openai/gpt-5.6-luna | d2_pgvector | 2 | pass | 3 | $0.013 | open → |
| openai/gpt-5.6-luna | d2_pgvector | 3 | pass | 4 | $0.017 | open → |
| openai/gpt-5.6-sol | d2_pgvector | 1 | pass | 6 | $0.119 | open → |
| openai/gpt-5.6-sol | d2_pgvector | 2 | pass | 4 | $0.094 | open → |
| openai/gpt-5.6-sol | d2_pgvector | 3 | pass | 3 | $0.075 | open → |
| qwen/qwen3-coder-plus | d2_pgvector | 1 | fail | 10 | $0.030 | open → |
| qwen/qwen3-coder-plus | d2_pgvector | 2 | fail | 7 | $0.015 | open → |
| qwen/qwen3-coder-plus | d2_pgvector | 3 | fail | 10 | $0.024 | open → |
| tencent/hy3 | d2_pgvector | 1 | pass | 15 | $0.015 | open → |
| tencent/hy3 | d2_pgvector | 2 | pass | 11 | $0.010 | open → |
| tencent/hy3 | d2_pgvector | 3 | pass | 10 | $0.008 | open → |
| xiaomi/mimo-v2.5 | d2_pgvector | 1 | pass | 14 | $0.007 | open → |
| xiaomi/mimo-v2.5 | d2_pgvector | 2 | pass | 14 | $0.007 | open → |
| xiaomi/mimo-v2.5 | d2_pgvector | 3 | pass | 10 | $0.004 | open → |
| z-ai/glm-5.2 | d2_pgvector | 1 | pass | 10 | $0.035 | open → |
| z-ai/glm-5.2 | d2_pgvector | 2 | pass | 8 | $0.022 | open → |
| z-ai/glm-5.2 | d2_pgvector | 3 | pass | 12 | $0.028 | open → |
| claude-opus-4-8 | d2_pooling | 1 | pass | 8 | $0.391 | open → |
| claude-opus-4-8 | d2_pooling | 2 | pass | 7 | $0.313 | open → |
| claude-opus-4-8 | d2_pooling | 3 | pass | 8 | $0.391 | open → |
| claude-opus-5 | d2_pooling | 1 | fail | 18 | $1.038 | open → |
| claude-opus-5 | d2_pooling | 2 | pass | 19 | $1.060 | open → |
| claude-opus-5 | d2_pooling | 3 | pass | 19 | $1.032 | open → |
| claude-sonnet-5 | d2_pooling | 1 | pass | 9 | $0.218 | open → |
| claude-sonnet-5 | d2_pooling | 2 | pass | 11 | $0.379 | open → |
| claude-sonnet-5 | d2_pooling | 3 | pass | 7 | $0.193 | open → |
| deepseek/deepseek-v4-pro | d2_pooling | 1 | pass | 6 | $0.013 | open → |
| deepseek/deepseek-v4-pro | d2_pooling | 2 | pass | 8 | $0.009 | open → |
| deepseek/deepseek-v4-pro | d2_pooling | 3 | pass | 7 | $0.008 | open → |
| google/gemini-3.1-pro-preview | d2_pooling | 1 | pass | 7 | $0.103 | open → |
| google/gemini-3.1-pro-preview | d2_pooling | 2 | pass | 6 | $0.093 | open → |
| google/gemini-3.1-pro-preview | d2_pooling | 3 | pass | 5 | $0.077 | open → |
| minimax/minimax-m3 | d2_pooling | 1 | pass | 5 | $0.010 | open → |
| minimax/minimax-m3 | d2_pooling | 2 | pass | 7 | $0.017 | open → |
| minimax/minimax-m3 | d2_pooling | 3 | pass | 8 | $0.017 | open → |
| moonshotai/kimi-k3 | d2_pooling | 1 | fail | 9 | $0.146 | open → |
| moonshotai/kimi-k3 | d2_pooling | 2 | pass | 10 | $0.211 | open → |
| moonshotai/kimi-k3 | d2_pooling | 3 | pass | 9 | $0.149 | open → |
| openai/gpt-5.6-luna | d2_pooling | 1 | pass | 2 | $0.011 | open → |
| openai/gpt-5.6-luna | d2_pooling | 2 | pass | 3 | $0.014 | open → |
| openai/gpt-5.6-luna | d2_pooling | 3 | pass | 4 | $0.028 | open → |
| openai/gpt-5.6-sol | d2_pooling | 1 | pass | 3 | $0.055 | open → |
| openai/gpt-5.6-sol | d2_pooling | 2 | pass | 3 | $0.055 | open → |
| openai/gpt-5.6-sol | d2_pooling | 3 | pass | 2 | $0.054 | open → |
| qwen/qwen3-coder-plus | d2_pooling | 1 | fail | 15 | $0.132 | open → |
| qwen/qwen3-coder-plus | d2_pooling | 2 | pass | 4 | $0.013 | open → |
| qwen/qwen3-coder-plus | d2_pooling | 3 | fail | 18 | $0.144 | open → |
| tencent/hy3 | d2_pooling | 1 | pass | 7 | $0.008 | open → |
| tencent/hy3 | d2_pooling | 2 | fail | 0 | $0.001 | open → |
| tencent/hy3 | d2_pooling | 3 | pass | 6 | $0.005 | open → |
| xiaomi/mimo-v2.5 | d2_pooling | 1 | fail | 6 | $0.004 | open → |
| xiaomi/mimo-v2.5 | d2_pooling | 2 | pass | 7 | $0.003 | open → |
| xiaomi/mimo-v2.5 | d2_pooling | 3 | pass | 9 | $0.005 | open → |
| z-ai/glm-5.2 | d2_pooling | 1 | pass | 8 | $0.029 | open → |
| z-ai/glm-5.2 | d2_pooling | 2 | pass | 9 | $0.025 | open → |
| z-ai/glm-5.2 | d2_pooling | 3 | fail | 28 | $0.126 | open → |
| claude-opus-4-8 | d2_rls_tenant | 1 | pass | 10 | $0.645 | open → |
| claude-opus-4-8 | d2_rls_tenant | 2 | fail | 14 | $1.021 | open → |
| claude-opus-4-8 | d2_rls_tenant | 3 | pass | 13 | $0.990 | open → |
| claude-opus-5 | d2_rls_tenant | 1 | pass | 15 | $0.987 | open → |
| claude-opus-5 | d2_rls_tenant | 2 | fail | 16 | $0.971 | open → |
| claude-opus-5 | d2_rls_tenant | 3 | pass | 16 | $1.065 | open → |
| claude-sonnet-5 | d2_rls_tenant | 1 | pass | 14 | $0.516 | open → |
| claude-sonnet-5 | d2_rls_tenant | 2 | pass | 14 | $0.496 | open → |
| claude-sonnet-5 | d2_rls_tenant | 3 | pass | 9 | $0.397 | open → |
| deepseek/deepseek-v4-pro | d2_rls_tenant | 1 | pass | 14 | $0.051 | open → |
| deepseek/deepseek-v4-pro | d2_rls_tenant | 2 | fail | 23 | $0.051 | open → |
| deepseek/deepseek-v4-pro | d2_rls_tenant | 3 | pass | 20 | $0.050 | open → |
| google/gemini-3.1-pro-preview | d2_rls_tenant | 1 | fail | 22 | $0.515 | open → |
| google/gemini-3.1-pro-preview | d2_rls_tenant | 2 | pass | 16 | $0.520 | open → |
| google/gemini-3.1-pro-preview | d2_rls_tenant | 3 | fail | 16 | $0.286 | open → |
| minimax/minimax-m3 | d2_rls_tenant | 1 | fail | 16 | $0.056 | open → |
| minimax/minimax-m3 | d2_rls_tenant | 2 | pass | 11 | $0.045 | open → |
| minimax/minimax-m3 | d2_rls_tenant | 3 | pass | 14 | $0.057 | open → |
| moonshotai/kimi-k3 | d2_rls_tenant | 1 | pass | 11 | $0.320 | open → |
| moonshotai/kimi-k3 | d2_rls_tenant | 2 | pass | 11 | $0.451 | open → |
| moonshotai/kimi-k3 | d2_rls_tenant | 3 | pass | 13 | $0.401 | open → |
| openai/gpt-5.6-luna | d2_rls_tenant | 1 | pass | 12 | $0.090 | open → |
| openai/gpt-5.6-luna | d2_rls_tenant | 2 | pass | 8 | $0.063 | open → |
| openai/gpt-5.6-luna | d2_rls_tenant | 3 | pass | 8 | $0.064 | open → |
| openai/gpt-5.6-sol | d2_rls_tenant | 1 | pass | 7 | $0.369 | open → |
| openai/gpt-5.6-sol | d2_rls_tenant | 2 | pass | 3 | $0.127 | open → |
| openai/gpt-5.6-sol | d2_rls_tenant | 3 | pass | 5 | $0.202 | open → |
| qwen/qwen3-coder-plus | d2_rls_tenant | 1 | fail | 12 | $0.048 | open → |
| qwen/qwen3-coder-plus | d2_rls_tenant | 2 | fail | 8 | $0.036 | open → |
| qwen/qwen3-coder-plus | d2_rls_tenant | 3 | fail | 11 | $0.045 | open → |
| tencent/hy3 | d2_rls_tenant | 1 | pass | 17 | $0.027 | open → |
| tencent/hy3 | d2_rls_tenant | 2 | pass | 13 | $0.023 | open → |
| tencent/hy3 | d2_rls_tenant | 3 | pass | 14 | $0.025 | open → |
| xiaomi/mimo-v2.5 | d2_rls_tenant | 1 | fail | 17 | $0.018 | open → |
| xiaomi/mimo-v2.5 | d2_rls_tenant | 2 | fail | 15 | $0.016 | open → |
| xiaomi/mimo-v2.5 | d2_rls_tenant | 3 | pass | 13 | $0.017 | open → |
| z-ai/glm-5.2 | d2_rls_tenant | 1 | pass | 19 | $0.126 | open → |
| z-ai/glm-5.2 | d2_rls_tenant | 2 | pass | 12 | $0.043 | open → |
| z-ai/glm-5.2 | d2_rls_tenant | 3 | pass | 19 | $0.123 | open → |
| claude-opus-4-8 | d2_webhooks | 1 | pass | 13 | $0.611 | open → |
| claude-opus-4-8 | d2_webhooks | 2 | pass | 16 | $0.650 | open → |
| claude-opus-4-8 | d2_webhooks | 3 | pass | 15 | $0.795 | open → |
| claude-opus-5 | d2_webhooks | 1 | pass | 21 | $1.001 | open → |
| claude-opus-5 | d2_webhooks | 2 | pass | 21 | $1.015 | open → |
| claude-opus-5 | d2_webhooks | 3 | pass | 20 | $0.977 | open → |
| claude-sonnet-5 | d2_webhooks | 1 | pass | 18 | $0.525 | open → |
| claude-sonnet-5 | d2_webhooks | 2 | pass | 15 | $0.402 | open → |
| claude-sonnet-5 | d2_webhooks | 3 | pass | 15 | $0.442 | open → |
| deepseek/deepseek-v4-pro | d2_webhooks | 1 | pass | 24 | $0.051 | open → |
| deepseek/deepseek-v4-pro | d2_webhooks | 2 | fail | 23 | $0.050 | open → |
| deepseek/deepseek-v4-pro | d2_webhooks | 3 | pass | 21 | $0.048 | open → |
| google/gemini-3.1-pro-preview | d2_webhooks | 1 | pass | 31 | $0.485 | open → |
| google/gemini-3.1-pro-preview | d2_webhooks | 2 | pass | 15 | $0.210 | open → |
| google/gemini-3.1-pro-preview | d2_webhooks | 3 | pass | 23 | $0.267 | open → |
| minimax/minimax-m3 | d2_webhooks | 1 | fail | 22 | $0.057 | open → |
| minimax/minimax-m3 | d2_webhooks | 2 | fail | 21 | $0.058 | open → |
| minimax/minimax-m3 | d2_webhooks | 3 | fail | 40 | $0.050 | open → |
| moonshotai/kimi-k3 | d2_webhooks | 1 | pass | 10 | $0.211 | open → |
| moonshotai/kimi-k3 | d2_webhooks | 2 | pass | 8 | $0.162 | open → |
| moonshotai/kimi-k3 | d2_webhooks | 3 | pass | 9 | $0.204 | open → |
| openai/gpt-5.6-luna | d2_webhooks | 1 | pass | 6 | $0.036 | open → |
| openai/gpt-5.6-luna | d2_webhooks | 2 | pass | 8 | $0.049 | open → |
| openai/gpt-5.6-luna | d2_webhooks | 3 | pass | 17 | $0.088 | open → |
| openai/gpt-5.6-sol | d2_webhooks | 1 | pass | 10 | $0.259 | open → |
| openai/gpt-5.6-sol | d2_webhooks | 2 | pass | 8 | $0.359 | open → |
| openai/gpt-5.6-sol | d2_webhooks | 3 | pass | 6 | $0.210 | open → |
| qwen/qwen3-coder-plus | d2_webhooks | 1 | pass | 23 | $0.131 | open → |
| qwen/qwen3-coder-plus | d2_webhooks | 2 | pass | 18 | $0.080 | open → |
| qwen/qwen3-coder-plus | d2_webhooks | 3 | pass | 25 | $0.130 | open → |
| tencent/hy3 | d2_webhooks | 1 | pass | 13 | $0.023 | open → |
| tencent/hy3 | d2_webhooks | 2 | fail | 5 | $0.004 | open → |
| tencent/hy3 | d2_webhooks | 3 | pass | 11 | $0.012 | open → |
| xiaomi/mimo-v2.5 | d2_webhooks | 1 | pass | 18 | $0.011 | open → |
| xiaomi/mimo-v2.5 | d2_webhooks | 2 | pass | 21 | $0.016 | open → |
| xiaomi/mimo-v2.5 | d2_webhooks | 3 | pass | 20 | $0.013 | open → |
| z-ai/glm-5.2 | d2_webhooks | 1 | pass | 25 | $0.132 | open → |
| z-ai/glm-5.2 | d2_webhooks | 2 | pass | 23 | $0.054 | open → |
| z-ai/glm-5.2 | d2_webhooks | 3 | pass | 23 | $0.052 | open → |
Source data, task definitions and verifier scripts: the repository.