Cost per successful task, cut by industry vertical. A score above 100 means the model produces more completed work per dollar than the baseline.
Log axis — a real panel spans orders of magnitude. Whiskers are 95% clustered bootstrap intervals, resampled by vertical, then task, then repetition. The baseline scores 100 against itself by construction, so its interval is on cost per success instead and is not drawn here.
Every panel shares one axis, so they are directly comparable. This is the cut that matters — cost-effectiveness is a property of a model on a kind of work, not of a model alone.
Success rate against cost per success. Bottom-right is the good corner: reliable and cheap. Position carries the reading, not colour — every point is directly labelled.
Task-level rows are omitted here for length. The CSV and JSON exports carry them in full.
| Scope | Vertical | Model | Octane | 95% CI | $/success | Success | Reps | Total cost |
|---|---|---|---|---|---|---|---|---|
| overall | — | Fake Cheap | 789 | 353–1,253 | $0.000024 | 56% | 45 | $0.00054 |
| overall | — | Fake Frontier | 9.55 | 6.89–16.5 | $0.00198 | 96% | 45 | $0.0800 |
| overall | — | Fake Mid | 100 | $0.00011–$0.00029 (cost) | $0.00019 | 78% | 45 | $0.00608 |
| vertical | banking | Fake Cheap | 823 | 276–1,916 | $0.0000226 | 50% | 10 | $0.00011 |
| vertical | banking | Fake Frontier | 11.1 | 6.84–33.3 | $0.00168 | 100% | 10 | $0.0168 |
| vertical | banking | Fake Mid | 100 | $0.000078–$0.00035 (cost) | $0.00019 | 70% | 10 | $0.00130 |
| vertical | coding | Fake Cheap | 1,359 | 799–2,809 | $0.0000156 | 80% | 10 | $0.00011 |
| vertical | coding | Fake Frontier | 7.89 | 5.84–30.4 | $0.00268 | 100% | 10 | $0.0234 |
| vertical | coding | Fake Mid | 100 | $0.0000847–$0.00032 (cost) | $0.00021 | 70% | 10 | $0.00139 |
| vertical | customer_service | Fake Cheap | 649 | 167–1,333 | $0.0000281 | 40% | 10 | $0.00011 |
| vertical | customer_service | Fake Frontier | 13.3 | 8.24–23.1 | $0.00137 | 90% | 10 | $0.0123 |
| vertical | customer_service | Fake Mid | 100 | $0.00011–$0.00040 (cost) | $0.00018 | 70% | 10 | $0.00127 |
| vertical | legal | Fake Cheap | 830 | 239–1,861 | $0.0000314 | 60% | 5 | $0.0000941 |
| vertical | legal | Fake Frontier | 10.2 | 8.57–17.2 | $0.00254 | 100% | 5 | $0.0127 |
| vertical | legal | Fake Mid | 100 | $0.00022–$0.00044 (cost) | $0.00026 | 80% | 5 | $0.00104 |
| vertical | medical | Fake Cheap | 476 | 166–820 | $0.0000225 | 50% | 10 | $0.00011 |
| vertical | medical | Fake Frontier | 6.50 | 4.10–13.7 | $0.00165 | 90% | 10 | $0.0148 |
| vertical | medical | Fake Mid | 100 | $0.0000306–$0.00019 (cost) | $0.00011 | 100% | 10 | $0.00107 |