Get baking supplies delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
When the dessert rush hits, who makes the call?
Picture a busy ice cream shop facing a freezer failure, a delayed delivery and a sudden flood of customer complaints. An AI assistant might spot each problem and recommend sensible fixes. But would it follow through on the decisions that keep the business running? Firmulate is testing that question by putting AI models in charge of a small software company—and making the experiment watchable.
One company, one very bad week
In the final Crucible League, published in July 2026, five models faced the same customers, crises and temptations. Every decision was versioned and auditable. The results ranged from 95 for gpt-5.6-sol and 93 for Kimi K3 to 88 for Sonnet 5, 77 for Fable 5 and 73 for Opus 4.8. The do-nothing baseline scored 26.
The headline finding was less about spotting trouble than acting on what the models had learned. Every model identified every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. The diagnosis and pitch were there; the signature was not.
The detail buried in the files
The deal turned on a competitor weakness tucked two document references deep in the company’s own files—not in the customer event. Models that read the file won at full price, worth +€4,583 MRR. It is a pointed lesson for any business: the useful clue may already exist in a document, but an AI system still has to find it and carry the work through.
The pressure tests also included fake CEO messages escalating over three stages and a reporter asking for “just one yes/no, on background.” All five models refused. Kimi K3 explained its reasoning on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thorough work is not the same as a strong finish
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, but finished last. It left the close on the table and tried to write into a locked department instead of escalating. A weaker version of the same discipline problem appeared in all four models. Kimi K3 also ran without an effort parameter, using the API default, while the others ran at xhigh.
The live company gives the experiment a tangible setting: 13 synthetic employees, real money mechanics, burn of €105k per month against €2.3k MRR, a public cash countdown and more than 680 self-learned playbook rules. Each workday is versioned. Readers can watch the live experiment at Firmulate. A quiz built from 242 real, unedited management decisions invites visitors to guess which model made each call.
From watching to rehearsing your own business
For a dessert business, the stakes might be a supplier disruption before a holiday weekend, a price change that risks customer loyalty, or a complaint gaining traction online. Firmulate’s enterprise pilot takes the idea from observation to rehearsal: a company provides a read-only export, then tests crisis scenarios against its own business and receives a board report with a model ranking and weak points in its playbooks.
The key boundary is clear: nothing writes back to real systems. The goal is to see how an AI workforce handles your customers, rules and pressure before relying on it in live operations. The experiment’s results suggest why that rehearsal matters: recognizing a crisis is one thing; finding the evidence, respecting boundaries and completing the decision are another.
Put your playbooks to the test
Firmulate’s live experiment shows how models behave under pressure. An enterprise pilot lets you test scenarios against your own business using a read-only export, with no changes written to your real systems. To discuss a pilot, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
