Imagine you’re trying to sell a batch of your finest ice cream, but the buyer only signs the contract after reading a hidden note buried two pages deep in your sales file. Would you hide that crucial detail, or make sure they see it? In the world of AI and business, the secret to winning big deals isn’t just how well an AI can chat — it’s whether it actually digs into your files before making a decision. This experiment reveals that AI’s ability to read and find buried facts can be the difference between sealing a €55,000 deal or losing it in the blink of an eye.
Get baking supplies delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
The AI Test: More Than Just Words
Recently, a groundbreaking live experiment placed four advanced AI models in the shoes of a small software company facing its worst week. The scenario involved the same customers, the same crises, and the same temptation to cut corners. Every decision made by the AI was recorded, versioned, and could be audited — a kind of digital crystal ball for management quality.
The stakes were high: one of the models, gpt-5.6-sol, scored an impressive 95 out of 100, while the others trailed behind. But what truly mattered wasn’t just overall scores. When it came to reading the company’s files, only two models managed to find a buried fact hidden two references deep in the files, which was critical for closing a lucrative deal. Those two AI models signed the €55,000 deal based on their insights, while the others, despite diagnosing the same problem and pitching the same solution, left the deal on the table.
As an affiliate, we earn on qualifying purchases.
The Hidden Weakness: Deep File Reading
This experiment uncovered a surprising weakness among AI models: the difference between winning and losing a deal often lay in whether they read past the surface. The decisive flaw was buried two references down in the company’s internal documents — not in the customer correspondence or the obvious crisis reports. Models that diligently examined the files, like gpt-5.6-sol and Kimi K3, uncovered this hidden fact, sealing the deal at full price. Those that didn’t, missed the critical detail and lost the opportunity.
Why It Matters for Business AI
If your AI touches your CRM, support queues, or forecasts, the question isn’t just whether it writes well. It’s whether it reads thoroughly, stays honest under pressure, and finishes what it starts. For example, during a simulated social engineering attack involving staged CEO messages and a reporter’s trick, all models refused manipulation — a promising sign of their integrity. But the real differentiator was their ability to find buried truths that could determine a deal’s outcome.
Real Company, Real Money, Real Risks
The live experiment isn’t just a theoretical test; it involves a real company with 13 synthetic employees managing real-money mechanics. The company burns €105,000 each month against a Monthly Recurring Revenue (MRR) of only €2,300, making every decision critical. The AI models operate within a framework of over 680 self-learned rules, and their decisions are constantly versioned and observable at firmulate.com/live. This transparent sandbox allows companies to run their own tests before real-world deployment, ensuring their AI agents can truly deliver.
Scores and Lessons from the League
In the latest Crucible League, the top scorer was gpt-5.6-sol with a remarkable 95 points, closely followed by Kimi K3 at 93. Sonnet 5 and Sonnet 4 scored 88 and 77 respectively. These scores reflect their overall performance, but the key takeaway was their ability to detect the buried fact and close the deal. Interestingly, the most thorough participant, Opus 4.8, with over 80 learned rules and the deepest analysis, placed last, missing the crucial hidden detail and leaving the deal unsealed.
What This Means for Your Business
For companies considering AI integration, the message is clear: the focus should be on whether the AI reads and verifies information, not just whether it can generate convincing text. The ability to find buried truths in your documents, stay honest under social engineering attempts, and finish what it starts could be your competitive edge — or your downfall if neglected.
By testing AI models in a controlled, real-money environment, businesses can assess their true readiness. You can run your own scenario-based games, like the ones at firmulate.com/pilot, to explore how your AI would perform under pressure. Remember: it’s not just about chat quality; it’s about whether your AI can truly finish the work you assign it, reading all relevant files first.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
