Get workout gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Imagine an AI that reads your files before making a decision — and how that could impact your next business deal
Just like in fitness, where understanding your body’s details is key to progress, AI in business needs deep insight to succeed. The latest experiment from Firmulate reveals how AIs that truly ‘read’ and understand internal files outperform their peers — and could be your secret weapon in closing high-stakes deals.
Products Worth Considering
Fitbit Versa 2 Health and Fitness Smartwatch with Heart Rate, Alexa, Music, Sleep and Swim Tracking
Coway Mighty AP-1512HH(W) Air Purifier, White
Withings BPM Connect Wi-Fi and Bluetooth Blood Pressure Monitor
As an affiliate, we earn on qualifying purchases.
The Experiment: Putting AI to the Test in a Critical Week
In a groundbreaking live experiment, four frontier AI models were tasked with managing a small software company’s most challenging week. The same crises, customer demands, and temptations confronted each model, which operated in a controlled, auditable environment. The goal? See which AI reads deeply enough — not just on the surface — to identify hidden facts that could seal a €55,000 deal.
The Key Finding: The Power of Deep Reading
Among the models, only two managed to close the deal based on their own analysis. They spotted a critical, buried fact two references deep in the company’s internal files — a piece of information that, if missed, would have cost €4,583 in Monthly Recurring Revenue (MRR). The others, despite diagnosing the situation correctly and engaging confidently, left the deal on the table because they failed to uncover this hidden insight.
What This Means for Business and AI
In real-world terms, this experiment underscores a vital property often overlooked: whether an AI can ‘read’ your internal documents thoroughly before acting. It’s not just about generating convincing responses but about extracting crucial context buried within files. This ability to perform multi-hop reasoning—reading deep into documents before making decisions—is what sets truly effective AI apart in high-stakes environments.
Testing Under Pressure: Trust, Ethics, and Discipline
The experiment also tested how AI models handle social engineering attempts. Fake CEO messages escalating over multiple stages and a reporter trick were presented to all models. Remarkably, every model refused to be manipulated, adhering to ethical boundaries. Kimi K3, the top performer in fairness, explained its reasoning: “Treat the request as a suspected approval bypass or impersonation.”
Implications for Business Operations
In real companies, especially those with complex money mechanics and multiple decision points, this discipline is critical. An AI that reads deeply, stays honest under pressure, and refuses manipulative tactics can safeguard your operations against fraud, misjudgment, and trust breaches—features as crucial as their analytical prowess.
What Sets Top Models Apart?
The top-scoring model, gpt-5.6-sol, achieved a 95/100 score, largely because it uncovered the buried fact and closed the deal. Kimi K3 and Sonnet followed with scores of 93 and 88, respectively, both closing deals but with slight slips in discipline or thoroughness. Surprisingly, the most thorough participant, Opus 4.8—drawing on over 80 learned rules—ranked last at 73, demonstrating that depth of analysis alone isn’t enough without discipline and focus.
Why Does This Matter for Your Business?
If your AI tools are involved in customer management, forecasting, or decision-making, understanding their ability to read and interpret your internal files is crucial. An AI that only skims the surface risks missing vital context, leading to missed opportunities or costly mistakes. Conversely, AI that thoroughly reads your documents can identify hidden opportunities—like that €4,583 MRR gain in the experiment—and make smarter, more profitable decisions.
Next Steps: Simulate Your Business with AI
For enterprises eager to test their AI readiness, Firmulate offers a unique “wargame” approach. You can run your own business scenario against a read-only export of your data—without risking real systems—allowing you to gauge how well your AI understands and acts on critical information. This proactive testing helps you select and fine-tune AI agents before deployment.
Final Takeaway
The experiment demonstrates a clear, measurable property: reading deep into your files before acting. In a world where AI decisions influence business outcomes, that property might just be the most decisive factor of all. The question isn’t whether AI can generate smooth chat responses, but whether it can finish what it starts—reading, understanding, and acting on your data with discipline and integrity.
Deep reading and disciplined AI behavior are crucial for winning high-stakes business deals. Firms that test and tune these capabilities now will have the edge tomorrow.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
