A global manufacturer spends eighteen months and $2.3 million building an AI system to automate order processing. Demo\u2019s flawless. Leadership loves it. Rollout approved.<\/p>\n\n\n\n
Just a few weeks into production, everything started to unravel. The AI system made a shocking mistake, automatically approving a massive $400,000 order for a customer who was already three months behind on their payments. At the same time, it flagged a completely normal shipment as \u201chigh risk\u201d and brought fulfillment to a grinding halt for their biggest client. When the VP of Operations demanded to know who was responsible for these decisions, nobody could give him a clear answer. It was as if the AI had acted on its own, without any human oversight or accountability. The whole situation was a mess, and it was clear that something had gone terribly wrong with the system.<\/p>\n\n\n\n
Here\u2019s the thing, the AI wasn\u2019t wrong. It was doing exactly what it was built to do, making decisions based on patterns in data. The problem? Nobody had thought through which decisions it should make. Or which ones it absolutely shouldn\u2019t because it requires human oversight.<\/p>\n\n\n\n
This isn\u2019t just an isolated incident, it\u2019s a widespread phenomenon that\u2019s occurring all around us, at this very moment. And what\u2019s really interesting is that it highlights a crucial aspect that often gets overlooked in most discussions about artificial intelligence.<\/p>\n\n\n\n
The real question isn\u2019t whether to deploy AI. It\u2019s whether you\u2019re ready to fundamentally rewire how your organization makes decisions.<\/strong><\/p>\n\n\n\n The companies that figure this out will pull ahead. The ones that don\u2019t will spend a lot of money learning expensive lessons.<\/p>\n\n\n\n You\u2019ve heard the pitch. It\u2019s in every boardroom. \u201cLet\u2019s deploy AI to automate this process.\u201d<\/p>\n\n\n\n That sounds good. It sounds like things are moving forward, evolving. But there\u2019s a big problem with that way of thinking. It assumes the system is actually working properly, and all we need to do is make it go faster.<\/p>\n\n\n\n Most companies are trying to use AI, but it\u2019s not working out. Researchers at MIT found that almost all AI projects in big businesses fail. It\u2019s not that the AI itself is the problem, it\u2019s just that it can\u2019t learn from the way things are already being done. AI tools are great for individuals, but when you try to use them in a big company, they don\u2019t work very well. They can\u2019t adapt to the existing workflows, so they just don\u2019t work. What\u2019s happening is that companies are trying to add AI to their old processes, instead of changing the way they do things to make AI work. This is what MIT calls \u201cthe learning gap\u201d.<\/p>\n\n\n\n From my experience working with clients, it seems that the issue of integrating new technologies is actually a matter of figuring out how to make decisions. Companies are not just having trouble incorporating AI into their daily operations, but they are also struggling with a more fundamental question. What\u2019s the best way to make decisions in our organization? Where should humans be involved in the decision-making process?<\/p>\n\n\n\n At its core, every organization is, in part, a machine that makes decisions. Think about it, every day there are tons of choices being made, like whether to approve or reject orders, how to handle exceptions, and which customers to prioritize. These decisions happen thousands of times a day and ultimately decide whether the business succeeds or fails. It\u2019s all about making the right calls, and it\u2019s what sets the winners apart from the losers.<\/p>\n\n\n\n These decision patterns were created a long time ago, in a different time with different technology. The people who set them up are no longer with the company, and the knowledge about how they work is scattered and hard to find. It\u2019s hidden in old ERP configurations, complicated approval processes, and informal conversations that happen in the hallways. Nobody has written it down or challenged it, so it just keeps going on as it always has.<\/p>\n\n\n\n And now we\u2019re layering AI on top of all that and wondering why stuff breaks.<\/p>\n\n\n\n McKinsey\u2019s 2025 State of AI report makes the point clearly. Workflow redesign drives more EBIT impact from AI than anything else they tested. More than model selection. More than data quality. Yet only 21% of organizations actually redesign their workflows when they deploy AI. <\/p>\n\n\n\n The other 79% are just automating their existing problems faster.<\/p>\n\n\n\n The market is slowly figuring out the most successful AI deployments aren\u2019t the most automated ones. They\u2019re the most intentionally designed.<\/p>\n\n\n\n Think about what actually happens when you give an AI agent full autonomy in a messy business environment. It optimizes for the metrics you handed it. But real business decisions aren\u2019t just about metrics. They involve context and relationships and exceptions and judgment calls that don\u2019t exist anywhere in the training data.<\/p>\n\n\n\n You know that customer who\u2019s really late with their payment, 90 days overdue? It\u2019s possible they\u2019re going through some changes and you\u2019ve already sorted out a new payment plan with them. And what about that shipment that\u2019s been flagged as high risk? Maybe your most important customer just asked to change the delivery address because they\u2019re opening a new location. The thing is, artificial intelligence doesn\u2019t have any way of knowing these details, it just can\u2019t, at least not yet.<\/p>\n\n\n\n AI systems are great at recognizing patterns, identifying unusual activity, and processing large amounts of data quickly. They can do all this much faster than any human team. However, they have a significant limitation, they can\u2019t know what they don\u2019t know. In complicated business situations, the things you\u2019re not aware of often have a bigger impact than the things you do know. This is because unknown factors can affect your decisions and outcomes in unexpected ways, making it crucial to consider both the known and unknown elements when making business decisions.<\/p>\n\n\n\n By 2027, Gartner expects that more than 40% of projects involving AI that can act on its own will be canceled. But this won\u2019t be because the AI models didn\u2019t work as planned. The real reason is that many organizations are skipping a crucial step, which is deciding when to let AI make decisions independently and when human judgment should still be involved. This is the hard part, and it\u2019s essential to get it right. If organizations don\u2019t take the time to figure this out, their AI projects are likely to fail. It\u2019s not just about having the right technology, but also about understanding how to use it effectively and responsibly. AI agents work. But automation without orchestrating human oversight is just chaos moving faster.<\/p>\n\n\n\n So what does it mean to orchestrate decisions?<\/p>\n\n\n\n It\u2019s not just about dividing tasks between AI and humans, it\u2019s about creating a harmonious workflow. True orchestration is when all the different parts of an organization, including lines of business, subject matter experts, processes, and decision-making, come together in a way that makes sense. Each person knows what they\u2019re responsible for and what\u2019s expected of them. The right information gets to the right people at the right time, and it\u2019s clear who\u2019s accountable for what. This way, everyone is on the same page and working towards the same goal.<\/p>\n\n\n\n It starts with a question most organizations have never actually sat down and answered. For each decision in a critical workflow, who should make it? With what information? And what happens when someone gets it wrong?<\/p>\n\n\n\n At Duro, we call this mapping the “decision architecture.” It’s the invisible structure underneath how choices move through your organization. When we map it for clients, we almost always find the same thing. Either the current setup was designed for a completely different era, or it’s a patchwork of processes and stitched together tech that’s creating drag everywhere. The organizations winning with AI right now aren\u2019t just buying new technology. They\u2019re building new ways of working that actually leverage what humans and machines each do best. Here\u2019s the framework we use. <\/p>\n\n\n\n You may know this as Human-in-the-Loop. We call it Human-in-Control.<\/p>\n\n\n\n\nThe Decision Engine Problem<\/h2>\n\n\n\n
Why Full Autonomy Is a Bad Bet<\/h2>\n\n\n\n
What Orchestration Actually Means<\/h2>\n\n\n\n