A Smart Model Does Not Make an Enterprise System

According to McKinsey’s latest 2025 survey, 51% of organizations reported experiencing at least one negative AI-related incident over the previous twelve months. The findings suggest that these incidents stem from the absence of the infrastructure needed to govern how AI operates within the enterprise.

Executive Summary

A model may understand a request, but it cannot determine on its own who is authorized to execute it.

The difference between a “smart answer” and a “trusted enterprise system” lies in the governance layer surrounding the model: data, permissions, tools, and escalation paths.

With digital agents, an error does not stop at an inaccurate recommendation. It can extend to a real-world actionsuch as updating a record or processing a transaction.

According to IBM’s 2025 CEO Study, 68% of CEOs believe that an enterprise-wide integrated data architecture is critical to collaboration. Yet only half have an infrastructure capable of supporting it in practice.

The organizations achieving the strongest results from AI adoption are also those most actively redesigning workflows and human oversight.

A Model May Understand the Request, but It Does Not Know Its Limits

An AI model may be able to read a customer’s request and understand its intent with remarkable accuracy. But on its own, it does not know whether the customer’s identity has been verified, whether the customer is authorized to access the specific data requested, whether the request complies with organizational policies, or whether it should be executed immediately or escalated to an employee for review.

The model does not answer these questions. The organization does.

This is where the distinction emerges between a model capable of generating an answer and an enterprise system capable of performing trusted work. The model represents the intelligence layer alone. The surrounding system determines which data it can access, which tools it can use, which permissions it holds, which actions it is authorized to perform, and when it must stop and request human intervention.

A Procurement Example: Where Does the Model’s Role End?

Consider an AI assistant working within a procurement department. It reads purchase requests, verifies supplier information, compares prices, and prepares recommendations. Up to this point, its performance may be excellent. But the real question is: should it be allowed to approve the request on its own?

The answer does not depend solely on the model’s accuracy. It depends on factors unrelated to its linguistic capabilities: the value of the purchase, the authority of the employee who submitted it, the organization’s procurement policy, the available budget, potential conflicts of interest, and the internal approval chain.

In other words, intelligence does not operate in a vacuum. It operates within a complete operational, legal, and regulatory context. The model itself has no way to interpret that context unless it is deliberately engineered into the surrounding system.

This is not merely a theoretical concern. IBM’s 2025 study of 2,000 CEOs worldwide found that 68% consider an integrated, enterprise-wide data architecture essential to cross-functional collaboration and innovation. Yet half of those same CEOs acknowledge that the accelerating pace of investment has left their organizations with fragmented and disconnected data systems.

In other words, awareness of the importance of integration remains far ahead of the actual ability to achieve it.

Integration alone, however, is not enough. Connecting a model to more systems without appropriate access controls may grant it broader permissions than necessary. Connecting it to execution workflows without clearly defined limits may turn an inaccurate answer into an incorrect real-world action.

In traditional AI systems, the problem often ends with an inaccurate recommendation that a human can review. But with digital agents connected to operational systems, the error may directly result in a record being updated, a message being sent, a transaction being approved, an external service being called, or even a financial process being initiated.

Capability and Control Must Advance Together

This is why capability and control must evolve in parallel rather than sequentially. Every new capability granted to the system should be matched by:

A precisely defined scope of authority

Reliable user identity verification

A clear data-use policy

A complete record of decisions and actions

Explicit limits on automated execution

A defined path for escalating cases to a human when those limits are exceeded

This is also why governance must be treated as operational infrastructure built into the system from day one.

Effective governance does more than answer the question, “What is the system not allowed to do?” It also addresses a more important question for growth: “How can we enable the system to operate safely and at greater scale?”

The NIST AI Risk Management Framework already provides a shared language for balancing enablement with risk controlone that organizations can build upon instead of creating an entirely new framework from scratch.

Redesign Workflows, Not Just Adopt a Model

McKinsey’s 2025 data reveals a clear pattern: organizations generating the greatest value from AI are distinguished not only by their use of better models, but also by their willingness to redesign workflows, build the supporting data and technology infrastructure, and clearly define where human review is required.

These high-performing organizations were nearly three times more likely to fundamentally redesign their workflows than organizations that simply added AI on top of existing processes without changing them.

In other words, high performance is not the result of a more powerful model. It is the result of an organizational decision to rebuild how work itself is carried out around AI capabilitiesa decision that precedes any technical choice.

The goal is not to give AI as much access as possible. It is to grant it exactly the level of access required to complete a specific task, within boundaries the organization can understand, monitor, and audit.

The model provides the intelligence. But integration, permissions, policies, and monitoring are what transform it from a response-generating tool into an enterprise system that can truly be trusted.

Sources: McKinsey, The State of AI in 2025: Agents, Innovation, and Transformation; IBM Institute for Business Value, 2025 CEO Study; NIST, AI Risk Management Framework.