The pattern has become hard to ignore. An enterprise rolls out a generative AI initiative with considerable fanfare and even more considerable budget. Pilots impress the board. Consultants produce roadmaps. Then, somewhere between proof-of-concept and production, the whole thing quietly stalls. The model is not the problem. The organisation around it is.
Three independent data streams now point in the same direction. McKinsey’s State of AI 2024 found that while 72% of large enterprises had AI systems in production, only 9% described their governance as mature. S&P Global’s follow-up survey, covering more than 1,000 firms, found that the share of companies abandoning most AI initiatives jumped from 17% in 2024 to 42% in 2025. MIT’s GenAI Divide research tracked tens of billions in enterprise AI spend and estimated that just 5% of generative AI projects produced measurable profit-and-loss impact.
None of those failures were caused by a bad foundation model. They were caused by decisions — or the absence of decisions — about who controls the system, what constitutes acceptable risk, and who answers when something goes wrong. That is a governance problem. And it is the central challenge of the current phase of AI adoption.
The Governance Gap Nobody Wants to Admit
Governance failures are uncomfortable to acknowledge because they implicate leadership rather than vendors. It is easier to blame the technology — the model hallucinated, the integration was messy, the data was dirty — than to admit that the organisation had no coherent decision-making structure for AI in the first place.
The PEX Report for 2025/26 found that only 43% of organisations have a formal AI governance policy. That means the majority of enterprises currently deploying AI systems — systems that are making hiring recommendations, flagging fraud, setting insurance premiums, or routing customer queries — are doing so without a documented framework for oversight, accountability, or remediation. The models are running. The governance is not.
The 2025 AI Governance Benchmark Report found that while 80% of enterprises had fifty or more generative AI use cases in active development, most had only a handful actually running in production. Among leaders surveyed, 58% cited disconnected governance systems as the primary reason they could not scale. Not compute costs. Not model capability. Governance.
“
“The models are not the failure point. The systems, people, and structures built around those models are. The bottleneck is not building AI — it is deciding who controls it, what risk is acceptable, and how quickly decisions can be made without breaking what matters.”
— Tolulope Michael, CVO, ExcelMindCyber Institute
The data from the growth enterprises market makes this concrete: late-stage AI deal sizes climbed from an average of $48 million in 2023 to $327 million in 2024, according to Pacific AI’s 2025 survey of 351 organisations. That acceleration in investment has compressed the time available for governance design, because speed-to-market pressure increases in direct proportion to investor expectation. Governance gets treated as something to retrofit. It rarely survives the retrofit intact.
Why the Models Are Not the Problem
There is a structural reason why AI governance failures get misdiagnosed as technology failures: the technology is visible and the governance is not. When an AI system produces a discriminatory credit decision or a recruitment algorithm systematically filters out certain demographic groups, the incident report typically describes a model behaving badly. The actual root cause — inadequate oversight, no defined review process, absence of bias testing, unclear accountability for remediation — stays buried in the post-mortem.
The AI Incident Database recorded a 32% increase in reported AI incidents in 2024, with early 2025 data indicating the trend continued. These incidents were almost never caused by model architecture. They were caused by breakdowns in oversight, by the absence of accountability structures, and by deploying systems into high-stakes contexts without adequate human review mechanisms.
Gartner’s Q3 2024 survey of 248 data management leaders found that 63% of organisations either lacked AI-ready data practices or were unsure whether they had them. Gartner subsequently predicted that 60% of AI projects would be abandoned through 2026 specifically because of data readiness problems. The underlying issue is not data engineering. It is governance: nobody defined what “AI-ready” meant, who was responsible for achieving it, or what the approval criteria were before deployment.
The Regulatory Landscape Reshaping Everything
For a long time, AI governance was framed as a competitive differentiator — something the more responsible organisations chose to do. That framing has been superseded by law. The European Union’s Artificial Intelligence Act entered into force on 1 August 2024, making it the world’s first comprehensive legal framework for AI. Its full applicability begins 2 August 2026, though key provisions — including the prohibition of unacceptable-risk systems and general-purpose AI model obligations — were already enforceable from February and August 2025 respectively.
The EU AI Act operates on a risk-tiered approach. Unacceptable-risk systems — those involving manipulative subliminal techniques or social scoring by public authorities — are banned outright. High-risk systems, which include AI used in employment decisions, credit scoring, educational assessment, and critical infrastructure, must meet strict obligations: robust risk assessment, high-quality training datasets, detailed technical documentation, human oversight mechanisms, and comprehensive post-market monitoring.
The IAPP’s 2025 survey found that 77% of organisations were working on AI governance, a figure that rose to nearly 90% among those already actively deploying AI. That is a significant number, but it also implies that nearly one in four organisations with AI in production has not yet begun the governance work the regulations now require of them.
Outside the EU, regulatory pressure is building through different channels. In the United States, the Senate voted 99–1 to strip a proposed federal AI moratorium from a 2025 budget reconciliation bill — but the near-unanimous rejection of preemption without a substantive federal replacement signals that national standards are coming. The Bipartisan Policy Center has noted that states will continue legislating regardless, and businesses are increasingly calling for the kind of consistency that only federal frameworks can provide.
The GDPR Parallel Worth Taking Seriously
When GDPR came into force in 2018, a considerable proportion of businesses treated it as bureaucratic overhead until enforcement actions started generating headlines. The compliance rush that followed was expensive, disruptive, and entirely predictable. AI governance appears to be following the same trajectory. The organisations doing the structural work now — building governance into their AI development lifecycle rather than grafting it on at the end — are spending less per project and accumulating substantially less regulatory risk than those waiting for enforcement pressure to force the issue.
Board-Level Accountability and Who Gets Left Holding the Risk
Deloitte’s 2025 global survey of 700 board directors and executives across 56 countries found that 66% of boards reported limited or no AI expertise among their members. Only 14% discussed AI at every board meeting. Nearly half had not placed AI on their agenda at all — even as AI systems were actively making consequential decisions inside their organisations.
That gap is not just a strategic problem. It is a legal liability. The Caremark doctrine in US corporate law — which holds directors personally accountable for failing to oversee mission-critical operational risks — is increasingly being applied to AI. As AI systems become central to core business functions such as lending, underwriting, hiring, and healthcare triage, the argument that a board should have known about AI governance failures becomes harder to dismiss in litigation.
“When boards lack visibility into AI systems, accountability becomes diffuse. When accountability is diffuse, nobody answers for what AI does after deployment. The risks accumulate silently until they surface as a crisis.”
— GossipWire Editorial Analysis
The EU AI Act has formalised this with explicit requirements. The Act elevates AI governance to a board-level responsibility, with directors facing potential personal liability under fiduciary duties if they demonstrably disregarded significant AI regulatory risks. Effective compliance under the Act requires clear accountability assignment, regular board-level reporting on AI risk, and measurable AI literacy among directors.
The ISACA identified a closely related problem in its December 2024 analysis: a significant portion of organisations simply cannot find people with adequate AI governance skills. In the IAPP’s governance survey covering 670 respondents from 45 countries, 23.5% cited finding qualified AI governance professionals as a core delivery challenge. The demand for that skill set has grown faster than the supply of people who understand both the technical behaviour of AI systems and the regulatory and ethical structures needed to govern them.
The Agentic AI Problem Nobody Has Solved Yet
Most existing AI governance frameworks were designed for a world in which AI generates outputs — text, classifications, recommendations — that humans then review and act upon. That world is receding. Agentic AI systems take actions: they execute transactions, trigger workflows, send communications, and make real-time decisions without waiting for a human to approve each step.
Consider a concrete scenario that has begun appearing in enterprise post-mortems: an autonomous procurement agent misreads pricing data during a high-volume trading period and executes purchase orders worth millions in excess inventory. The error is discovered 72 hours later. The question that follows — “who approved that action?” — has no clean answer, because the governance framework was designed for AI that generates recommendations, not AI that executes decisions. That is not a technology failure. It is a governance architecture failure.
Agentic AI requires action-authorisation before the fact rather than output-checking after it. That demands a governance model built around pre-approval thresholds, real-time audit trails, and circuit-breaker mechanisms — none of which are standard features of legacy compliance frameworks. The World Economic Forum has articulated this directly: AI governance cannot be a top-down mandate layered over an existing organisational structure. It has to be embedded in the operational architecture of how AI systems are designed, deployed, and monitored.
Data Governance as the Prerequisite
Before governance frameworks can address accountability and oversight, they have to address the foundational problem of data. Every AI model inherits the properties of its training data. Weak, biased, or poorly documented inputs produce unreliable outputs at scale. Gartner predicted in February 2025 that 60% of AI projects would be abandoned through 2026 specifically because of data readiness problems — not model capability, but the quality and structure of the data feeding the models.
The governance implication is direct: no organisation should be deploying AI systems at scale without a documented data governance policy that covers training data provenance, bias testing, ongoing data quality monitoring, and clear ownership of data correction when errors emerge. The EU AI Act specifically requires high-quality datasets and traceability for high-risk systems. Building those requirements into development pipelines from the start is substantially cheaper than retrofitting them under regulatory pressure. Deloitte’s 2024 analysis found that post-deployment governance retrofitting costs between three and five times more than building controls upfront.
What Governed Enablement Actually Looks Like
The organisations making measurable progress on AI adoption share a structural pattern that is not particularly glamorous: they treat governance as an operational discipline rather than a compliance checkbox. That means designated AI system owners with clear accountability, mandatory gates between experimentation and production, and governance structures that are embedded into GRC (governance, risk, and compliance) frameworks rather than maintained as separate AI-specific silos.
Effective AI governance requires coordination across legal, data science, privacy, product, and internal audit functions. The EU AI Act makes this explicit by linking AI governance to existing GDPR obligations — high-risk AI systems processing personal data must satisfy both a Fundamental Rights Impact Assessment under the AI Act and a Data Protection Impact Assessment under GDPR. The organisations that have integrated these requirements into a single unified assessment process are building sustainable compliance infrastructure. Those running them as separate projects are doubling their workload for no additional protection.
Pacific AI’s 2025 survey found that monitoring AI in production is the most commonly implemented governance control, at 48% of organisations, followed by risk evaluation at 45%. That ordering reflects the typical reactive pattern: the monitoring gets built after deployment, when the system is already operating in production. The organisations that are further ahead have inverted this sequence — they establish risk evaluation, documentation standards, and human oversight requirements before any system reaches production, treating deployment approval as the outcome of governance rather than the starting point for it.
The Skills and Talent Dimension
The OECD’s June 2025 report on governing AI in government identified a systemic risk that applies equally in the private sector: without proactive skills development, organisations will find themselves reacting to technological shifts rather than steering them. The report noted that governments face a particular disadvantage because hiring contractors for AI work can cost three to four times as much per person as employing permanent staff — meaning the skills gap has both structural and financial consequences.
The broader point extends beyond government. If an organisation’s board, legal team, and compliance function collectively lack the AI literacy to understand what its AI systems are doing, the governance framework will be built on guesswork. Sustained investment in AI literacy — not just for technical staff, but for the people responsible for oversight, accountability, and regulatory compliance — is a governance prerequisite, not an optional enrichment programme.
The observation made by academics and intellectuals who have written on institutional accountability for decades is relevant here: powerful systems that operate without adequate human oversight tend to generate the conditions for their own eventual constraint. The question for AI is whether organisations build that oversight proactively or have it imposed through regulatory enforcement and public failure.
Further Reading: Business Intelligence and Enterprise Strategy
For deeper analysis of how institutional structures shape enterprise performance across industries, the growth enterprises market overview provides useful strategic context for understanding where AI governance intersects with broader questions of organisational maturity and competitive positioning.