Connecting Legacy Systems to AI Agents: Integration Architecture Strategies

Every enterprise architecture conversation today ends in the same place: how do we get AI agents working against our real systems? Not demo systems. The systems that hold your orders, claims, and customer records, many of which have been running for twenty years or more.

The honest answer is uncomfortable. The systems that matter most to your business are the hardest ones to connect to AI, and replacing them is not a realistic short-term option. This article argues that you should stop treating replacement as the prerequisite for AI adoption. Instead, make your legacy systems AI-ready by introducing a middle integration layer and let legacy and modern systems coexist while each does what it does best.

Why you can't just replace legacy systems

The systems holding your most valuable data are end systems: ERPs, CRMs, core banking, and claims platforms. Replacing one of these is a multi-year program, not a project, and the reasons have little to do with the technology being old.

Start with the business logic. Decades of rules, edge cases, and regulatory requirements are baked into these systems, much of it undocumented, some of it understood only by people who have retired. An ERP customized for fifteen years is not a product anymore; it is a record of how your company actually works. Re-implementing that faithfully, without quietly dropping a rule that some downstream process depends on, is the hard part of any replacement.

Then there is the data. Systems of record accumulate years of transactional and master data that have to be migrated, reconciled, and validated against the old system before anyone trusts the new one. This is routinely the phase that overruns, because the data is messier than anyone admitted.

On top of that sits the dependency web. A core ERP or CRM is connected to dozens of other systems, and every one of those integrations has to be re-pointed and re-tested. The system you want to replace is rarely the only thing that changes when you replace it.

Finally, these systems do not tolerate downtime. You cannot take core banking offline for a weekend to swap it out. Replacement happens through long parallel runs and phased cutovers precisely because a big-bang switch of a payments or claims system is how outages and lost transactions happen.

None of this means you shouldn't modernize. It means modernization runs on its own clock, and that clock is slow by necessity.

The AI era doesn't wait

Here is the tension. Your migration roadmap says three years. Your business says now.

AI agents are already capable of resolving customer issues, processing documents, and orchestrating workflows. But an agent is only as useful as the systems it can reach, and the highest-value data and transactions in your enterprise sit inside exactly those legacy systems you cannot quickly replace. If AI readiness has to wait for migration to finish, you forfeit years of business value, and you hand that time to competitors who found another way.

So the real architectural question is not “how fast can we replace legacy?” It is “how do we make legacy usable by AI agents today, without destabilizing it?”

The middle layer: making legacy AI-ready without replacing it

The answer is to put a modern integration layer between your AI agents and your legacy systems. Agents never call legacy interfaces directly. The middle layer does four jobs:

It translates protocols

Agents consume tools described in JSON, increasingly over the Model Context Protocol (MCP). Legacy systems speak SOAP, message queues, EDI, and file transfers. The integration layer bridges the two, which is the same transformation and connectivity work integration platforms have always done.

It adds the meaning agents need

A legacy operation with cryptic field names is unusable to a model deciding which tool to call. In the middle layer, you wrap that operation as a clearly named, well-described tool with typed parameters. This is where your team's institutional knowledge about what the system actually does becomes explicit and readable.

It reshapes granularity

Legacy interfaces are often too fine-grained or return far more data than an agent needs. The middle layer composes multiple backend calls into one coherent tool and filters responses down to what is relevant, so the agent gets a usable answer in one call.

It protects the backend

Agents are non-deterministic callers. They retry, they explore, and they generate load patterns your legacy system was never tested for. Throttling, caching, and circuit breaking in the middle layer mean the system of record never has to absorb that unpredictability.

The practical effect: the thousands of services and integrations you already run become a catalog of AI-ready tools, without touching the legacy code behind them.

Five patterns that make legacy agent-ready

Inside the middle layer, a small set of integration patterns covers most legacy-to-agent scenarios.

Wrap one operation as one tool

The simplest pattern: take a single legacy operation, such as a policy lookup or a balance check, and expose it as one well-described tool. It is low effort and low risk, which makes it the right starting point, but it inherits the granularity of the underlying interface and is rarely enough on its own.

Compose many calls into one tool

When answering a realistic agent question takes several backend calls, do the orchestration in the middle layer and expose one coarse-grained tool, such as a customer summary that fans out to the CRM, billing, and ticketing systems. The principle behind this pattern matters more than the pattern itself: deterministic orchestration belongs in the integration layer, and judgment belongs in the agent. Letting the agent stitch together even low-level calls itself is slow, expensive, and error-prone.

Bridge batch into real time

Many legacy flows are batch jobs, file drops, or queue-based exchanges that cannot answer an agent synchronously. The fix is to feed an operational data store from those flows and expose query tools against it. Make the freshness window explicit in the tool's response, for example, “data as of 06:00”, so the agent can reason about staleness instead of assuming the data is current.

Put a human in the loop for writes

For state-changing operations, the middle layer pauses the flow, routes an approval to a person, and resumes or aborts based on the decision. The agent simply sees a tool that returns “pending approval” and later a final outcome. This pattern adds latency, and it is also the single most effective way to earn stakeholder trust for agent-driven writes.

Expose legacy data for retrieval, not just transactions

Some of what agents need from legacy systems is knowledge rather than action: contract terms, historical records, product documentation locked in old databases. The middle layer can serve this through governed query tools, or by syncing selected slices of legacy data into a search or vector index that agents retrieve from. The data never has to migrate, but you do have to decide deliberately which data is exposed, because this pattern is where data governance and AI governance meet most sharply.

Governance, security, and trust

Exposing core systems to autonomous software raises the stakes on governance, and the mediation layer is where that governance lives.

Identity for agents

Every agent needs its own identity, not a shared service account, and certainly not a human's borrowed credentials. OAuth2 with fine-grained, per-tool scopes is the workable baseline; token exchange allows an agent acting on behalf of a user to carry both identities downstream, so the legacy system's existing authorization logic still applies. The question every audit will ask is: which agent, acting for whom, did what, and why was it allowed? The architecture must be able to answer it.

Authorization and blast-radius control

Define, in policy rather than in prompt text, what each agent may never do. Prompts are suggestions; gateway-enforced policy is a guarantee. Read/write separation, value thresholds, environment restrictions, and explicit deny-lists per agent identity all belong here.

Auditability and observability

Agent-initiated transactions need end-to-end traceability: the originating conversation or task, the reasoning step, the tool call, the mediation flow, and the backend transaction, correlated under one trace. When an agent does something unexpected, the difference between a quick diagnosis and a week of detailed analysis is whether this correlation exists.

Rate and cost control, in both directions

Throttle agent traffic to protect legacy capacity, and meter tool usage to control LLM and infrastructure spend. Agents in retry loops are a new and creative way to burn tokens.

A reference architecture for coexistence

Putting the pieces together, the architecture has four layers. Agents sit at the top and legacy systems at the bottom, and they never touch each other directly.

Figure1: Four layers separate agents from legacy systems,so you can swap in a modern backend and nothing above is aware.

The flow is consistent regardless of what sits at the bottom. An agent calls a tool. The governance layer checks who the agent is and what it is allowed to do, then passes the call to the mediation layer. The mediation layer translates the request into whatever the backend understands, whether that is a SOAP call, an MQ message, or a database query, and then shapes the response into something the agent can reason over.

The dotted line that matters most in this diagram is the one between the mediation layer and the legacy systems. That is your migration boundary. Everything above it stays stable while everything below it changes. When you eventually move a backend from the legacy platform to a modern one, you re-point the mediation layer, and nothing upstream notices.

Coexistence is the strategy, not a compromise

Architects sometimes treat this middle layer as scaffolding, a temporary fix until "real" modernization finishes. That framing undersells it. Coexistence is the strategy.

Legacy systems remain what they are good at being: stable, proven systems of record. Modern AI agents do what they are good at: reasoning, orchestrating, and interacting with users. The integration layer in between lets each side play to its strengths and deliver value on day one, rather than forcing a choice between them or waiting for one to replace the other.

This is not a stopgap. Your systems of record will change over time: some modernized, some replaced, some retired, but there will always be a mix of older and newer systems behind your agents, because enterprises never finish modernizing all at once. The enduring part is the pattern: a stable layer in between that lets the systems behind it evolve on their own timelines while the experiences in front of it keep moving. That layer is strategic architecture, not temporary scaffolding.

Crawl, walk, run: earning agent autonomy in stages

The AI enablement track needs its own phased plan. Trying to launch fully autonomous agents against core systems on day one rarely ends well: the agent does something unexpected against a system no one was watching closely enough.. Three stages of work:

Crawl: read-only tools over what you already have

Expose your existing services and integrations as read-only tools, ideally generated rather than hand-built. The risk is minimal, and your team learns what good tool design looks like. Most organizations are surprised by how much value pure read access unlocks: status checks, lookups, and summaries cover a large share of what users actually ask agents for.

Walk: writes behind approval gates

Introduce state-changing tools, every one of them behind the human-in-the-loop pattern described above. This stage builds the audit trails, the approval workflows, and the organizational confidence that autonomy later depends on. Measure approval rates per tool. When a tool's actions are approved 99% of the time for months, you have evidence rather than opinion for the next stage.

Run: policy-governed autonomy

Remove human gates selectively where the data supports it, replacing them with automated policy checks and value thresholds enforced in the governance layer. Autonomy is granted per tool and per context, never globally, and the gates can always come back.

Each stage delivers working capability on its own. You are not waiting for some end state to see value.

Conclusion: build the layer, not the rip-and-replace

The case for rip-and-replace is not going away. There will always be a vendor, a consultant, or an internal champion arguing that the right answer is to start clean. Sometimes they are right. But for most enterprises, that argument is a reason to delay, and delay has a cost that compounds every quarter AI capability advances.

The integration layer described here does not prevent modernization. It makes modernization optional as a prerequisite. Your teams can begin delivering agent-driven value against the systems you have, while the longer migration programs run on their own timelines in the background.

Build the layer. Start with read-only. Earn autonomy in stages. The architecture will outlast any individual system behind it.

Take the next step

See how WSO2 Integration Platform connects AI agents to legacy systems without disrupting what's in place. If you're designing the integration layer for your agentic enterprise,reach out to our team.

→ Contact WSO2