Securing AI Agents in the Enterprise

The essentials in 5 points

• An AI agent is not a chatbot: it acts on your systems. The risk lies in what it does, not in what it says.
• The right approach is to treat your own agent as a potentially compromised component, not as a trusted tool.
• There is no such thing as a “tamper-proof” prompt. Only deterministic guardrails, independent of the model’s good will, withstand an attack.
• Secrets (keys, passwords) must never enter the model’s context.
• Before any tool: a threat model. Without a map of the attack surface, tools add alerts without reducing risk.

The short answer

Securing an AI agent is not about writing a better prompt. An agent holds real access: files, databases, business APIs, sometimes sending emails or running commands. The security question is therefore not “what can it say?” but “what can it do, triggered by whom, and what stops it?”.

An effective approach has three stages: map the attack surface (which agents, which access, which inputs), instrument in observation mode to understand the real traffic, then harden layer by layer with controls that do not depend on the model. The rest of this article spells out what that means for a company that is deploying, or about to deploy, agents.

Why an AI agent is not secured like an application

A conventional application does what its code specifies. An AI agent interprets natural-language instructions, including the ones it comes across in the data it processes: a web page, an incoming email, a customer document, a dependency’s README. That is the mechanism of prompt injection: text the agent reads becomes an instruction it executes.

You do not patch this mechanism, it is intrinsic to the technology. Serious vendors in the field draw the same conclusion: you have to model your own agent as a potentially compromised component. The entire security architecture follows from that reversal. You are not trying to make the agent incorruptible, you are bounding what a corrupted agent could do.

The worst pattern, the one that keeps showing up in recent public incidents: irreversible write access to a sensitive resource, reachable from an unauthenticated input. All three conditions are necessary. If your agent can delete production data and reads content that anyone can send it, you are exactly in this situation.

The three trade-offs to settle before tooling up

Probabilistic or deterministic? An AI-based filter (a classifier, an LLM judge) catches a wide range of attacks but can be bypassed. A hard rule that blocks any destructive command is unbreakable but narrow. The answer is not one or the other: it is stacking both. The probabilistic filter stops a large share of attacks, the deterministic rule guarantees a minimum level of protection.

What do you do when a control fails? When a scanner refuses to analyze a file, or a filter cannot decide, two behaviors are possible: let it through (fail-open) or route it to human review. For an agent that acts on real systems, fail-open is a mistake: a refusal to analyze is a warning signal, not an absence of a result.

Observe or block? Switching straight to blocking mode disrupts production and turns teams against you. The right sequence: log first (who calls what, which tools, which data), establish a baseline, then switch the controls into blocking mode one at a time.

The seven layers of defense in depth

1. Threat model. Map every agent: its access, its inputs, its possible side effects. The classic frameworks (OWASP, STRIDE) apply, rounded out with agent-specific concerns (injection through data, the tool chain).
2. Admission control. Scan everything entering the fleet before it runs: MCP servers, plugins, skills, instruction files. An agent configuration file is executable code and must be audited as such, including for invisible Unicode characters that hide instructions.
3. Runtime inspection. A checkpoint that observes prompts, responses, and tool calls, with rules and a classifier. At scale, the effective pattern is a cascade: a lightweight filter on all traffic, a heavyweight filter on what looks suspicious.
4. Deterministic guardrails. Hard blocking of destructive commands (mass deletion, writes to production, transfers) at the harness level, not the prompt. This layer does not depend on the model’s good faith: it does not hallucinate.
5. Secrets out of the model. API keys and passwords must never appear in the agent’s context. They are injected at the moment of the network call by a separate component, out of reach of a prompt injection.
6. Isolation and out-of-band human approval. Least privilege per connector, execution of critical components in an isolated environment, and for any irreversible action, a human confirmation on a channel separate from the one where the agent receives its instructions.
7. Durable logging. Record tool calls and agent traffic in a usable format (SIEM export). This is what lets you detect abnormal behavior, respond to an incident, and prove compliance.

The two most common traps

Mistaking an evaluation score for real security. An agent that scores well on a security benchmark is not a secure agent: evaluations can be gamed, and a model can deliberately underperform on the tests that measure it. Before a sensitive deployment, independent verification (a red team) remains necessary.

Believing a model’s built-in guardrail is enough. Alignment-based protections come off in a matter of tens of minutes when an adversary has access to the weights, and are bypassed by injection when there is no such access. They are useful, but real protection comes from the external layers: access control, deterministic guardrails, isolation.

Where to start

If your teams already use agents (code assistants, business agents, automations), the first step is to take stock: which agents run, with what access, and what triggers them. This few-day exercise yields a map of the attack surface and a list of fixes ranked by priority; it is carried out in observation mode, without touching production.

Colombani.ai carries out this kind of AI security audit (surface mapping, agent-assisted penetration testing, covered by professional liability insurance dedicated to penetration testing) and trains technical teams through the AI agent security course (OWASP LLM Top 10, red teaming, sandboxing), delivered by a Qualiopi-certified provider and eligible for OPCO funding.

Sources

• OWASP, Top 10 for Large Language Model Applications
• NIST, AI Risk Management Framework
• Anthropic, Building trustworthy agents