A confession to start with
A few months ago I was reviewing access groups in a healthcare cloud environment — the kind of unglamorous IAM work that keeps regulated companies out of the news. Service accounts, group membership, a BigQuery dataset here, a storage bucket there. Routine.
Then it occurred to me that one of the "users" I was about to grant access to was not a user at all. It was an AI agent. It would read data, make decisions, and call other systems on its own, around the clock, faster than any human on the team.
I had spent eight years building a mental model of who gets access to what and why. And in about thirty seconds, that model quietly broke.
This post is what I learned putting it back together. If you are deploying AI inside a company — especially a regulated one — I want you to finish this with one specific feeling: the slightly cold realization that some of what you already have in production deserves a second look. Not panic. Just a clear head and a plan.
Part 1: Why AI is not "just another application"
Every few years our industry gets a new toy and insists it changes everything. Most of the time it doesn't. We wrap it in the controls we already have and move on. Cloud was mostly like that. Containers were mostly like that.
AI agents are not like that, and here is the precise reason.
Every piece of software you have ever secured has one thing in common: a human wrote down, in advance, everything it is allowed to do. The logic is fixed. You can read the code. Given the same input, you get the same output. Security, at its core, has always been the art of drawing a box around predictable behavior.
An AI agent breaks the one assumption underneath all of that. It is non-deterministic. It decides what to do at runtime, based on language it just read. The same request can produce different actions on Tuesday than it did on Monday. It interprets instructions instead of executing them.
Traditional App
= A Train
Fixed path. Predictable. Secure the track.
AI Agent
= A Driver
Same destination. Different path every time. Secure the decision-maker.
One more shift, and it's the important one. For decades the dangerous attack surface was code — and we got good at defending it. The new attack surface is language. An AI agent treats the text it reads as instructions. That text can come from a document, an email, a webpage, a database record, the output of another agent. Which means a sentence buried in a PDF can now be an attack. We have spent thirty years learning not to trust user input. We now have to learn not to trust content.
Part 2: The old foundations still hold — use them
Here's the good news, and it's the part the hype merchants skip: the fundamentals of security did not get repealed. They matter more now. AI doesn't replace your foundations — it stress-tests them and finds every place you were sloppy.
CIA — Confidentiality, Integrity, Availability
The triad isn't a poster on the wall. It's a checklist for every AI capability you ship. The flavor of each risk shifts with agents:
- Confidentiality — agents are talkative. They will summarize whatever's in their context window, including the sensitive bits, to whoever frames a clever enough request.
- Integrity — an agent can be confidently wrong. Poisoned training data, a tampered document in a vector store, or a manipulated tool response all turn the agent into a high-fidelity liar.
- Availability — agents are expensive. A million junk queries is a credit-card denial-of-service. The attacker doesn't take you down. Your finance team does, after the bill.
Hover each vertex to see how AI changes the risk.
IAAA — Identity, Authentication, Authorization, Accountability
This is the model AI breaks the most cleanly. IAAA assumes the principal is a person or a well-defined service. An autonomous agent is neither. It's a long-running reasoning loop with persistent credentials and unpredictable behavior, and we're issuing it the same kinds of tokens we issue to humans.
Every line of IAAA has to be re-asked for agents: how do we identify them uniquely, authenticate them strongly, authorize them per-action (not per-session), and hold them accountable with logs we can actually reconstruct an incident from?
The Evolution of Identity
Human Identity
Employees, contractors
Decades old
Service Account
System-to-system auth
20+ years
Machine Identity
VMs, containers, workloads
Cloud era
Autonomous Agent
Reasons. Decides. Chains actions.
THE NEW CITIZEN
Not "non-human" — we've done non-human. Autonomous.
Principle of Least Privilege
Humans tolerate extra access. They don't notice the doors that don't open in their daily workflow. Agents are different. An agent given access to ten systems will eventually touch all ten — either because it reasoned its way there, or because someone told it to.
Human with Excess Access
Rarely notices the extra doors.
Agent with Excess Access
Finds every door. Tries every door. That's what it does.
Separation of Duties & People-Process-Technology
Don't let one agent both draft and approve. Don't let one agent both produce data and ship it. The same boring SoD rules that prevent fraud in finance teams prevent runaway behavior in agent fleets.
And remember the oldest formula in the field: People, Process, Technology. Buying an AI gateway doesn't fix governance gaps. Someone has to own the agent. There has to be a process for changes. Tools come last.
Part 3: Zero Trust gets a second life
For years "Zero Trust" was a marketing word stapled to networking gear. With agents it suddenly becomes literal and unavoidable. Trust no principal implicitly — verify every action, every time, in context.
That means per-call authorization, not "the agent logged in at 8 AM and is now empowered for the day." It means continuous evaluation of whether the session still looks like the agent it started as.
Is this action authorized?
Is the context still safe?
Has the session been compromised?
Is this consistent with baseline behavior?
Zero Trust means: verify every action, not just the login.
Part 4: Environments — keep AI in a real sandbox
The first time an agent's tool call hits production is the wrong time to find out it deletes things. Every AI capability needs to ride the same dev → staging → prod pipeline as the rest of your stack, with progressively realistic data and progressively narrower blast radius.
In healthcare we don't let new ETL jobs touch real PHI in development. The same rule applies to agents: synthetic data first, scoped credentials second, production access last, and only behind change control.
Part 5: How AI systems are actually built
You don't have to understand the math to secure this. You do have to understand the shape. A modern AI system has at least four moving parts: a model (the brain), a prompt/context (the instructions and recent memory), tools (functions it can call — databases, email, APIs), and a data layer (often a vector database).
Every one of those is a control point. The model can be jailbroken. The context can be polluted. The tools can be overprivileged. The data layer can be poisoned. If you don't know which of those four your team has shipped, you don't know what you're protecting yet.
Part 6: RAG — the most common pattern, and the most quietly dangerous
Retrieval-Augmented Generation is the dominant pattern in enterprise AI right now. It works because it grounds the model in your documents instead of making things up. It's also a beautiful supply-chain attack surface.
RETRIEVE
Vector DB
AUGMENT
Q: user question
+ context
GENERATE
The threat model is simple: control any of the documents in the vector store, and you control part of the answer. In regulated environments, "the document was wrong" is not a defense — the answer went out to a clinician, a customer, an auditor. The output is the liability.
Part 7: How an AI incident actually escalates
Every senior responder has the same nightmare: a breach without a single exploit in it. With agents, that's the default shape. Each step is a normal, authorized action by a legitimate identity. Strung together, they're a breach.
Anatomy of an AI-Driven Breach
Every step uses authorized actions by a legitimate identity.
Foothold
Agent reads email containing hidden instruction. Treats it as part of its task.
Discovery
Agent inventories everything it can reach — tools, systems, data. In seconds, not hours.
Lateral Movement
Uses legitimate access to reach further — from email toward the database, toward other systems.
Exfiltration
Uses its email tool — exactly as designed — to send data out.
No malware. No exploit. No alarm. Every step was an authorized action by a legitimate identity.
This is why "the firewall didn't catch it" is not a useful comment in an AI postmortem. There's nothing for the firewall to catch. The control plane has to be identity, authorization, and behavior — not packets.
Part 8: We are not making this up — ATLAS and NIST exist
Two frameworks are worth knowing by name even if you never get past the table of contents.
MITRE ATLAS is ATT&CK for AI — a structured catalog of how attackers reconnaissance, gain initial access, evade, and exfiltrate against AI systems specifically. If your team is mapping detections, this is the matrix to map against.
NIST AI RMF is the governance counterpart — a risk-management framework you can actually point an auditor at. It's the closest thing the industry has to a shared vocabulary for "we took this seriously."
You don't need to adopt either one purely. You do need someone on the team who has read them, so when leadership asks "are we aligned with anything," you have a real answer.
Part 9: A plan of action you can actually run on Monday
If you only take one thing from this post, take this: there is a sane order to do this work in, and skipping ahead is what creates the breach stories. Visibility before governance. Governance before identity. Identity before runtime defenses. Resilience underneath all of it.
You cannot secure what you cannot see.
Inventory every AI system. Map what each touches: data, systems, permissions. Find the shadow AI.
Click each phase to explore. The order is the point — each depends on the one before it.
Where this leaves us
AI doesn't make security obsolete. It makes the boring parts urgent. The teams that win this next chapter are not the ones with the flashiest models. They're the ones who can answer four questions at any time of day: What agents do we run? Who owns them? What can they reach? How would we know if one went wrong?
If those four questions feel hard to answer in your environment, you're not behind — you're normal. Everyone is figuring this out at the same time. The advantage goes to whoever starts, honestly, with what they actually have in production today.
The new insider isn't an employee. It's a process with a credential, running 24/7, that you taught to read instructions from anywhere. Treat it accordingly.

Written by Salman Tabrez
Senior Cloud Security & DevSecOps Engineer · 8+ years in healthcare enterprise security
Read Next
Why IAM Sits at the Center of Modern Security Architecture
Identity is the new perimeter. The architectural reality of 2026.