---
Title: "AI agent security review: vendor risk assessment for CISO teams"
Url: "https://devrev.ai/blog/ai-agent-security-review"
Published: "2026-09-17"
Last Updated: "2026-09-17"
Author: "Nivedita Bharathi"
Category: "Blog, Computer"
Excerpt: "An agent's attack surface isn't a model's. Here's the vendor-risk checklist CISO teams use to review AI agent platforms: what to verify, what to ask, and what good looks like."
Reading Time: 10
---

# AI agent security review: vendor risk assessment for CISO teams

If your AI agent security review uses the same checklist as your SaaS vendor assessment, you’re missing the attack surface. Agents call tools, modify data, persist memory, and inherit permissions, and those are threats a model-focused review never catches. This is the vendor-risk checklist built for what agents actually do, as of September 2026.

## TL;DR – the security review in 30 seconds

- An AI agent security review is not a model security review. The attack surface is the agent’s ability to act, not just generate text.
- The agent-specific risks model checklists skip: tool-calling exploits, memory poisoning, permission escalation, and prompt injection in an agentic context.
- Use the [vendor risk checklist](#the-vendor-risk-checklist) below to score any vendor on 10 areas, each with the exact question to ask. That column is what separates enforcement from policy.
- Map the OWASP LLM Top 10 to your agent’s attack surface, then test the vendor’s claims in a sandboxed proof of concept before anything touches production data.

## What is an AI agent security review?

An AI agent security review is a vendor risk assessment that evaluates how an agent platform handles data, governs actions, manages identity and permissions, maintains audit trails, and meets compliance standards. It focuses on the agent-specific attack surface, where the agent can take real actions, not just the risks of the underlying model.

## Why agent security ≠ model security

Most AI security guidance still describes a model: it hallucinates, leaks training data, or repeats something biased. Those are text problems. An agent is different. It calls tools, runs queries, modifies records, and acts across your stack. The question shifts from “what might it say?” to “what can it do, and who decides?” That’s the CISO’s real worry, and it’s why a model-focused checklist leaves the biggest risks unexamined. The reframe your review needs: show me the enforcement, not the policy.

### The agent-specific attack surface

Four categories separate agent security from model security. Each exists because the agent can act on the world, not because the model generates weak text.

### Tool-calling exploits

An agent with tool access can be manipulated into executing unintended actions: SQL queries, API calls, file operations, or record changes. The threat isn’t a bad sentence, it’s a real action on your systems, one that can delete data, move money, or expose a customer record. Ask what the agent is allowed to call, under what conditions, and what stops a malicious instruction from becoming an executed action.

### Memory poisoning

If the agent persists memory across sessions, an attacker can plant information that influences future interactions, not just the current one. A poisoned memory is worse than a poisoned prompt because it lingers: one crafted input can shape every later decision for that user, team, or account. Ask how memory is written, who can write to it, whether writes are attributed and auditable, and how the platform contains a poisoned entry before it spreads.

### Permission escalation

If the agent runs on a shared service account instead of the requesting user’s inherited permissions, every user effectively gets the agent’s permission level. A junior support rep and a finance admin end up with the same reach, because the agent, not the person, is the identity doing the work. This isn’t hypothetical: [Cloud Security Alliance research](https://cloudsecurityalliance.org/press-releases/2026/04/16/more-than-half-of-organizations-experience-ai-agent-scope-violations-cloud-security-alliance-study-finds) found more than half of organizations (53%) have had AI agents exceed their intended permissions. Ask whether the agent executes as the requesting user or as its own account, and how downstream systems see that identity.

### Prompt injection in agentic context

Prompt injection against an agent is more dangerous than against a chatbot, because the agent can act on the injected instruction. A tricked chatbot produces bad text; a tricked agent can execute a tool, read data it shouldn’t, or modify a record. The blast radius is the agent’s full set of permissions and tools. Ask what detection fires before a tool call, and whether an injected instruction can reach an action path without a human in the loop.

## The vendor risk checklist

Each row is independently useful. The “question to ask the vendor” column is the real value, because it forces a “show me” answer instead of a “trust us” one.

| Security area | What to verify | Question to ask the vendor |
| --- | --- | --- |
| Data handling | Data isolation, encryption at rest and in transit, no training on customer data | “Is our data used to train models? Show us the data isolation architecture.” |
| Data residency | Region-specific storage, compliance with local regulations | “Where is our data stored? Can we specify the region?” |
| Action boundaries | Human-in-the-loop approval for high-risk actions, configurable autonomy levels | “Show us the human-in-the-loop approval flow for a payment action.” |
| Permission model | Inherited user permissions, not a shared service account | “Does the agent execute with the requesting user’s permissions or its own?” |
| Identity & auth | SSO and SCIM, per-agent identity, integration with enterprise IAM | “How does the agent authenticate to downstream systems?” |
| Audit trail | Immutable logs, session traces, guardrail triggers visible end to end | “Show us the audit trail for a completed action, from trigger to execution.” |
| Guardrails | Content filtering, prompt-injection detection, PII masking | “What guardrails fire before the agent executes a tool call?” |
| Compliance | SOC 2 Type II, ISO 27001, GDPR, and HIPAA where applicable | “Share your SOC 2 Type II report and your last pen-test summary.” |
| Incident response | Published SLA, defined escalation path, breach notification process | “What is your incident response SLA and breach notification timeline?” |
| Model governance | Model-agnostic, no vendor lock-in on the foundation model | “Can we bring our own model? How do you handle model updates?” |

## The questions that separate enforcement from policy

Every vendor has a security page. The questions that separate real enforcement from a policy page ask “show me” instead of “tell me.”

### Data handling and residency

Start with the two claims that matter most: does the platform train on your data, and where does your data physically live? A trustworthy answer comes with an architecture diagram and a residency option, not a marketing sentence. Ask how tenant data is isolated, how it’s encrypted at rest and in transit, and how deletion works when you off-board. If a vendor can’t show you the data isolation model, treat that as unresolved.

### Action boundaries and HITL controls

This is where agent security lives or dies. Ask the vendor to demonstrate the human-in-the-loop approval flow for a genuinely high-risk action: a refund, a payment, or a record deletion. Good looks like configurable autonomy, where the agent proposes, a human approves, and the platform enforces the boundary, not a prompt asking the model to behave. “The agent is instructed not to do that” is weak. Instructions aren’t controls.

### Identity, authentication, and permissions

Ask one question and listen hard: does the agent execute as the requesting user, or as a shared service account? Inherited permissions mean the agent can only do what the person behind it can. A shared service account means every user borrows the agent’s reach, the seam where permission escalation hides. For how agents get their own identity and scoped permissions, see [non-human identity](https://devrev.ai/blog/ai-agent-security); this review just verifies the agent has one.

### Audit trail and observability

Ask to see a real audit trail for a completed action, from the triggering event through every tool call to the final execution. Good looks like an immutable log you can’t quietly edit, session traces you can replay, and visible records of which guardrails fired. A summary that can be modified after the fact won’t hold up in an incident review. The design of the [AI guardrails](https://devrev.ai/blog/ai-agent-security) themselves is a separate deep-dive; here you’re confirming the triggers are logged.

### Compliance certifications

Ask for the actual reports, not the badges. A SOC 2 Type II report covers a period of operation, not a point in time, so request the current one. Add ISO 27001 for information-security management, GDPR for data subject rights, and HIPAA if you handle protected health information, then ask for the latest penetration-test summary. Certifications tell you a vendor has been audited; the reports tell you what the audit found.

## Testing in practice – the security POC

A security questionnaire only gets you so far. The strongest reviews test the vendor’s claims in a sandbox before production data is involved. A CISO should be able to run agent behavior in an isolated environment, verify the audit trail captures every action end to end, and confirm human-in-the-loop controls actually fire when a high-risk action is attempted.

Computer, by DevRev supports this hands-on check. In Agent Studio, a security team can exercise an agent in a Playground and eval environment, watch the immutable audit trail record each step, and confirm the approval controls behave as documented. That’s the pattern to look for in any platform: a place to test the enforcement yourself, not just read about it. The enforcement a review should be able to reproduce is consistent: human-in-the-loop approval, inherited user permissions rather than a shared account, and an audit trail you can’t rewrite.

For Computer’s own posture, the public [DevRev trust center](https://security.devrev.ai) documents SOC 2 Type II, ISO/IEC 27001, GDPR, and HIPAA coverage, plus the commitment to never train on your data. It’s what a verifiable compliance answer looks like: a public, standing reference, not a claim in a sales deck.

## OWASP LLM Top 10 – how it maps to agents

The [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) is the authoritative reference for LLM risk. The point isn’t to reproduce the list, it’s to show how the agent’s ability to act turns several entries from a text problem into an action problem.

- **LLM01: Prompt injection** – against an agent, an injected instruction doesn’t just alter output, it can hijack a tool call. Maps to your prompt-injection and action-boundary checks.
- **LLM05: Improper output handling** – an unvalidated model output feeding a downstream tool call is how a bad response becomes a bad action. Maps to your guardrails and action-boundary review.
- **LLM06: Excessive agency** – overly broad tool and action permissions are the single most agent-relevant entry. Maps to your permission model and inherited-permissions check.
- **LLM07: System prompt leakage** – a leaked system prompt exposes the agent’s architecture and tool inventory, handing an attacker a map. Maps to your data-handling and observability review.
- **LLM10: Unbounded consumption** – recursive agentic loops can run away with resources and cost. Maps to your action-boundary and incident-response checks.

Your checklist is the framework; OWASP adds shared vocabulary and external rigor for the vendor conversation.

## FAQ

### What’s the biggest security risk with AI agents?

Tool-calling with inadequate permission controls. An agent that can execute actions, and does so with broad or shared permissions, can cause real-world damage from a single manipulated instruction. Hallucination is a model problem; the agent-specific risk is unbounded action.

### How do you assess AI agent vendor risk?

Use a checklist built for the agent attack surface: data handling, data residency, action boundaries, permission model, identity and auth, audit trail, guardrails, compliance, incident response, and model governance. For each area, ask a “show me” question, require evidence, then test the answers in a sandboxed proof of concept.

### What compliance certifications should an AI agent platform have?

At minimum, SOC 2 Type II and ISO 27001, plus GDPR alignment. Add HIPAA if you handle protected health information, and data-residency options if you operate in regulated regions. Ask for the actual reports and the latest penetration-test summary, not just the logos.

### How is AI agent security different from LLM security?

LLM security focuses on the model: hallucination, data leakage, biased output. Agent security focuses on action: tool-calling exploits, memory poisoning, permission escalation, and prompt injection that reaches an execution path. The attack surface is what the agent can do, not just what it can say.

## Sources and methodology

This checklist draws on the OWASP LLM Top 10 and anonymized enterprise security-review patterns from regulated buyers. Compliance references for Computer trace to the public [DevRev trust center](https://security.devrev.ai). This review feeds the security dimension of the [vendor selection scorecard](https://devrev.ai/blog/ai-agent-vendor-selection-scorecard) and sits inside the broader [build-vs-buy framework](https://devrev.ai/blog/build-vs-buy-ai-agents). For agent security concepts rather than this buyer’s checklist, see [AI agent security](https://devrev.ai/blog/ai-agent-security).

The stakes are widely felt: [the Cloud Security Alliance and Oasis Security](https://cloudsecurityalliance.org/press-releases/2026/01/27/79-of-it-pros-feel-ill-equipped-to-prevent-attacks-via-nhi-csa-oasis-survey-finds) found 79% of organizations rate their confidence in preventing attacks via non-human identities as low or moderate.

A security review for AI agents isn’t harder than one for traditional software, it’s different. The attack surface is the agent’s ability to act, not just generate. The checklist above gives your team the right questions. Next: feed the security dimension into the vendor scorecard, and test the vendor’s claims in a sandboxed POC before anything touches production data.