02Program
AI security services: LLM application testing, threat modeling and governance
Security for organizations building with and adopting AI: threat modeling for LLM applications, AI usage governance, data-leakage controls and security assessment of AI-enabled systems.
- Name
- AI Security
- Line
- 02 · AI
- Type
- Outcome program, combines several practices
01The problem
AI adoption is moving faster than AI governance.
AI adoption is outrunning AI governance. Employees paste sensitive data into public tools, product teams ship LLM features without threat models, and prompt injection turns helpful assistants into data-exfiltration paths. The risks are real, and so is the cost of banning tools that people will use anyway.
02Scope
What the AI security program covers.
- AI usage governance and policy
- LLM application threat modeling
- Prompt injection and adversarial testing
- Data-leakage controls for AI workflows
- Third-party AI tool risk assessment
- Secure architecture review for AI features
03Approach
How an AI security engagement runs.
Grounded in published guidance including the OWASP Top 10 for LLM Applications, MITRE ATLAS, the NIST AI Risk Management Framework and ISO/IEC 42001. Testing methods draw on [AI security research](/security-labs/ai-security) in Security Labs.
Govern usage first
AI usage policy, approved-tool guardrails and data-handling rules that let people use AI safely instead of secretly.
Threat-model AI features
LLM applications assessed for prompt injection, data leakage, excessive agency and insecure output handling before launch.
Test and monitor
Adversarial testing of AI-enabled systems and monitoring patterns for the failure modes unique to AI. Fixes can be built with our AI engineering team.
04Outcomes
What the AI security program is built to achieve.
- AI usage governed by policy and guardrails, not prohibition
- LLM features threat-modeled and tested before release
- Sensitive data protected across AI workflows
- A defensible answer when customers and auditors ask about AI risk
When this work fits
- Product teams adding an LLM, RAG or agent feature to a customer-facing application
- Organizations whose staff use public AI tools with company or customer data
- Security teams asked to approve an AI system without a method for assessing it
- Companies preparing for ISO/IEC 42001 or answering customer questions about AI risk
05Questions
AI security: questions we are asked
A standard test checks authentication, authorization, injection and logic. An LLM test adds the model as an attack surface: direct and indirect prompt injection, insecure handling of model output by downstream code, data leakage through retrieval, excessive agency in agents, and abuse of tool or MCP permissions. Both are needed, so the work is usually scoped together with penetration testing of the surrounding application.
Penetration testing looks for exploitable security flaws in a defined application: what an attacker can read, change or trigger. AI red teaming is broader and more open-ended. It probes model behavior for harmful, biased or policy-violating output as well as security failures. Teams shipping an LLM feature usually start with a penetration test, then add red teaming when the model's behavior is itself a risk.
Yes. The vendor secures the model and its hosting. Your application decides what the model can see and do: the system prompt, the documents it retrieves, the tools it can call and how its output is used. Most findings sit in that layer, and no vendor assessment covers it. Our threat model for LLM applications sets out where those boundaries fall.
Testing is planned to avoid it. Work normally runs against a staging environment with production-like data flows and test accounts. Where production testing is unavoidable, the rules of engagement set limits on volume, on actions an agent may trigger and on data that may be retrieved. Anything that could send messages, move money or change records is tested with those actions stubbed or sandboxed.
Findings are reported against the OWASP Top 10 for LLM Applications, which names the main risk classes, and MITRE ATLAS, which catalogs adversary techniques against AI systems. Governance work aligns with the NIST AI Risk Management Framework and with ISO/IEC 42001, the management system standard for AI. We support readiness against these. We do not certify.
Often alongside
Related services, industries and guides
Industries where this matters most
Guides and products
