AI Penetration Testing: From Reconnaissance to Exploitation—The 6 Practical Steps

Whereasa functional audit—which we’ve already covered in detail in our dedicated article—is limited to interviews and a document review, an AI penetration test takes action: it actively exploits systems identified as priorities to demonstrate a real impact rather than merely assuming one. The methodology follows a six-step attack chain, modeled after the progression of a real adversary rather than a list of isolated checks.

1. AI-Based Surface Recognition

This step technically confirms or refutes the inventory reported during the functional audit. It scans the internal network for exposed self-hosted model servers, Ollama, vLLM, and LM Studio —each of which has well-known signature ports—fuzzes undocumented AI API endpoints, and technically confirms the presence of Shadow AI by analyzing DNS logs and proxy traffic to known providers. A specific component searches for MCP configuration files on accessible systems, prior to the step dedicated to this protocol.

2. Initial Position: Developer

In a developer role, the key test involves manipulating an AI agent configured in the development environment—using instructions hidden in a code file, a comment, or a README—to attempt to exfiltrate files outside the permitted scope or execute unintended commands, a scenario documented in detail in our article on prompt injection.

3. Initial Access: Standard User Account

From a compromised standard user account, the objective shifts: to retrieve tokens and keys already present on the machine, environment variables, the system credential manager, and currently running processes—and then access the locally stored AI conversation history, which is often rich in sensitive information never intended to be extracted from its original context.

4. Security of the MCP Agent Protocol

This section, which directly builds on our article dedicated to the MCP, covers the authentication of discovered servers, checking for typosquatting in the names of installed MCP packages, and a test of poisoning tool descriptions exposed to the agent—a malicious instruction hidden in a tool’s metadata rather than in processed data.

5. Supply Chain for AI-Generated Code

This step first verifies whether the review of AI-generated code is actually performed by a human by intentionally submitting code containing a subtle vulnerability and observing whether it is detected. It then searches for the “hallucinated” packages suggested by code assistants and checks whether they have been preempted by an attacker on public registries—a risk detailed in our article on slopsquatting, as well as poisoning attempts via the project’s AI context files—CLAUDE.md and .cursorrules—which share the same exploitation logic as the poisoning of MCP tool descriptions seen in the previous step.

6. Operating the Internal AI Infrastructure

In the final step, it actively exploits what the reconnaissance phase uncovered: unauthenticated access to self-hosted models, prompt injection on internal chatbots with system prompt leakage testing, querying a RAG system beyond the document access rights normally granted to the tested user, and testing the circumvention of safeguards using classic jailbreak techniques, role-playing, alternative encoding, and language switching—all of which have already been detailed in our article on LLM jailbreak. This stage also covers resilience against context saturation, which can push system instructions—intended to govern the model’s behavior—outside the active context.

Frequently Asked Questions

Test the actual resilience of your AI systems

Our experts use this six-step methodology to assess the resilience of your AI systems against realistic attack scenarios. Would you like to discuss your project or assess your needs? Contact our experts.