Hackmosphere presents its three action plans dedicated to AI: two support offerings to help you master and secure your AI use cases, and a third that provides you with our AI-enhanced expertise. This specific approach is essential, as traditional cybersecurity remains blind to emerging risks: a standard web scan does not detect prompt injection, just as a standard IT inventory overlooks shadow AI.
To address this challenge, our services follow a logical progression: from mapping to risk assessment, all the way to specialized penetration testing. This structured approach allows us to address the two aspects of the threat separately but methodically: the security of the application that incorporates AI and the security of the model itself (its resistance to manipulation and misuse).
This need for appropriate protection becomes critical in the face of increasingly sophisticated autonomous attacks, as illustrated, in July 2026, the incident involving AI agents developed by OpenAI, which escaped their test environment and autonomously exploited a vulnerability to compromise the Hugging Face platform. It is precisely to address these types of automated threats that our human expertise comes into play: only human expertise can interpret the true scope of vulnerabilities and transform this assessment into an operational security plan.
AI Security Audit: Documentation and Configuration Analysis
The AI security audit verifies the system’s compliance with access management, data governance (RAG), and standards (OWASP LLM Top 10, ISO 42001, EU AI Act). It provides evidence of due diligence to regulators and identifies obvious vulnerabilities to optimize the budget before a penetration test.
AI mapping identifies actual usage patterns by cross-referencing stakeholder interviews, network traffic analysis, and a review of SaaS tools. This approach reveals the use of shadow AI and third-party integrations that employees have not disclosed.
The technical registry derived from the mapping exercise classifies each system according to its criticality and data access rights. It provides the essential factual basis for the governance committee to make decisions and guide future security measures.
The risk assessment then prioritizes assets based on their actual exposure, ranging from a simple FAQ chatbot to a RAG agent connected to the CRM. It is based on the EBIOS RMmethod, enhanced by theOWASP LLMand MITRE ATLAS, to integrate directly with a penetration test.
2. AI Penetration Testing: Actively Test the Resilience of Your AI Systems and Applications
AI penetration testing goes beyond a document-based audit: it involves actively attempting to bypass the protections of an AI system to demonstrate a real-world impact, just as a traditional penetration test seeks to exploit a vulnerability rather than simply document it. The scenarios tested typically include direct and indirect prompt injection, jailbreak attempts to bypass the model’s safeguards, extraction of the system prompt,poisoning of a RAG knowledge base, and attempts to exfiltrate data via the model’s outputs.
The methodology increasingly relies on AI-powered tools to accelerate the recognition phase and the generation of payloads, but validating the actual impact and interpreting the results within the client’s business context remain tasks requiring human expertise and cannot be fully automated at this stage of market maturity.
3. UMBRA: our AI-powered detection tool, validated by experts
It is precisely with this in mind that we developed UMBRA, our proprietary AI-powered penetration testing tool equipped with over 100 highly specialized AI agents. UMBRA is deliberately positioned between automated scanning and fully manual testing: it scans the system under test and identifies potential vulnerabilities, and then each reported finding is manually verified by one of our experts before being considered confirmed. AI accelerates detection, but it never replaces human judgment when determining whether a vulnerability is truly exploitable in the specific context of the audited system; this systematic validation step is what sets UMBRA apart from a simple scanning tool.
This approach allows us to conduct more penetration tests at a lower cost to our clients than a fully manual process, without ever compromising the assurance provided by human verification of every result we deliver.
How to Choose an AI Penetration Testing Provider
Between 2025 and 2026, the range of security solutions for AI systems has become significantly more diverse. It now includes automated scanners for LLM models and applications, continuous red teaming platforms, services that combine automation with human expertise, and agent-based solutions capable of orchestrating more complex attack scenarios targeting agents, tools, and AI supply chains.
There are a few criteria that help distinguish a reliable service from a simple, crude scanning tool that generates noise:
- Systematic human validation: Even an automated tool (including one powered by AI) must always be supervised by experts to eliminate false positives and assess the actual impact.
- Adapting to the architecture: the methodology and test agents must adapt to the target (simple chatbot, RAG pipeline, multi-tool autonomous agents) rather than applying a generic list of prompts.
- Compliance requirements: For organizations operating under high levels of regulatory pressure, special attention must be paid to recognized certifications (such as ANSSI’s PASSI certification, which is required for certain OIV scopes).
A truly effective test harnesses the power of automation to cover as much of the attack surface as possible, while basing its analysis on human expertise capable of translating the discovered vulnerabilities into an operational remediation plan.
In what order should these services be utilized?
A coherent sequence, observed in most successful approaches, generally follows this logic:
- Mapping: Compiling an accurate inventory of AI systems is a prerequisite for any serious initiative.
- Risk Assessment: Prioritize systems based on their actual exposure in order to allocate the budget for the following services to the appropriate areas.
- Compliance: Verify the configuration and compliance of priority systems at a lower cost than a full penetration test.
- Penetration testing : For systems deemed critical following the audit, actively test their resilience against real-world attack scenarios.
It is still possible to jump straight into a penetration test without prior mapping or assessment, but this generally spreads the effort too thinly across a poorly prioritized scope—a common pitfall for companies that address AI security as an urgent matter rather than through a structured approach.
Frequently Asked Questions
Which AI systems should we actually include in the scope?
How should we prioritize the systems to be tested?
– process sensitive or personal data;
– are accessible via the Internet;
– influence an important decision;
– have access to the information system or business tools;
– can automatically trigger an action;
– rely on third-party data or components;
– are used in a regulated or critical context.
What are the risks associated with AI agents and the tools they can invoke?
The test must examine:
– the tools available to the model;
– the privileges associated with each tool;
– the separation between read and write operations;
– human validation of sensitive actions;
– validation of transmitted parameters;
– frequency and volume limits;
– the ability to interrupt or revoke an action.
Securing Your Artificial Intelligence Systems
Our experts guide you through every step of this process, starting with an initial assessment using a methodology tailored to your level of maturity and your actual deployed systems. Would you like to discuss your project or assess your needs? Contact our experts.