{"id":5473,"date":"2026-08-28T14:34:51","date_gmt":"2026-08-28T14:34:51","guid":{"rendered":"https:\/\/www.hackmosphere.fr\/system-prompt-leakage-risks-and-security-of-ai-chatbots\/"},"modified":"2026-09-03T12:35:48","modified_gmt":"2026-09-03T12:35:48","slug":"system-prompt-leakage","status":"publish","type":"post","link":"https:\/\/www.hackmosphere.fr\/en\/system-prompt-leakage\/","title":{"rendered":"System Prompt Leakage: Risks and Security of AI Chatbots"},"content":{"rendered":"

[et_pb_section fb_built=”1″ _builder_version=”4.16″ global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_row _builder_version=”4.16″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_column type=”4_4″ _builder_version=”4.16″ custom_padding=”|||” global_colors_info=”{}” custom_padding__hover=”|||” theme_builder_area=”post_content”][et_pb_text _builder_version=”4.27.8″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” hover_enabled=”0″ global_colors_info=”{}” theme_builder_area=”post_content” sticky_enabled=”0″]A developer might be tempted to temporarily embed an API key in the system prompt of a customer service chatbot, with the intention of removing it later. If the model then discloses its instructions in response to a user request, that key may be exposed. This scenario illustrates the risk described by OWASP in the category LLM07:2025 \u2013 System Prompt Leakage<\/strong><\/u><\/a> : a system prompt should never be treated as a secret or used as a security mechanism. API keys, credentials, and connection strings must be kept outside the prompt and protected by access controls independent of the model <\/p>\n

<\/p>\n

<\/p>\n

Prompt Leak, Prompt Injection, and Jailbreak: What Are the Differences?<\/h2>\n

<\/p>\n

System prompt leakage<\/em> is technically related to prompt injection and jailbreaking\u2014which were already discussed in previous articles\u2014but serves a different purpose. Whereas prompt injection hijacks a system\u2019s behavior and jailbreaking bypasses its content safeguards, system prompt leakage specifically aims to make the system reveal the instructions that define its own operation: business logic, authorization rules, integration details, and safeguard configurations. Once these instructions are extracted, an attacker has an exact blueprint of how the system works, which then allows them to craft much more targeted attacks to bypass the defenses they have just uncovered. <\/p>\n

<\/p>\n

<\/p>\n

Prompt Extraction Techniques: How Attackers Trick Large Language Models<\/h2>\n

<\/p>\n

Extraction methods range from the most trivial direct query\u2014such as asking the model to repeat previous sentences or instructions\u2014to obfuscation techniques designed to bypass text filters, such as Base64 or hexadecimal encoding. OWASP classifies encoding as one of the obfuscation techniques<\/u><\/a> used in prompt injection attacks and recommends not limiting defenses to a simple keyword search.<\/p>\n

<\/p>\n

Academic research has also shown that it is possible to automate the generation of extraction queries. The PLeak framework<\/u><\/a> gradually optimizes an adversarial query to reconstruct the system\u2019s prompt, while targeting an application accessible only through its query-response interface. The attacker therefore does not need direct access to the target system\u2019s internal parameters. This approach was evaluated on real-world LLM applications hosted on Poe. <\/p>\n

<\/p>\n

In January 2026, the researchers at Praetorian<\/u><\/a> demonstrated that an LLM assistant could disclose its system prompt through editable fields in the interface, even though its chat responses were limited to predefined messages. In their scenario, the model entered fragments of the prompt into form fields and encoded them in Base64 to make them usable and bypass any filters. This example shows that a restriction limited solely to the text displayed in the chat is not enough: every field, action, log, or output controlled by the model can become a channel for data exfiltration. <\/p>\n

<\/p>\n

<\/p>\n

Examples and Real-World Cases of System Prompt Leaks (Bing Chat, ChatGPT)<\/h2>\n

<\/p>\n

One of the first widely publicized cases was the leak of Bing Chat\u2019s system prompt in February 2023. This Microsoft conversational assistant, integrated into the Bing search engine and based on an OpenAI language model, quickly became the target of attempts to uncover its internal instructions. Through simple injection queries, users obtained initial instructions revealing, among other things, the internal codename \u201cSydney,\u201d behavioral constraints, and operational rules. <\/p>\n

<\/p>\n

The episode demonstrated that a prompt hidden within the model’s context does not, on its own, constitute a reliable privacy boundary.<\/p>\n

<\/p>\n

Since 2025, several public repositories have archived system prompts assigned to major chatbots such as ChatGPT, Claude, Gemini, Grok, and Perplexity. These collections document the existence of extraction techniques and provide access to numerous examples obtained through injection or reverse engineering. <\/p>\n

<\/p>\n

OWASP illustrates the risk using the example of a banking application<\/u><\/a> whose system prompt reveals a specific operational rule: a transaction limit set at $5,000 per day per user, as well as an overall loan limit. Disclosing this information could help an attacker bypass the associated controls, for example, by tailoring their transactions or credit requests to the known thresholds. This is a direct example of how a simple instruction leak can lead to exploitable financial fraud, without any technical vulnerabilities being required elsewhere in the system. <\/p>\n

<\/p>\n

<\/p>\n

Best practices for system prompt security.<\/h2>\n

<\/p>\n

<\/p>\n