GPT-4 Attack Techniques — A Security Researcher’s Complete Breakdown

This tutorial deepens the model-specific layer of what we covered in how to test ChatGPT ethically. For the jailbreaking techniques that cut across all major models including GPT-4, the next article — AI jailbreak techniques that still work — covers the cross-model jailbreak landscape. The AI Elite Hub connects this GPT-4 specific content to the broader curriculum.

GPT-4’s Architecture and Why It Matters for Security Testing

GPT-4 is a large language model trained with Reinforcement Learning from Human Feedback (RLHF) to follow instructions and avoid producing harmful content. From a security research perspective, three architectural characteristics shape the attack surface in specific ways.

Safety training as a separate layer: GPT-4’s safety behaviour is primarily trained in rather than hard-coded. This means jailbreaking attacks are fundamentally attacks against trained behaviour rather than code-level constraints — which is why they’re probabilistic rather than deterministic, why they vary between model versions (each fine-tuning cycle changes the safety layer), and why novel framing attacks continue to find success even as specific known attacks are patched through further training.

Transformer context window: GPT-4 processes all content in its context window — system prompt, conversation history, and user input — through the same mechanism. There’s no hard architectural boundary between “trusted” and “untrusted” content from the model’s perspective. This is the fundamental condition that makes prompt injection possible: the model can’t reliably distinguish an instruction in the system prompt from a plausibly-formatted instruction embedded in user content.

Tool use via function calling: GPT-4’s ability to call external tools (web search, code execution, custom APIs) through the function calling interface creates an attack amplification layer. Getting the model to call a function with attacker-controlled parameters produces real-world effects — not just text outputs. This makes function calling exploitation one of the highest-severity attack categories against GPT-4-based applications.

Attack Category 1 — Prompt Injection Specific to GPT-4

GPT-4’s prompt injection attack surface has some specific characteristics that differ from other models. Understanding these characteristics helps target testing more efficiently.

Instruction sensitivity: GPT-4 is highly instruction-following compared to earlier models. This makes it more susceptible to cleanly formatted injection payloads that look like legitimate instructions. Payloads that use authoritative framing — “SYSTEM:”, “INSTRUCTION:”, markdown headers — have higher success rates against GPT-4 than against models that are less instruction-tuned.

Context position sensitivity: Injection payloads at the end of long contexts perform differently from payloads near the beginning. GPT-4’s attention mechanism distributes attention across the full context window, which means a payload buried deep in a long document retrieves less “weight” than the same payload at the end. I systematically test payloads at both positions on any RAG-based application.

Attack Category 2 — System Prompt Behaviour Exploitation

GPT-4’s handling of system prompts has specific exploitable patterns that I’ve confirmed through authorised research. The model treats the system prompt as a trusted source of instructions, but “trusted” in the RLHF sense — the model is trained to follow it, but has no architectural mechanism to prevent user input from influencing how it interprets or follows those instructions.

The most reliable GPT-4 system prompt extraction techniques, in order of success rate across my authorised testing:

Attack Category 3 — GPT-4-turbo vs GPT-4o Attack Surface Differences

GPT-4o (the “omni” model) introduced meaningful changes to the attack surface compared to GPT-4-turbo. Having tested both in authorised contexts, the differences I’ve confirmed are significant enough to adjust testing strategy by version.

GPT-4o safety training: GPT-4o received more extensive safety training than GPT-4-turbo, particularly around certain content categories. In my comparative jailbreaking tests across matched scenarios, GPT-4o consistently showed higher refusal rates on a subset of techniques that worked reliably on GPT-4-turbo. This doesn’t mean GPT-4o is more secure overall — it means the safety layer is different, not absent.

GPT-4o multimodal attack surface: GPT-4o processes images, audio, and text natively. This opens attack surfaces that GPT-4-turbo doesn’t have — vision-based prompt injection being the most significant. I’ve confirmed working image-based injection techniques against GPT-4o that have no equivalent on text-only models.

GPT-4o context window and attention: GPT-4o’s extended context window means injection payloads at different positions behave differently from GPT-4-turbo. In my testing, GPT-4o’s attention is less position-sensitive than GPT-4-turbo — payloads buried in middle-context positions retain more influence relative to the model’s total context. This matters for indirect injection via large documents.

Attack Category 4 — Vision Model Attacks (GPT-4o)

GPT-4o processes images as well as text, and images are an injection surface that most application developers haven’t thought about. The core attack pattern — image-based prompt injection — involves embedding text within an image that contains adversarial instructions. The model reads the text in the image and processes it as content, including any instructions it contains.

I’ve confirmed three distinct image-based attack patterns against GPT-4o in authorised research:

Direct text injection: An image containing printed text that instructs the model to override its system prompt or perform actions beyond its intended scope. The model reads the text and may follow the instructions if they’re formatted authoritatively. This bypasses text-based input filters because the injection arrives as an image, not as text.

QR code injection: Encoding injection payloads in QR codes within images. When the model scans the QR code (if it has web browsing enabled), it may navigate to attacker-controlled URLs. Even without navigation, the model reading the QR code content directly introduces an injection vector through the vision channel.

Adversarial visual context manipulation: Images designed to influence the model’s interpretation of accompanying text, not through explicit instruction injection but through visual context priming. This is a subtler attack that I’ve confirmed in research but that’s harder to reproduce reliably — it’s sensitive to image content, prompt framing, and model version.

Attack Category 5 — Function Calling Exploitation

Function calling is GPT-4’s mechanism for taking real-world actions — calling APIs, executing code, searching the web, managing files. From a security perspective, function calling transforms a language model into an agent that can perform actions on behalf of the user. That transformation also transforms the consequences of successful injection: instead of getting the model to produce inappropriate text, you can get it to call a function with attacker-controlled parameters.

The attack pattern I test systematically on any function-calling-enabled deployment:

GPT-4 vs Claude vs Gemini vs Llama — Attack Surface Comparison

Security research on the four major model families has produced comparative data that’s useful for practitioners who need to scope assessments across different deployments.

GPT-4 vs Claude 3: In my authorised comparative testing, Claude 3 (Opus/Sonnet) shows lower jailbreaking success rates than GPT-4 on a subset of role-play and persona attacks — Anthropic’s Constitutional AI training appears to produce more robust refusal on certain content categories. However, system prompt extraction rates are comparable between the two models. The Claude jailbreaking research covers model-specific techniques in more detail.

GPT-4 vs Gemini 1.5 Pro: Gemini’s expanded context window creates a different injection landscape — larger context means more potential for indirect injection via retrieved content. Gemini’s multimodal capability from the base model (compared to GPT-4o’s vision bolt-on) means the vision attack surface has been present throughout its development. In my testing, Gemini shows comparable injection susceptibility to GPT-4 but different jailbreaking patterns.

GPT-4 vs Llama 3: Llama 3 base models (without additional safety fine-tuning) show significantly higher jailbreaking success rates than GPT-4 or Claude. Meta’s instruction-tuned variants are more resistant but still generally less hardened than the major commercial models. For practitioners testing local deployments, Llama 3 without additional RLHF fine-tuning represents the most accessible attack target for technique development and calibration.