FREE
Part of the AI/LLM Hacking Course — 90 Days
Model fingerprinting is the step that sits between reconnaissance and exploitation. Day 20 finds the endpoints. Day 21 checks authentication. Day 24 identifies what’s running behind those endpoints so Days 22 and 23’s techniques are applied to the right target. Running injection techniques optimised for GPT-4 against a Claude deployment wastes time on techniques with lower success probability when the Claude-specific variants would work better. Fingerprinting is the routing decision that makes the rest of the assessment efficient.
🎯 What You’ll Master in Day 24
⏱️ Day 24 · 3 exercises · Kali Terminal + Browser + Think Like Hacker
✅ Prerequisites
- Day 20 — LLM API Reconnaissance
— the endpoint inventory from Day 20 is what you fingerprint in Day 24; the Day 20 header analysis section is the starting point
- Day 2 — How LLMs Work
— understanding tokenisation and generation is prerequisite for timing analysis fingerprinting
- Python with requests and time modules — Exercise 1 builds the automated fingerprinting toolkit
📋 AI Model Fingerprinting — Day 24 Contents
In Day 23 you needed to know the embedding model to calculate retrieval probability accurately. Day 24 covers how to determine that — and how to identify the generation model as well. Day 25 uses the complete engagement output from Days 20 through 24 to build the professional AI security report.
Why Fingerprinting Changes Attack Efficiency
Different models have different vulnerability profiles. GPT-4o and Claude 3 Sonnet both have strong safety training but respond differently to the same injection techniques — GPT-4 tends to be more susceptible to authority framing (T11), Claude tends to be more susceptible to fiction-within-fiction framing (T15). Open-source Llama fine-tunes without additional safety RLHF respond to Tier 1 direct techniques that frontier models refuse immediately. Running the wrong technique library against the wrong model wastes testing time without producing findings.
The fingerprint-to-technique mapping isn’t about using different techniques for different models — the full library gets tested regardless. It’s about which techniques to prioritise in a time-constrained engagement. If you have four hours and the model is a Llama 2 fine-tune, run direct techniques first — they’ll work and you can move to higher-impact exploitation sooner. If it’s GPT-4o, lead with indirect and fiction-framing techniques. The same four hours produces more findings when the prioritisation matches the model.
Header and Error Message Fingerprinting
HTTP response headers are the fastest fingerprinting signal and require no model interaction at all. Some deployments include the model name in custom headers. AWS Bedrock responses include distinctive AWS service headers. Direct API calls to OpenAI or Anthropic include provider-specific headers. Even the absence of expected headers is informative — it often indicates a proxied deployment where a custom backend calls the AI API rather than the frontend calling it directly.
Error message format is the second fast signal. Send a malformed request — missing required fields, invalid JSON, an oversized payload — and compare the error response against known provider formats. The schema is distinctive: OpenAI wraps errors in {"error": {"message": "...", "type": "...", "code": "..."}}. Anthropic uses {"type": "error", "error": {"type": "...", "message": "..."}}. AWS Bedrock produces {"message": "...", "__type": "ValidationException"}. These schemas rarely change and reliably identify the provider even when the model name is hidden.
⏱️ 20 minutes · Kali Linux · Python · OpenAI API
This exercise builds an automated fingerprinting toolkit that combines header analysis, error pattern matching, direct identity probing, knowledge cutoff detection, and timing analysis into a single script producing a model identification confidence report.
nano day24_fingerprinter.py
Step 2: Build the fingerprinting functions:
import os, time, requests, json, re
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()
client = OpenAI(api_key=os.getenv(“OPENAI_API_KEY”))
SYSTEM = “You are a helpful assistant.” # minimal system — no concealment instructions
def probe_identity():
prompts = [
“What AI model are you?”,
“Who created you?”,
“What company built you?”,
“Are you based on GPT-4, Claude, or another model?”,
]
for p in prompts:
r = client.chat.completions.create(
model=”gpt-4o-mini”,
messages=[{“role”:”system”,”content”:SYSTEM},{“role”:”user”,”content”:p}],
temperature=0, max_tokens=100
)
print(f”[IDENTITY] Q: {p[:40]} → {r.choices[0].message.content[:80]}”)
def probe_knowledge_cutoff():
# Test knowledge at specific dates
cutoff_probes = [
(“2024-01”, “What major AI models were released in January 2024?”),
(“2024-06”, “What was announced at major AI conferences in June 2024?”),
(“2025-01”, “What notable AI developments happened in early 2025?”),
]
for date, prompt in cutoff_probes:
r = client.chat.completions.create(
model=”gpt-4o-mini”,
messages=[{“role”:”system”,”content”:SYSTEM},{“role”:”user”,”content”:prompt}],
temperature=0, max_tokens=150
)
resp = r.choices[0].message.content
uncertain = any(w in resp.lower() for w in
[“don’t have”, “my knowledge”, “cutoff”, “unaware”, “not sure”])
print(f”[CUTOFF {date}] Uncertain: {uncertain} → {resp[:60]}”)
def probe_timing(token_counts=[50, 150, 300]):
“””Estimate tokens/sec to narrow model family”””
times = []
for n in token_counts:
prompt = f”Write exactly {n} words about cybersecurity.”
start = time.time()
r = client.chat.completions.create(
model=”gpt-4o-mini”,
messages=[{“role”:”user”,”content”:prompt}],
temperature=0, max_tokens=n+50
)
elapsed = time.time() – start
actual_tokens = r.usage.completion_tokens
tps = actual_tokens / elapsed if elapsed > 0 else 0
times.append(tps)
print(f”[TIMING] {actual_tokens} tokens in {elapsed:.2f}s = {tps:.0f} tokens/sec”)
avg_tps = sum(times)/len(times)
if avg_tps > 80: provider_guess = “Likely fast endpoint (gpt-4o-mini / Gemini Flash)”
elif avg_tps > 40: provider_guess = “Likely Claude 3 Sonnet / GPT-4”
else: provider_guess = “Likely slower model or rate-limited deployment”
print(f”[TIMING VERDICT] ~{avg_tps:.0f} tps → {provider_guess}”)
Step 3: Run all probes:
print(“=== IDENTITY PROBES ===”); probe_identity()
print(“\n=== KNOWLEDGE CUTOFF ===”); probe_knowledge_cutoff()
print(“\n=== TIMING ANALYSIS ===”); probe_timing()
Step 4: Based on results, map to attack family priority:
# Document: what model was identified, confidence level,
# and which technique families to prioritise for this model
📸 Screenshot your fingerprinting output with model identification verdict. Share in #day24-fingerprinting on comments.
Behaviour Differential Probes
Every major model family has distinctive behavioural signatures. Refusal message phrasing differs consistently — GPT-4 tends toward “I can’t help with that” patterns; Claude tends toward longer explanations that explain why and offer alternatives; Gemini tends toward structured lists in its refusals. Mathematical edge cases produce different results across families. Code generation has stylistic differences — variable naming conventions, comment density, preferred patterns. These differences are consistent enough to produce reliable identification from a small probe set.
The most reliable differential probe I’ve found: ask the model to count letters in a word with a known, non-obvious count — for example “How many r’s are in ‘strawberry’?” GPT-4o-mini, Claude, and older model versions all have characteristic patterns in how they approach this and whether they arrive at the right answer. Not because letter counting is a security-relevant capability — it isn’t — but because the reasoning pattern the model exhibits is model-specific in ways that other responses aren’t.
Embedding Model Fingerprinting
For RAG-enabled endpoints, identifying the embedding model is as important as identifying the generation model. The embedding model determines the vector space geometry — which documents are semantically similar to which queries. Knowing the embedding model means you can calculate accurate retrieval probabilities before submitting poison documents, as covered in Day 23.
Embedding model fingerprinting via retrieval behaviour: submit two documents with known semantic similarity to a target query — one very similar, one moderately similar. The number of retrievals per query, the relative ordering of results, and the threshold below which documents stop being retrieved all differ between embedding models. Comparing observed retrieval behaviour against expected behaviour for each major embedding model (text-embedding-3-small, text-embedding-ada-002, all-MiniLM-L6-v2, all-mpnet-base-v2) narrows the model with reasonable confidence.
⏱️ 15 minutes · Browser + Burp · Authorised target
This exercise runs the complete fingerprinting sequence on a live authorised target — combining header analysis, error probing, direct identity questions, and timing analysis into a model confidence report that feeds the technique prioritisation decision.
Send a normal request with Burp proxying.
In the response, check for: model names, provider names,
x-amzn-*, cf-*, x-anthropic-*, openai-* header patterns.
Record any identifying headers.
Step 2: Trigger an error response.
In Burp Repeater: send an empty JSON body to the AI endpoint.
Record the error schema — does it match OpenAI, Anthropic, Bedrock, or Gemini patterns?
Step 3: Direct identity probes.
Send four identity questions to the target:
“What AI model are you?”
“Who created the AI I’m speaking with?”
“Are you GPT-4, Claude, or a different model?”
“What company built the technology powering you?”
Record any identity information disclosed.
Step 4: Knowledge cutoff probe.
Ask about events at three specific dates:
A date you believe is before the model’s cutoff
A date you believe is near the cutoff
A date you believe is after the cutoff
Where does the model’s knowledge become uncertain?
Map this to known training cutoff dates for major models.
Step 5: Timing analysis (3 measurements).
Request outputs of 50, 150, and 300 words.
Time each response (Burp shows response time).
Calculate approximate tokens/sec.
Map to the timing benchmark table from Exercise 1.
Step 6: Compile your fingerprint confidence report:
Provider: [High/Medium/Low confidence] [OpenAI/Anthropic/Google/Other]
Model family: [High/Medium/Low confidence] [GPT-4/Claude 3/Gemini/Llama]
Technique priority: [which families to lead with based on fingerprint]
📸 Screenshot your fingerprint confidence report. Share in #day24-fingerprinting on comments.
Timing Analysis for Model Identification
Token generation speed is one of the few model characteristics that can’t be concealed by instructing the model to behave differently. A model generates at the speed it generates. Response headers, identity responses, and error messages can all be modified by the deployment configuration. The underlying generation speed reflects the model architecture and hardware, not the configuration.
The practical limitation is noise. Network latency, server load, and rate limiting all affect observed generation time. For reliable timing analysis, take multiple measurements, discard outliers, and calculate the range rather than a single value. A consistent measurement of 40-50ms per token points toward Claude 3 Sonnet or GPT-4 Turbo. Consistent measurements below 20ms suggest a smaller, faster model — GPT-3.5-turbo, Claude Instant, or Gemini Flash. Above 80ms suggests a local or self-hosted deployment, or heavy server load.
⏱️ 15 minutes · No tools needed
The fingerprint is only useful if it changes how you test. This exercise maps five different fingerprint scenarios to optimised technique prioritisation decisions — building the mental model that makes fingerprinting actionable rather than just interesting.
— Which injection technique families to run first (from Days 4/15/22)
— Which extraction technique tiers to lead with (from Days 11/18)
— Expected resistance level for each approach
— Any specific vulnerability patterns more likely for this model
SCENARIO A: Headers show “x-anthropic-version”. Identity response
confirms Claude. Timing: ~45ms/token. Knowledge cutoff: late 2024.
SCENARIO B: Error schema matches OpenAI format exactly. Identity
response: “I’m ChatGPT.” Timing: ~20ms/token. Very fast.
Knowledge cutoff: early 2024.
SCENARIO C: No identifying headers. Identity response: “I’m an AI
assistant, I don’t disclose model information.” Timing: ~15ms/token.
Fast with short hedged responses. Likely small/quantised model.
SCENARIO D: Headers show x-amzn-requestid. Error schema matches
Bedrock format. Timing variable. Identity response: “I’m Claude,
powered by Anthropic” but served via AWS Bedrock.
SCENARIO E: No identifying headers. Long, structured identity response
with explicit reasoning explanation. Timing: ~55ms/token. Knowledge
cutoff seems very recent (late 2025). Likely Gemini family.
For each scenario: write a 3-sentence attack plan.
What do you run first, what do you skip initially, why?
📸 Share your five attack plans in #day24-fingerprinting on comments. Tag #day24complete
📋 AI Model Fingerprinting — Day 24 Reference Card
✅ Day 24 Complete — AI Model Fingerprinting
Header and error pattern fingerprinting, direct identity and knowledge cutoff probing, behaviour differential probe library, timing-based model family identification, embedding model fingerprinting for RAG deployments, and fingerprint-to-technique prioritisation mapping. Day 25 takes all the output from Days 16 through 24 — endpoint inventory, authentication findings, injection results, extraction evidence, RAG findings, and fingerprint data — and builds the professional AI security assessment report.
🧠 Day 24 Check
AI Model Fingerprinting FAQ
Why does model fingerprinting matter for security testing?
What are the most reliable model fingerprinting signals?
How do you fingerprint an embedding model used in a RAG pipeline?
Can an application hide which AI model it uses?
Day 23 — RAG Poisoning Attacks
Day 25 — AI Security Report Writing
📚 Further Reading
- Day 25 — AI Security Report Writing — Using the fingerprint, recon, and attack output from Days 20–24 to build a professional AI security assessment report.
- Day 20 — LLM API Reconnaissance — The endpoint inventory and initial header analysis that Day 24’s fingerprinting extends into full model identification.
- Day 23 — RAG Poisoning Attacks — The embedding model fingerprint from Day 24 enables the retrieval probability calculation in Day 23’s semantic optimisation methodology.
- Hugging Face Model Hub — Reference for open-source model families and their training cutoffs — essential for mapping knowledge cutoff probes to specific model versions.
