SE Bug Bounty Arena / Hunting with AI / Lesson 5
Lesson 5 of 6
Keep a hunting notebook the AI can read
AI features do not behave the same way twice. Without notes you will find a bug, fail to repeat it, and have nothing to report. This is the notebook habit that fixes that.
- 6 min read
- Free
In this lesson
- What to write downRead
- Count your attemptsRead
- Let the assistant read it backRead
- 3 prompts to copyCopy
- Three-question check+20 XP
What to write down
For every attempt: the time, the account you used, the exact input, the exact response, and whether it worked. Copy the text. Do not describe it from memory later.
It takes ten seconds per attempt, and it is the difference between "I think it leaked something" and a report with steps.
Count your attempts
A language model may give in on the fourth try and refuse on the fifth. Triagers know this and will ask how reliable your finding is. If your notes say "worked in 4 of 10 attempts, each in a fresh conversation", you have answered before they ask.
Note the conditions too: fresh conversation or not, which account, anything you uploaded first.
Let the assistant read it back
At the end of a session, paste the notebook and ask for three things: what worked, what the working attempts had in common, and the shortest set of steps that reproduces it.
Then run those steps yourself, from a clean start. If they work, you have your steps to reproduce. If they do not, the assistant guessed at the pattern, and you keep narrowing it down.
Prompts to copy
Replace anything in square brackets with your own material.
Time: Account / role: Fresh conversation? (yes/no): What I uploaded or set up first: Exact input: Exact response: Worked? (yes / no / partly): Notes:
Below is my testing notebook. Tell me: which attempts worked, what those attempts have in common that the failed ones lack, and the shortest sequence of steps that should reproduce the result from a fresh start. Mark anything you are unsure of. [paste notebook]
From the notebook, write one sentence for my report stating how reliable the finding is: the number of successful attempts out of the total, and the conditions they were run under. Use only the numbers in the notebook.
Check yourself
Three questions. Sign in to save your progress and earn 20 XP.