Day 2 of 1060 minShell Walker
Day 2: Metadata and File Structure Examination
Taught by Mr Elite Updated
By the end of today you can
- Extract EXIF, XMP and file header details from an image with ExifTool and xxd
- Tell a file's real type from its extension using magic bytes
- Read camera, software and timestamp fields and rate how much each one proves
- Demonstrate how easily metadata is edited or stripped, and say what that means for your findings
Warm-up
Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.
Why do we hash the original file before doing anything else with it?
The hash records the file's state at the start, so we can later show the original was not altered and that our working copies match it.
If a working copy's hash matches the original, what have you proved and what have you not?
Only that the copy is identical to the original at the time of hashing. It says nothing about whether the content is genuine or was manipulated before hashing.
Someone sends you a photo and says it was taken at a site on Monday morning with a phone. The file is called IMG_4471.jpg, which looks reassuring. You open the metadata and find the software field says an image editor, and the creation time is three weeks earlier. One glance at that data has changed the conversation.
Today you learn to pull that information out properly and, just as important, to hold it loosely. Metadata is a set of claims written inside the file. It is useful, often revealing, and very easy to fake or remove. A good analyst reports both what it says and how much weight it deserves.
What metadata is and where it lives
Metadata is data about the file, stored inside the file. A camera or phone writes it when you take a photo. Editing tools add to it or rewrite it. Nothing checks that it is true, which is the key idea for today.
Think of it like the label on a jar. The label tells you what the person who filled the jar says is inside. It is a good place to start, but you would not stake a report on the label without looking at the jar as well.
- EXIF
- Exchangeable Image File Format. Camera fields such as make, model, exposure, timestamps and sometimes GPS position.
- XMP
- Extensible Metadata Platform. An XML block, often added by editing software, holding things like edit history and tool names.
- Magic bytes
- The first few bytes of a file that identify its true format, whatever the filename says.
- Header
- The structured start of a file that describes how to read the rest.
There are two layers to look at. The first is the descriptive metadata (EXIF and XMP). The second is the file structure itself: the header, the segments of a JPEG, the encoding choices. The second layer is harder to tamper with casually, so I look at both.
A JPEG is built from segments, each starting with a marker. The file opens with the marker FF D8 (start of image). Camera data usually sits in a segment called APP1, which begins FF E1 and contains the EXIF block. Other tools add their own segments. You do not need to memorise the list today. You only need to know that the file has an internal layout, and that the layout can tell a story the tags do not.
Quick check
A file named report.png starts with the bytes FF D8 FF. What is the most likely situation?
FF D8 FF is the JPEG signature, so the extension is wrong or deliberately misleading.
Watch me do it: read one image three ways
I will examine a photo from start to finish. Use any JPEG on your own machine, ideally one from your phone. Everything here runs locally on your own files. I will call the file photo.jpg, so swap in your own name.
file photo.jpg
xxd photo.jpg | head -n 2Here is the kind of output you should expect. Your numbers will differ, but the shape will match. The first line of xxd output has three parts: the offset on the left, the hex bytes in the middle, and the same bytes as text on the right. Do not worry about the text column for now.
photo.jpg: JPEG image data, Exif standard: [TIFF image data, little-endian], baseline, precision 8, 4032x3024
00000000: ffd8 ffe1 1a45 4578 6966 0000 4d4d 002a .....Exif..MM.*exiftool photo.jpg | head -n 40The top of the listing is file facts such as ExifTool Version Number, File Name, File Size and File Type. Camera and time fields come further down. If a field you expect is missing, do not assume anything yet. Scroll the full listing first, because some tools write tags in groups you did not look at.
exiftool -a -G1 -s photo.jpg | grep -iE 'software|date|make|model|history|creator'[IFD0] Make : Apple
[IFD0] Model : iPhone 13
[IFD0] Software : 16.5
[ExifIFD] DateTimeOriginal : 2024:05:06 09:12:44
[ExifIFD] CreateDate : 2024:05:06 09:12:44Now read what you got. Make and Model tell you what device claims to have taken it. Create Date is when the camera says it was taken. File Modification Date is when this file last changed on your disk, which is not the same thing and often reflects when you copied it. Keep those two apart in your notes.
Reading the fields and rating them
Not every field is worth the same. I sort them into rough strength levels, because your Day 1 output is rated signals, not a verdict. A rating is my honest answer to the question: if this field were forged, how much would I lose by believing it?
The thumbnail deserves a closer look. Many cameras store a small preview inside the EXIF block when the photo is taken. Some editors update the main image but leave the old preview untouched. If the two do not match, that is a concrete, visible inconsistency you can show to a reader.
exiftool -ThumbnailImage -b photo.jpg > thumb.jpg
file thumb.jpgIf it worked, file will report a small JPEG image, often 160 pixels wide. If it reports empty, delete thumb.jpg and move on. When you compare, look at framing, objects and colour. A cropped main image next to a full-frame thumbnail is a classic mismatch, and a good example of a moderate lead.
As you work, write each observation in one line with its rating, for example: Software tag names an editor, moderate lead, needs corroboration. That habit makes your final report easy to write and hard to challenge.
Quick check
Create Date says 09:12 on Monday but the filesystem modified time is Thursday. What should you conclude?
Filesystem times change with copying and transfer, so a gap is a prompt to investigate, not a conclusion.
Why metadata is easy to strip or forge
Now the lesson that keeps your reports honest. I will show you how simple it is to change this data, on a throwaway copy. Do this on a copy of your copy, never on evidence.
cp photo.jpg test.jpg
exiftool -Make='Fake Camera Co' -Model='Not A Real Model' test.jpg1 image files updated
$ ls
photo.jpg test.jpg test.jpg_originalexiftool -all= test.jpg
exiftool test.jpg | head -n 20
sha256sum photo.jpg test.jpgAfter stripping, the listing should shrink to file name, size, type, dimensions and a few format details. The picture itself is untouched. That is the point: the pixels carry on looking the same while the claims about them have vanished or been rewritten, and neither change leaves an obvious mark.
So what do metadata findings prove? Present data can support a story, and contradictions can raise suspicion. Neither is proof. Absence is ambiguous, presence can be forged. Write it that way: observed, rated, and what would confirm it.
Confirmation comes from outside the tags. Ask whether the picture content fits the claimed place and time, whether other copies of the image exist, and whether the person who sent it can supply the original file. Metadata gives you questions to ask, and the next days teach you ways to test the pixels as well.
If it did not work
Running ExifTool edits on the working copy or original
Only read commands on evidence. Practise writing on a separate throwaway copy, and re-verify hashes with sha256sum -c if unsure.
Treating missing metadata as proof of faking
Platforms and screenshots strip it routinely. Record it as a neutral observation and look for other signals.
Confusing filesystem dates with embedded dates
Keep them in separate lines in your notes. Embedded Create Date comes from the file's content, filesystem times come from your system.
Trusting the extension
Always run file and look at magic bytes before opening anything.
Recap
- Metadata is a set of claims inside the file, not verified facts
- EXIF holds camera data, XMP often holds editing history, headers show the true file type
- Use file and xxd to check the real format regardless of extension
- Rate each field: camera and timestamps are weak alone, thumbnails and software tags are useful leads
- Metadata is trivial to forge or strip, so absence and presence are both inconclusive
- Edit only throwaway copies and keep the Day 1 hash routine
Cheat sheet
file name- Identify the real file type from magic bytes
xxd name | head -n 2- Show the first bytes in hex
exiftool name- Print all metadata
exiftool -a -G1 -s name- Show duplicate tags with their group and short names
exiftool -ThumbnailImage -b name > thumb.jpg- Extract the embedded thumbnail
exiftool -all= name- Strip metadata (throwaway copies only)
FF D8 FF- JPEG magic bytes
Before the next day
Mission: Three images, one comparison note
Run ExifTool on three images of your choice, one of them a photo you took yourself, and write a short note comparing what each reveals. This builds the habit of separating observations from conclusions, which you will rely on in Day 4's triage case.
About 30 min
You are done when: A note (text or markdown) with the three saved ExifTool outputs and your comparison table with rated observations.
Want more? Forge the timestamps on a throwaway copy with exiftool, then see which other fields or file times still betray the edit.
Quiz
4 questions. Your score is saved with your progress.
1. An image has no EXIF data at all. What is the best way to record this?
Missing metadata has many innocent causes, so it is a note to follow up, not a conclusion.
2. Why is xxd useful alongside ExifTool?
Magic bytes reveal the true format even if the filename and metadata are misleading.
3. You run exiftool -all= on your working copy. What happens to its hash?
Any change to the file's bytes changes the hash, which is why we edit only throwaway copies.
4. Which statement would best belong in a defensible report?
A defensible statement records the observation, rates it, and notes its limits.
Finish the day
Mark the day complete to save your progress.
Next: Day 3 moves from the file's claims to the picture itself, looking for artefacts that suggest an image was AI generated. Today's metadata habits become one signal among several.
Your notes
Private to this device. Write what you want to remember from today.