Integrity

How do you show that the dataset you hand over is the same one you get back months later?

Encryption is not integrity

BitLocker protects against reading along, not against altering. Whoever has the key can change the contents and close the medium up again neatly. What is left is still an encrypted volume, only with different data inside it.

So never write in a report that a medium is “protected against tampering” because it is encrypted. That is a statement that collapses at the first critical reading, and it is not needed either: that is what a checksum is for.

What a checksum does do

A SHA-256 hash is a fixed value of 64 hexadecimal characters that follows from the contents of a file. One message changed, added or removed produces a different value. It then no longer matches the reference value, and that is verifiable by anyone holding the file and the value. You do not have to be there yourself and nobody has to take your word for it.

What a checksum does not do: it says nothing about what should have been there. A file that quietly disappears leaves no deviating hash behind, because there is nothing left to hash. That is why the media should carry an inventory alongside the hash values.

The construction that makes it binding

A CHECKSUMS_SHA256.txt on the medium proves nothing on its own. Whoever can alter the files can alter that text file along with them, and then everything matches again.

The reference values therefore have to sit beyond the reach of anyone who can get at the medium. Put them in the report that you sign digitally. That way every hash value hangs from your signature and from the moment of signing, and what the contents were when you handed them over is fixed.

What sits on every medium

  • the data files themselves;
  • MANIFEST.txt with the contents and the scope of the dataset: what is in it, what deliberately is not, and why;
  • CHECKSUMS_SHA256.txt with one hash value per file.

Include the hash of MANIFEST.txt itself in the report as well. Without it, the description of the dataset is the one part that can be adjusted unnoticed afterwards, and it is precisely the part that holds the scope.

Computing

One file, with the PowerShell that sits on every Windows machine:

Get-FileHash -Algorithm SHA256 -LiteralPath .\file.pst

The whole tree in one go, written out to a CHECKSUMS_SHA256.txt one level above the data folder:

Get-ChildItem -Recurse -File |
    Get-FileHash -Algorithm SHA256 |
    ForEach-Object { '{0} *{1}' -f $_.Hash, (Resolve-Path -LiteralPath $_.Path -Relative) } |
    Set-Content -Encoding utf8 ..\CHECKSUMS_SHA256.txt

For anyone who would rather click there is fHash: free, open source, for Windows and macOS, with an entry in the File Explorer context menu. It computes MD5, SHA-1, SHA-256 and SHA-512. Name that tool in your report, or an equivalent one. A party wanting to check one file has to be able to do so without first learning a command line.

Verifying

Supply the method of checking as well. Run from the data folder:

$issues = 0
Get-Content ..\CHECKSUMS_SHA256.txt -Encoding utf8 | Where-Object { $_ } | ForEach-Object {
    $expected, $path = $_ -split ' \*', 2
    if (-not (Test-Path -LiteralPath $path)) { $issues++; "MISSING: $path"; return }
    if ((Get-FileHash -Algorithm SHA256 -LiteralPath $path).Hash -ne $expected) { $issues++; "MISMATCH: $path" }
}
if ($issues -eq 0) { 'All files match.' } else { "$issues problem(s) found." }

This reports both altered and missing files. What it does not report is a file that has been added and is absent from the list. For that you still need MANIFEST.txt, and the observation that the number of lines in the checksum list matches the number of files on the medium.

Pitfalls

Hash on receipt, not only on delivery. A hash value that comes into being only at the end covers only the end. Computing on receipt fixes the initial state and makes every later operation visible as a deliberate step instead of an unexplained difference.

Put -Encoding utf8 on Set-Content. Windows PowerShell 5.1 writes in the system ANSI code page by default. A file with an accent or a Cyrillic character in its name then lands corrupted in the list and cannot be found again afterwards. In a file with email attachments that happens on the very first attachment.

-Path silently skips files with square brackets in the name. Names out of mailboxes contain them regularly. PowerShell reads the brackets as wildcards, finds nothing, and reports nothing. Use -LiteralPath everywhere, otherwise exactly the files with the most awkward names fall outside the check.

Do not write the checksum list into the folder you are hashing. Otherwise the file takes itself in half written, or is just missing. Write one level up and move it afterwards.

Budget for the throughput of the medium. The hashing itself costs little: 2 GB out of the file cache is a matter of seconds, in the order of tens of GB per minute. The moment you read from an external disk or a network share, that connection decides how long you sit waiting. Verifying a full disk therefore takes about as long as reading it out.

Clearing up afterwards

Delete the local working copy as soon as the media are ready and checked, certainly when the judgment determines who may hold the data in custody. Note in the report that you did it and when.

The signed report with the hash values does survive, and with it, it can still be established years later whether the medium put on the table carries the same contents.