Data sterilization
From Sy
Security > Computer security > Data > Data security > Data sterilization
- Speed issues
- During my initial testing, it appears that I had HDD access issues which made those early attempts very very slow. After some fiddling, I can confirm that dd and cat work fine now.. I haven't re-tested other methods, but I assume everything would work fine without mouse-hopping and other nonsense.
The secure erasure of data.
Contents |
[edit] Multi-platform notes
[edit] Your used space is insecure
An aside..
Ever wonder why sterilizing software writes up to 35 times on the same file? It's because it's theoretically possible to recover past writes of data underneath current data. I imagine it as deposits of sand and muck which slowly disolves away as stronger writes are deposited on top.
Well, the data which is occupying your hard drive is itself not secured, and represent a thin deposit of sand overtop of earlier sensitive data. So if you are editing a super secret file, and you re-save it.. it may get saved to a new location on your disk and have its old location re-used by some other file. That file could be thought to be partially hiding it.
So even if sterilizing a partition in a paranoid fashion, it's quite possible that legitimate data is acting as a thin veil over previous editions of data which should have been wiped.
What's the solution? I've never heard of a tool to work in this manner. Really what would have to happen is that files are moved to create free space at the beginning of the disk, that free space is sanitized and files are slowly moved into that space and their previous location is sanitized.
That's paranoia though..
[edit] Sterilizing a file
File sterilization is not entirely trustworthy under all conditions. Various levels of caching mechanisms will deposit data in various places. Traditional file sterilization assumes that the file and all its parts will stay in place on the disk. man shred is quite up-front about discussing some of its known issues.
Here are some general examples:
- Low-level disk caching.
- Disk S.M.A.R.T.
- A block detected as bad will have its data migrated transparently and the old unreadable block will be marked as bad. That old block may yet be readable or partially readable through better technologies, and its data remains unsterilized.
- NAS / network drive caching.
- Operating system-level caching.
- filesystems that cache in temporary locations, such as NFS version 3 clients
- RAID mechanisms and filesystems that write redundant data and carry on even if some writes fail.
- One example is that bad block migration will leave deposits of partially-readable unsterilized data.
- Compressed filesystems
- filesystems that make snapshots, such as Network Appliance's NFS server
- Other Wierdness (caching done by VMware for a guest, etc)
In addition, file system backups and remote mirrors may contain copies of the file that cannot be removed, and that will allow a sterilized file to be recovered later.
Most importantly, log-structured or journalling filesystems do all sorts of things and can be suspected. Luckily I use ext3 (under Linux), and mount -t ext2 will work to mount the partition as ext2 (i.e. without journalling). Examples include AIX and Solaris filesystems and JFS, ReiserFS, XFS, Ext3, and probably more.
It is for these reasons that I prefer to encrypt the entirety of a filesystem and not individual files.
[edit] Sterilizing boot disks
You cannot sterilize the entire drive if you're actively using it. So you have to boot off a disk/cd/device to free up that drive and then wipe it.
- Darik's Boot And Nuke (DBAN)
- BCWipePD
- One of many Linux distributions are bootable, then one of the below Linux solutions would work. I recommend PCLinuxOS.
[edit] Linux
It's probably a good idea to do a "sync" before and after using tools like these, just to flush any data out to the disk.
- How do I make a file of an arbitrary size?
- How do I just make a file without making it the slow way? Just magically whip up a file of a certain size?
[edit] Sterilizing a file
- Debian has a program called "wipe".
[edit] shred
shred -n 2 -z -v /path/to/file
- shred comes with PCLinuxOS and probably most other Linux distributions.
- two passes of "random" data and one pass of zeroes:
- This is tested and works.
[1]:
- What this tells shred, is to overwrite the partition 2 times with random data (-n 2) then finish it up by writing over it with zeroes (-z) and show you its progress (-v). Of course, change /dev/hda1 to whatever your partition is. Each pass can take some time, which is why I set it to only do 2 random passes instead of the default 25. You can adjust this number, of course, to your particular level of paranoia and the amount of time you have.
Two passes of dd the first writing a one, the second a zero, followed up by shred -n 1 would perform the DoD method.
[edit] BCWipe
bcwipe -v /path/to/file
- bcwipe is a freely available commercial product.
- Its default setting is to overwrite the file 35 times, which take a very long time. When I tested it, it ground my system to a halt intermittantly. I aborted before finishing.. it was just too outrageous to work on my system while wiping a large file.
[edit] Sterilizing a partition
Don't sterilize a partition which you're actively using. umount it.
- shred doesn't seem to work when pointed at the partition itself. It was tested with shred -n 2 -z -v /dev/<partition> -- that bails out with a complaint about running out of disk space, before completing its first pass.
- mkswap or formatting may be able to do something useful to a partition.. I don't know.
- Various methods for creating an encrypted partition may also be a solution.
[edit] cat
cat /dev/zero > /dev/<device> cat: write error: No space left on device
This takes a very long time (over 2 hours for 10GB), essentially locked up my system (even with nice -n 10 - although I did two partitions at once), and doesn't show a progress meter.
Curiously cat didn't fill my partitions completely. df --block-size=1 reported a capacity of:
- Partition 1: 9844854784 with 9843179520 used
- Partition 2: 9844854784 with 9843236864 used
The partitions were mounted as ext3 for the cat.
or with random information:
cat /dev/random > /dev/<device>
This would take a very very long time. Some replacements for /dev/random are found elsewhere in this document.
[edit] dd
dd if=/dev/zero of=/dev/<device> dd: writing to `/dev/<device>': No space left on device
or with random information:
dd if=/dev/random of=/dev/<device>
or with whatever character you want:
while :; do echo -n -e '\377'; done | dd of=/dev/<device>
That would write 0xFF to the device.
It would be faster than /dev/random
Something faster still might be:
export STRING="" for i in `seq 0 1 511`; do export STRING="$STRING`echo -n -e '\0377'`"; done while :; do echo -n -e "$STRING"; done | dd bs=512 of=/dev/device unset $STRING
That way echo is only run once every 512bytes rather than every byte, which should save a lot of system calls.
Repeat with some different values for overwriting as you see fit.
outputting a file full of 1
yes "*empty"|dd of=/dev/<device> bs=16k
If you need to print something fancy try:
yes `echo -e "\xxx\yyy"`|dd of=/dev/<device>
[edit] Sterilizing free space
Don't sterilize a partition which you're actively using. umount it and then re-mount it. This has been tested on ext3 partitions mounted with "mount -t ext2 /dev/<partition> /mnt/<mountpoint>"
The instructions for Sterilizing a partition (making a file and then wiping it) will work fine. Instead of using /dev/<device> mount the partition and then use /mnt/<mountpoint>/junkfile
[edit] Sterilizing slack space
The method for sterilizing the slack space between the end of real data and tne end of actual blocks of data (i.e. free space in clusters in Fat16) is unknown.
[edit] Sterilizing a drive
If you want to sterilize the drive that you're presently booted into, then you want a Sterilizing boot disk.
Otherwise just Sterilize that file.
i.e. use:
/dev/<device>
- shred has been tested and works well.
Alternately, use dd:
dd if=/dev/zero of=/dev/<device>
or with random data:
dd if=/dev/random of=/dev/<device>
- This only writes a single pass of zeroes to the device. I don't know of a way of getting a progress meter.
- dd comes with PCLinuxOS and probably most other Linux distributions.
- This is tested and works.
[edit] Articles
- Secure Deletion of Data from Magnetic and Solid-State Memory -- Peter Gutmann, Department of Computer Science / University of Auckland
