Data Recovery

I can replace the remainder of this page with the following simple suggestion:

  1. split data into multiple files,
  2. backup often,
  3. keep lots of backups (backup files and backup disks),
  4. and never assume your disks are safe.

Experience has taught me that problems happen at the worst possible moment (just after an all–night effort to finish a 100 page document, and 30 minutes before the deadline). I've learnt to assume that if it can go wrong, it will go wrong, and at the worst possible moment. Sounds dramatic, but it's a fairly accurate description of most cases I've seen.

Save Yourself the Trouble

Before I describe how to recover lost data, here are some suggestions on how to prevent data loss in the first place.

The Nature of Data Loss

By data loss, I mean that you had or have some data that is compromised in some way. In this sense, it refers both to data that is not accessible via normal means, or is present but damaged.

Data is valuable but inconspicuous: disks are small, usually hidden, and often cheap, yet may contain thousands of person–hours of work that can't be replaced. On the other hand, it's so easy to store data, trivial data can accumulate quickly. Thus, one has to balance the effort and cost of backing– data against the consequences of losing it. It's up to you to decide how far you need to go in securing your data.

The causes of data loss can be physical or logical. Physical damage often renders the entire device unreadable, meaning the services of a data recover company are required (which are not cheap). Logical errors are usually more manageable.

Note: “Physical device” refers to hard disks, floppy disks, CDROMs, backup tapes, USB memory–sticks, etc..

Physical errors often render the device completely unreadable, so don't keep backups on the same physical device only; by all means keep backups on the same device, but make sure you also have up–to–date backups on different physical devices. Thus, if you leave your disk/tape/USB-stick on the bus, you should still have a backup at home.

The Logic of Data Loss

File System Index versus File Contents

It is often possible to recover data that otherwise seems lost because of the way operating systems and hard disks interact when handling files. Files are managed using a two–tier system much like the card index at a library. In this analogy, think of the library as being the hard disk (rooms might correspond to partitions), the card index would be the operating system's file index (the FAT in DOS and VFAT in Windows), and the actual data (the contents of the files) would be the books. If you want to take this to another level, the bookcases represent clusters, and the shelves represent sectors.

When files are deleted in DOS/Windows, all that happens is that the corresponding entry in the FAT is marked in a special way; the first letter of the filename is replaced withe the code 229 (Hex E5), which is a Greek sigma symbol in DOS's default character set (code page 437, English). The actual contents of the file as stored on the disk is untouched at first. Using our library analogy, this is akin to marking the index card for a given book — the index card, its information about the book, and the corresponding book are still largely intact.

But there's a catch. Once the file has been deleted in this way, the operating system feels free to use the space taken up by the index entry, and the file's contents for other files. Fortunately, if the disk is large enough (most are these days), it will probably be a long time before the operating system overwrites the index entry and the file contents. But don't take this for granted, especially with respect to the file contents. Chance plays a large part in deciding whether the file is overwritten or not.

File Contents Storage

For efficiency reasons, operating systems divide disks up into small (about 8K) sections known as clusters. These are the smallest units of hard disk space that the operating system can manage. When saving a file, the operating system may have to divide the file up and store it across numerous clusters. As files of varying sizes are added, moved and deleted, gaps of different amounts of clusters open up between the remaining files. Individual gaps may be too small for a new file, but several gaps in different locations on the disk can be linked together by the operating system to provide enough space to contain the new file. Thus, this new file is stored across multiple non-neighbouring clusters. This is known as fragmentation. Fragmentation makes recovery difficult, since it is hard to piece together a damaged file when it is scattered in many parts across the disk.

Because of fragmentation, after a file has been deleted, perhaps only a part of the file will be overwritten by some new file. In this case, if the file is in a binary format, you're probably out of luck, depending on the particular format and the program's ability to handle broken files. If, in contrast, the file is in text format, you can piece together the remaining parts into a new file, and re–enter the missing bits yourself.

Data stored in text format can be read by humans, unlike binary format. The problem is that most files are fragmented into an unknown order. Computers are not very good at working out the correct order of a file's fragments, and humans are only any good at it if they can understand the contents of the file.

There are a variety of data recovery tools, requiring various levels of user input. These are some that I have used:

Data Recover Strategies

There are many potential means of recovering data. Which method you use will depend on the cause of the data loss.

Critical Data Recovery

If the damaged disk/files are large, numerous, or very important, it may be wise to leave the recovery process to the experts. There are rofessional recovery services, which are able to deal with very badly damaged data (both physically and logically), and we have used them successfully to recover an entire hard disk. The service I used is OnTrack.

Recovering from Physical Damage

This usually involves floppy disks, as hard disks rarely suffer this problem. USB memory–sticks should be even more reliable since they have no moving parts (they are new on the scene, so time will tell how reliable they are in reality). If you are dealing with a floppy disk, ensure the disk is clean and serviceable before putting it in the drive.

For floppy disks, the first step is to confirm that the problem is caused by disk damage, and not a problem with the drive. You can do this by trying the disk in a different drives, and by making repeated attempts to read the disk or file, perhaps removing and replacing the disk between attempts. If this fails, go on to the next step. For both floppy and hard disks, the next step is to run ScanDisk on the affected disk. You will need to select the option for running a surface test, as this is not performed by deafult.

It may be wise, for critical files, to create a copy of the disk or file, if possible before running any test/repair software on the damaged disk. Scandisk may ask if you want to create an undo file before it makes any changes. I find it is usually pointless, as if the disk is so bad that it needs repair, then you wouldn't want to put it back in that state, unless you want to try different recovery software. If you anticipate using other software, you should consider making copies of the damaged disk, though this will probably be impossible. The best bet is to simply use recovery software you trust, and let it do its work.

Logical Corruption of Files

This can be caused by power-cuts, software bugs or misuse. ScanDisk and DiskDoctor can be used (you don't need to select the surface-check for this test) to recover logically damaged files. It is debateable whether an undo disk is worth creating; Use your instincts.

When Scandisk has completed its repairs, it may have created files with the extender .CHK (._DD for DiskDoctor) on the disk. These files will contain any data (ASCII and binary) that Scandisk found but could not associate with a particular file. They may or may not contain useful data, and you may well have to tidy it up by hand, since it is likely to be fragmented, and to contain garbage data.

Disk and Hex Editors

Even after all the automatic recovery has been completed, there may be missing data. If there is, it may be possible to recover with a disk editor (or a hex editor that supports low level access of disks). These editors can read the contents of a disk, whilst ignoring the file structure. This means that you can find data, even if it is stored in the parts of the disk which the operating systems reports as empty. If the disk has been badly damaged and can't be read at all by the operating system, a disk editor may still be able to read the disk if it is operating in a special mode, usually known as "Physical Disk" mode (as opposed to Logical Disk" mode).

Computer Freezes

If the computer freezes while a user is working on a document, they will obviously be unable to save their document. In some cases, it may be sufficient to be very patient as they computer may start responding even after ten minutes. Some delays can be caused by heavy network traffic, printer problems, and hidden background processes. You can determine what background tasks are running (at least in Windows) by pressing CTRL-ALT-DEL. Try ending any tasks that seems unnecessary, but be careful not to end the task that contains the data in jeopardy!

If all else fails, it may be necessary to give up and hope that the application containing the data has an autosave feature. Word is our most used application, and this does have an autosave option. However, this is not infallible. If you close and restart Word, it should automatically open the document in question. The document's name followed by the word recovered, in brackets, will be displayed in the title bar. If it doesn't automatically recover the file, it may have already been deleted by Word, so you may still be able to restore it by using an undelete utility.

Finding the autosave file is not always easy. It should be saved in the folder specified in the Word options location (Tools | Options | File Locations). If it is not there, it is usually in one of three places. Either the folder in which the document was saved, or the system temporary folder (e.g. c:\windows\temp\ or c:\documents and settings\<username>\local settings\temp\), or the root folder of the hard disk (c:\), in decreasing order of chance. The best way to find the document within these folders is to have the undelete utility to list the deleted files in date order. Look for the file(s) which were deleted around the time that the computer started having a problem, but be careful to allow for the fact that the computer's clock may be wrong.

When undeleting files, the problem of overwriting files is less of a concern, as the files have already been written, so undeleting them does not save any extra data to disk. Having said that, I have noticed that recovering one deleted file sometimes makes another one unrecoverable. It may be that both files were pointing to the same area of the disk, especially if their date-time stamps are very different. If there is a big overlap in the locations of these files, then this may not be a problem, but it is not clear how much of an overlap exists, so be careful. It is probably best to recover the most recent file first.

Accidental Deletion

This is often the easiest problem to resolve. The first step is to ensure that the computer doesn't write to disk after the file has been deleted. This is because the new data being written to disk may be written over the old (wanted) data, because the operating system treats the area used by deleted files as free space. Once you have ensured that the disk will not be written to, run an undelete utility and look for the file in the place it was saved. It will have the same filename, except that the first letter is missing (DOS deletes files by annulling the first letter of a filename).

Accidental Change

Suppose you change a file, then save it, then close the application. Then you decide you want to reverse the changes you made, but after exitting the application, the undo history is lost. In these cases, it is unlikely you will be able to recover your data. The reason for this is that Windows tends to overwrite the original file with the changes. Once overwritten, a file cannot be restored, although I gather that data recovery specialists are able to restore files that have been overwritten multiple times.

You might try seeing if you can retrieve one of the auto–recovery backups (if the application supports that feature) for the changed file. If it has already been deleted, try Unerase. Windows' Recycle Bin won't contain it, but Norton's Protected Recycle Bin might.

Remember!

The most important thing to remember about file recovery (apart from reassuring the user) is to avoid as much writing to disk as possible. If there are other files waiting to be saved, then save them to another disk. If this can't be done, then then you and the user need to decide whether you can risk losing the data on the disk to other processes (e.g. cache-writes), and any data waiting to be saved.


Home About Me
Copyright © Neil Carter

Content last updated: 2008-03-26