Let’s say you use an SD card-base portable audio recorder for work – doing an interview, perhaps. Things go well until one day, you turn the recorder off before stopping the recording. Without pressing that big red Stop button, the file doesn’t close, and you’re left with a very large 0kB file on the SD card. How do you get it back? There are tools that will do it for you, but they cost money. You can do it yourself with a hex editor, though, and it’s actually pretty easy.
The software required for this feat of data recovery is Roadkil’s Disk Imager to dump all the bits on the SD card to an image file, the free version of ISO Buster to show the block addresses and length of each file, and the hex editor of your choice. The process starts as simply an experiment for hot to create an MP3 file by cutting and pasting bits into a hex editor. A good file was found in the hex editor, copied to a new file, and played. Everything works so far; great.
For the actual data recovery, a spreadsheet was created to make an educated guess as to where the lost file should be. Starting at this address, about 90MB of data was copied into a new hex editor window. This is where the recovery hit a snag. Because the SD card was plugged into a Mac before, a bunch of data was written on the card. This went into the first available place on the disk, which just happened to be the header of the lost MP3 file.
That’s not a problem; there’s already the header from an MP3 file sitting in a hex editor from the first experiment to see if this was possible. By copying a few hundred bytes to the front of the lost file, the file was corrected just enough that an MP3 player could reconstruct the file.
It’s not perfect – the first fifty seconds of the interview was garbled. The rest of the interview was saved, though, and that’s much better than losing the entire thing. Thanks [Lewin] for sending this one in.
https://www.youtube.com/watch?v=IHSCVdF8YYU&feature=youtu.be
It happend to me. A 0kb file, the horror i was recording 2 mono tracks. I found that the datastream switches every 6 sectors. Split the raw image into two files with the offset for the file. After all this i used Audacity to import the RAW wav files.
Exactly why I tell people to stop using their SD card or device if they have ever ‘lost’ a file. The longer you wait, the more likely it will get overwritten.
Good job on the recovery.
He should probably stop using that hardware. What sort of device does not properly close a file when it’s shut down? The off button should be a soft off button, which closes files and stops tasks before powering off.
In the video he explained the usefulness of this method if you were to just take out the battery, not even giving the device a chance to safely shut down. The article doesn’t really mention this however, so I can see why you said that.
It is a very common problem with a lot of current devices.
I would say it is a problem with todays file formats. Tape was superior in this case.
I wonder if setting the write protect switch on the sd card (or the microsd to sd adapter) would have prevented the mac book from destroying the data…
In theory this should work! I wish I had thought of that when my sd card got corrupted the other day.
I still to this day eject floppy disks and flip their write protect tabs as I pull them out. Super old habbit. Thinking about starting to do the same with SD cards…. especially before putting in a random persons Mac!
I don’t think the WP switch on SD cards works the same was as on floppies. I think on SD cards is more of a suggestion and the driver decides whether or not to honour it, where on floppies it actually activates a physical switch that makes it impossible for drivers to write to the floppy. Anybody know for sure?
I think you’re right. http://superuser.com/a/676157
I’m just starting a long weekend, how dare you teach me something!
It’s a good reason to never delete individual files from these kinds of devices. Copy the whole thing off and wipe, that way if bad records ever happen you don’t have fragmented files which make the above process a math nightmare. I’ve recovered a file split into 3 pieces and it took hours manually. Windows typically writes files if it’s never seen the raw drive before. I tend to dd a copy in linux before letting windows touch it.
Oh and if the file format is mp4 then you may need a program to add data records at the END of the file before anything, including vlc, will play it.
to be honest the guy got a bit lucky, the 64 block “padding”, copy pasting a random size section of a good mp3 file just to get a working header, etc.. He also constantly says different numbers than he’s typing.
I realize my reading / typing disconnect – it was 1am and I was super tired reading off a UHD monitor. I quickly recorded this video for some friends who were interested in the outcome. It’s actually set to “unlisted” on YouTube but a mate saw it on my personal f-book and sent it in! (I don’t have an issue with this, it’s just a bit of a sloppy video) I plan to re-record a better version in the future with mutliple devices and compare how they all deal with battery pulls / mem card ejects before stop / etc. And yes – you are very right in point out I got very lucky with the 64 block pad / space / whatever you want to call it. I’m aware that many filesystems are a lot more complex than this. It’s worth a shot sometimes right?!
Whatever gets the job done. MP3 is very tolerant of corrupt data, it may produce weird sounds but continues playing. You could probably prepend an existing header plus any number of additional bytes to the raw disk image, and play back the *entire* disk sequentially. Seeking would likely be impossible, but the uninterrupted sequential playback could be transcoded to a new MP3 file that when complete, would be seekable. How’s that for brute force? :)
Since the beginning of when Macintosh first had PC media support added, the first thing a Mac does to any unprotected “DOS” media is scribble grafitti on it, creating desktop and trash folders and folders and files to stash the resource fork data. Macs do that to HFS and HFS+ media, check and update the desktop, trash etc without asking permission.
DOS and Windows do NOT immediately write anything to a disk or other media that will overwrite deleted file data. The only time I’ve encountered a situation when Windows had to not be allowed to access a floppy in any way was with MCA setup disks.
Windows (vista and above) DOES immediately write to the disk. When you put any removable media it will write a readyboostperftest.tmp file, (huge! several MB, i think like 90MB in case of a 32gb card). I accidentaly erased the sd card on my phone, so took it out, inserted it in a w7 computer to do a clone and recovery, and since it was a bit slow i saw the readyboostperftest.tmp show up on explorer and be immediately erased. No need to say that it messed up my recovery.
This is really good to know about. Sounds like always write-protecting memeory cards by default is the way to go. I wonder if it’s absolute write protection via hardware or protection at a driver level? You’d have to trust that all SD card readers support the write protect switch too….
IIRC this behaviour can be disabled by default
HxD can open raw disks on Windows.
Interesting. I don’t know if I would have thought of that. That there’s a bit of guesswork involved could be a bit off-putting, but you can avoid that with a bit of optimism! It’s certainly better than losing everything right off the bat. Just have to make sure you stop using the device as soon as you realize the data’s gone so you don’t overwrite it on accident. Really clever.
the spotlight stuff is for the search feature used by todays osx
In a previous life, i often had to restore or indeed recreate corrupted database tables and indices “by hand”. Hellish job which involved a fair amount of guestimating and trial & error but i did get quite good at it eventually.
Any of you gents mind saving my broken heart? Do any of you know how hex edit applies with FLP files? I have been working on a track for a few years now, I definitely saved it, then turned off my laptop, although, the following session took my head off my shoulders with a message that said the file is corrupted due to perhaps vst plugin error. Can I apply the previous saved data that is on the system auto save, or do I have to find individual inconsistent bytes? How do I find where the file becomes corrupted in the hex editor. Thanks so much for any comments or suggestions.