It is a bit of a paradox that we are storing more and more information digitally, yet every year more and more of it is becoming harder to access. Data on a variety of tapes and disks that were once common, is now trapped on media due to lack of hardware to read it. Do you have a ZIP drive? Do you have a computer that it will work with? Floppies are problem too. You might think you beat the system just by having a USB floppy drive. While these do exist, they typically won’t read oddball formats. That is, except for Flux Engine, an open source USB floppy drive.
The device uses a $15 Cypress development board and just some wiring (along with a 3.5 or 5.25 floppy drive, of course). Currently, the firmware only supports read only access to IBM standard disks and Acorn DFS/ADFS disks. It can also read and write Brother word processor disks. However, being open source, it could do more. The author, [David Given], is looking for Commodore 1541 and Apple CLV disks to borrow so he can get those working. He’s also offered to entertain other formats if you are willing to loan him a disk.
The software uses libusb and is known to work on Linux and Windows with Cygwin. It should also work with OSX. However, you will need a Windows box of some sort to build the Cypress firmware because the Cypress tools won’t work anywhere else. [David] wants to change processors because of this, but if he does, he’ll miss the PSoC function blocks, we are guessing.
The design is actually rather simple. The firmware only measures the time between flux transitions and sends them to the attached PC. All the heavy lifting occurs on the PC, which means it should be pretty easy to analyze and decode new formats. While writing is possible, it appears there is more work that needs to happen to make it reliable. [David] comments that you really need a real drive to test your writing with so you don’t write things only you can read back. Makes sense.
This certainly is more user-friendly than the last method we looked at. We had to wonder if [David] has thought about 8-inch floppies.
Awesome! I’ve been wanting a KryoFlux for quite some time now. But I don’t NEED one. So I can’t justify the price. This might be the answer for me!
It’s a bit of a paradox alright. How the device that is supposed to save us only offers read only access to IBM formatted disks. Must be a slow news day.
From the second paragraph:
“…supports read only access to IBM standard disks and Acorn DFS/ADFS disks. It can also read and write Brother word processor disks. However, being open source, it could do more. The author, [David Given], is looking for Commodore 1541 and Apple CLV disks to borrow so he can get those working. He’s also offered to entertain other formats if you are willing to loan him a disk.”
This is very interesting… I’ve got a pile of rotting Apple ][ and IIgs disks with the first code I ever wrote on them. It’d be interesting to see how far I’ve come ;-)
Right on! You dont have to take this kind of garbage sitting down! In your armchair.
You should post the link you your floppy drive project, on that hacking blog you write for!
But until you do, feel free to give chase and make love to thine self.
Just yesterday I tossed some 5 1/4 floppies. The data that was on them got moved forward as the years went by so nothing was lost. I still have some floppy drives but none of my computers have a floppy interface. They don’t spark joy so I know what I need to do. I also found a bunch of cassettes in my closet so they also went in the garbage.
Opps, forgot to chuck my Apple I in the trash!
Don’t throw them away! Sell them! They don’t make 5.25″ floppies any more and they’re getting increasingly rare… they go for over a dollar a piece on ebay.
When they hit $5 a piece my time will be worth spending on selling them vs trashing them.
TLDR: hassle factor
When they hit $5 a piece my time will be worth spending on selling them vs trashing them.
TLDR: hassle factor
I still have some 8″ floppies lying around, and a magneto optical drive and a shark drive… (shark beat the zip drive in almost every way… pity it never really took off)
Neat! I’ve made something similar too (with an overclocked PIC and usb2serial dongle), with the option to control the stepper motor using an A4988 step stick so that disks where the donut came loose from the metal hub can be glued back, then read using micro stepping since the donut usually ends up off center. I’ve recovered about 10 disks that way (mostly MSX/PC/Amiga). I just added Kryoflux file format support today (read only). I also added a couple of wires to connect the analog flux reversal signal output (amplified) to an oscilloscope to get a better picture of what the signal looks like. The application can capture this from the ‘scope over an ethernet connection and decode the track automatically. Fun stuff!
Here’s a video I made how to glue the donut back in place:
https://youtu.be/Ou9541pU8dA
Cleaning a 3.5″ floppy using a 3D printed bracket and turning tool (free to download):
https://www.youtube.com/watch?v=I-MKpgO4nOY
And the FloppyControl application in action:
https://youtu.be/8nWMTbBg1HY
Your application looks very interesting. Is there anywhere I can download it and did you make the schematics and firmware of your board available somewhere?
Sure, please sent me an email using the form on http://www.makercentral.net. If there’s enough interest I’ll put it up on github. Thanks for your interest!
Your webserver appears to be misconfigured:
403 Forbidden
Request forbidden by administrative rules.
Strange, it works from where I am. Could you try again? Maybe the hosting company was doing some maintenance.
The server returns a 403 error if the user agent string contains “Firefox/52.0”.
Would it be possible to try a different browser or newer one? Also check if any plugin or firewall or vpn is blocking the request. From what country do you try to access the website? I can check with my vpn from that country. Thanks!
Very cool that it is so inexpensive, but it does have some limitations, which means 90% of the disk formats I care about can’t be read.
It doens’t read the index hole sensor, so it can’t handle hard-sectored disks. My Sol-20, Northstar, and most of my Wang disks can’t be read by it.
The counter is 8b so the longest flux interval it can read is 21 microseconds. Unfortunately, compucolor disks have intervals that are larger than this. (The bit transition time is 13 us, but there are longer gaps between the header and body of a sector, and an even bigger gap between sectors).
I could be wrong about this, but it seems to me that the sector holes are only required for writing, since the sector headers should contain enough information to know what sector you’re reading. Wikipedia isn’t being very helpful here; are there floppy formats that don’t include sector IDs?
I’ve found a reference on the Compucolor physical format, which is… very primitive. It’s literally RS232 spewed onto the disk, with no encoding scheme at all, which means that if you write zeros, you get no flux transitions. (At least there’s a start and stop bit.) http://www.compucolor.org/tech.html Other than that it does contain normal sector IDs.
Jim B is right; without the ability to correctly read long sequences with no flux transitions, FluxEngine can’t decode these. I’ll think about this. The current behaviour is mainly for simplicity, and I’m sure it’s possible to treat timer rollover more intelligently.
Does anyone have a Kryoflux dump of one of these?
That’s the big point of hard sectored floppies. They use a ring of holes to denote the sector boundaries, plus an extra index hole between two of the sector index holes. Thus it’s impossible for sector separation or identification data to become corrupted because there isn’t any.
The software says “Read track 10, sector five.”. The controller steps out to track 10 then waits for the extra blip of the index hole, then counts 10 sector blips, then begins to read data.
Soft sectored disks use the single index hole to keep track of where the first sector of each track begins. From there it’s relied upon for the speed of the drive to be *exact* so the host computer and disk controller can time when to start reading, and to use the directory data to know when to shift tracks and skip sectors to read fragmented files.
A few formats, like Commodore 1541, don’t use the index hole at all. That’s part of why they’re slow. The drive must stream/read the data until it finds the arbritrary position where the first sectors of tracks begin, then it can use the disk speed to pick up the first sector of the requested file. Or does the 1541 have to constantly read everything to feel its way around a disk?
Soft sectoring is not why the 1541 is slow. The Apple II floppy is soft sectored and it’s blazing fast. The 1541 is slow because it’s a mediocre serial design with an actual bug on the motherboard that made it even slower. Fast load cartridges worked around this bug to a large degree, but it was still not hampered by being soft sectored.
The 1541 is not just soft sector, it’s also soft index. Does NOT use the hole in the media. That’s why a 1541 flippy only needs a second write protect notch. For my TI-99/4A I also had to carefully punch two more holes in the disk jackets.
Is the apple ][ also soft index?
You can either have the controller continuous reading the raw data to decode a soft sector one or have it *WAIT* until the drive say it is there. Either case, the mechanical drive still have to spin to the right location which takes same amount of time before you can read the sector.
If you have a large enough buffer for the entire track, you could start buffering the entire track as soon as the head is there. (This might not be the case back then, but is possible these days)
1) I think you’re oversimplifying, here. I know that hard sectors were used to FORMAT floppy discs because this made it easier and more consistent to format the discs, but in all of the examples I’ve seen, the sector headers also contained the sector number, for verification that it was really at the right sector. They ALL had the track number in the header, because you really couldn’t depend on the head stepper not skipping a step now and then, and once you have the track number in the header, it costs very little more to put the sector number there as well. Do you know of a specific format that didn’t do this?
2) The performance issues of the 1541 had nothing to do with the sector format of the discs – its performance was far worse than what difference it would make reading sectors instead of tracks. As I understand it, it had more to do with the ridiculous way the microcontroller in the drive was communicating the system CPU.
Belatedly:
I have added support for long transitions (so the Compucolor should be decodable); also I got two hard-sector floppy Kryoflux images. One of them was an AES Data Superplus / Lanier No Problem word processor, and was absolutely fine as the images contained a traditional record separator and sector header. (Also, MMFM.) http://cowlark.com/fluxengine/disk-aeslanier.html
The other one was a Zilog MCZ 1/20. This one is tougher; there is no record separator, which means that without access to the index hole information, there’s no way to identify where a sector starts. (What you do is wait for the pulse, synchronise the clock against the clock bits, and then when that’s done wait for the first data bit to get the byte alignment.) I can’t distinguish between actual data and a sector header from looking at the data alone.
Which means I’m going to have to rewrite the firmware… again. Oh goody.
Gotta say, great project name.
interesting and neat project. I know during the late 80’s and early 90’s there were many CNC manufacturers that used very proprietary floppy drives for their systems. I know plenty of people who would love to retrieve data to make these machines more functional than they currently are. Looks like a very nice project.
I rely on Kryoflux, which at $125 was not a bad deal at all.
Having alternatives out there is a great thing, so I wish this project good luck.
Reading Apple II floppies is a particular challenge, because the CPU had direct low-level access to the floppy drive motor and head stepper. This was exploited to tremendous effect in copy protection, whereby things like spiral tracks, random entropy, overlapping tracks, and irregular sector counts were used. As a result, all modern floppy imagers fail on most Apple II disks, except one that just came out: The Applesauce. It necessitated creation of a new procedural floppy description format (a static flux image structure definition is not enough for the Apple II). That format is called WOZ and is being supported by emulators now. Google “Applesauce John Morris” to learn more.
Sadly, applesauce does fail with the majority of copy protections too. Even with many that are copyable (along with the protection) on the native apple2 platform itself! The initial (as the author calls it) “cleaning” of the captured data before writing it to a2r file is one of the many problems. The non-existence of a2r “player” or drive emulator is a major problem too that obscures early fundamental problems of this project. But if one needs an expensive USB disk II controller for standard 13 and 16 sector diskettes that works only under OS X that’s the thing one can invest.
Hello, author here… so *this* is where all my traffic has been coming from! (*cough*referrer links please*cough*)
I’m glad there’s interest. Let’s be clear, there’s nothing new here; people have been reading floppies using bodged-together microcontrollers before. I just happened to get lucky with the combination of a suitably powerful microcontroller which needs _no modification_ to do it.
I get away with really simple hardware because the decoder software is clever — it detects the bit rate per track, for example. So it should be able to read those weird Apple II and Macintosh disks just fine (I found a Kryoflux image of a Mac disk and have it about 1/2 done. Also Woz was mad). It’ll read the Brother disks even though the Brother drives don’t even spin at the normal rate.
Regarding hard-sectored disks, provided they have identifiable sector headers they should decode just fine (just a small matter of software). Send me a Kryoflux (or FluxEngine!) dump and I’ll have a look.
Also, I’ve just merged Amiga decode support.
TI-99 support should be easy. There’s a ton of disk images out there, mostly in two formats. One has all the info, one only has the file data. AFAIK copy protection was rare on the TI-99 and never involved any really weird tricks like were done with apple ][.
Unprotected Apple II disks are straightforward. It’s a 13 or 16 sector format with epilogue and prologue nibbles demarcating the sectors. The nibbles are an intermediate encoding format to get around the fact that you can’t have more than two zeros in a row or the auto-gain in the head gets confused.
Once you get to copy protected disks though, the method used by Applesauce and the new .WOZ format is the only viable solution. You can read the disks without it, but the images won’t be bootable, and if written back out, won’t be valid. For example, copy protections relied on unwritten bits being random every time they are read. Any value you write to those bits will fail the protection check because they are now the same each time. Other formats exploited abuse of the auto-gain in the head to create other effects that can’t be reproduced with a real drive. There are protection formats involving physical layouts that the drive itself cannot create, thus making images unreadable. And so on. All of this is why a procedural format is needed if you want the images to be bootable and readable by emulators on the Apple II. It’s a fascinating topic, and I recommend following @a2_4am on Twitter if you’re interested in learning more.
“relied on unwritten bits being random every time they are read”
In other words the protection had to see random garbage from a part of the disk that wasn’t just unformatted, it had to have *never been formatted*.
Regarding the ZIP Drive comment. As of macOS 10.13 (haven’t tested Mojave yet), a USB ZIP Drive “just works”. The biggest caveat is that you can’t Quicklook the files/directories on the drive without hanging Finder.
Sure it does. All I need is a SCSI port … oh, okay then, an ECP-enabled parallel … crap. I guess it’s my own damn fault I never bought a USB one.
Good for me then for still having a working Win9x machine at hand that can do all that voodoo. And has a Voodoo 3, just to nail it. :P ;)
I actually have five ZIP drives — I use one as a hard disk substitute for my IBM PC Convertible. Two more are defunct and the rest are spares for when the one I am using becomes defunct.
We can read something written by a Babylonian on a rook 5,000 years ago but for a disk written 20 year ago we’re out of luck. Great project, nice to see he added Amiga, next C64 and PET.
GCR encoded floppies (1541 and co) are immediately unreadable once you simply stuck them into a pc drive. not sure what a standard drive does to them, but its reproduceable. try yourself……
I know Windows 95 would do *something* to unprotected floppies that made some little alteration. That made it a bit tricky to create PS/2 configuration floppies. One had to run the disk creation program then as soon as it finished, eject the disk. If you looked at the disk contents with Explorer, the PS/2’s configuration system couldn’t read it because some bit inconsequential to any Microsoft OS would get altered, while the PS/2 hardware setup used it.
I dunno if any later Windows version was fixed to not do that.
While we’re talking floppy drives, does anyone know of open-source codebase for emulating a floppy drive on the signal level? Like how those USB-floppy converters must work on the inside?
I want to mount a device (in old synthesizers) and spoof the floppy. But it’d be fun to be able to generate the data on the fly, so that it can be different each time it’s read.
(It’s a 720 kB drive, if that matters, but I’m willing to hack on 1.44 MB code if need be.)
There’s open source firmware for the Gotek floppy emulators: https://github.com/keirf/FlashFloppy Or were you more thinking of the hardware?
Old synthesisers, BTW, use a slightly different pinout and signalling to modern drives — I ran into this a while back when replacing the floppy drive in an old Toshiba T3100SX which used one. Just so you’re aware you might need to tweak the emulation. (I don’t have the details to hand.)
You can also run the HxC floppy emulator software on Gotek hardware: http://torlus.com/floppy/forum/viewtopic.php?f=33&t=1683
This lets you work with a bunch more formats.
ive been having this problem with cd drives. i hadn’t put an optical drive in a computer in at least the last 10 years, trying to move to driveless mini itx builds (m.2 <3). one day i decided that it would be a good idea to rip all my cds (mostly 90s games) to image files, and i had the storage to do it. but every cd/dvd drive i had was dead. different computer, also dead. unless i find a working drive in junk box somewhere, that data is staying on plastic. sometimes you can get an unmodified image file from sometimes dubious sources and use your license key, but id rather do it the right way.
I think Gear Cd Replicator could do a great job at the times with specially made defective bits n the cd ???
WHY CANT I FIND A FLOPPY EMULATOR WITH A FLOPPY DRIVE INTEGRATED? THAT WOULD MAKE TRANSFERRING DATA SO MUCH EASIER SINCE NO COMPUTERS HAVE FLOPPY DRIVES NOW. IT WOULD NEED A SELECTOR SWITCH TO ACTIVATE EACH DRIVE, BUT I COULD SEE THAT SAVING A LOT OF TIME!
Yer caps lock mate, turn it the hell off.