Floppy Drives - Single Density and Double Density, FM, MFM & GCR

In the IBM PC world, there is no such thing as "single density" floppy drives.  Floppy drives started out as double density and later matured into high density and even extra high density.  Those of us who grew up in a PC world had no need to know about "single density".  But where did single density come from and which systems used it?  Moreover, what happens when system builders try to stretch single density reading into double-density capacities?  The truth is a bit more complex than you might think.

Any 5.25" floppy disk drive and any floppy disk can read and write single density or double density, it is the capabilities of the floppy drive controller which determines whether it supports single density, double density or something in-between.

Single Density

The first floppy disk controllers used Frequency Modulation to read and write flux reversals.  This simplest modulation scheme was designed to avoid lengthy intervals without a flux transition.  The signal from/to the disk head was self-clocking, it does not use a separate clock signal to keep the data

With Single Density controller, a single flux reversal took 4us.  In this scheme, every flux reversal (a "1") could be followed by one non-flux reversal of the same 4us period (a "0").  A data 0 bit was encoded as flux 10 and a data 1 bit was encoded as flux 11.  This scheme ensures there will be no strings of zeroes, which may cause the data to lose sync with the clock.  Clock pulses are embedded into the data stream, and because so many clock pulses are added, FM is less efficient than later encoding schemes.

The text HI is encoded as follows in Hex ASCII :

48 49

And in binary data bits :

0100 1000 0100 1001

This sequence is encoded in FM flux as follows :

10111010 11101010 10111010 11101011

It takes 21 flux transitions to store the ASCII representing those letters.

Single density data is written at 125kbits/second.  The data rate is half the clock rate (250KHz) because clock bit always accompanies a data bit.

Single Density in its "native form" was primarily encountered with Atari 8-bit systems, TRS-80 Model 1/3/4, the BBC Micro and the TI-99/4A.

Double Density

Eventually, floppy drive controllers were able to decode a Modified Frequency Modulation scheme.  MFM was more efficient at encoding data because it required fewer "clock-only" flux transitions to be inserted.  In MFM, a data 0 bit was encoded as flux 10 if it followed a data 0 bit and flux 00 if followed by a data 1 bit, and a data 1 bit is always encoded as flux 01.  This scheme ensures that there will be no more than three clock cycles without a flux reversal.  

To encode our previous example of HI in MFM, we either use this sequence (if the previous data bit was a 1) :

10010010 01001010 10010010 01001001

or this sequence (if the previous data bit was a 0) :

00010010 01001010 10010010 01001001

This gives us 12 flux transitions to store these letters. On average, the same data encoded in MFM will have a 50% reduction in the number of flux transitions needed over FM encoding.  The reduction in the number of clock pulses permits an MFM data stream to be read and written twice as fast, using a 2us window.  That is why MFM-encoded disks are referred to double density and FM disks are referred to as single density.

Double density data is typically written at 250kbits/second.  "Single Density" and "Double Density" disks can be used interchangeably.  The clock rate (500KHz) is double the speed of the data rate because every data bit is still encoded as two flux transition bits.

In addition to the IBM PCs and compatibles, the TRS-80 Color Computer, MSX and the Amstrad CPC disk's controllers used MFM.  The Atari ST and Commodore Amiga also use MFM with their double density drives.

One well-known and intentional instance where the MFM rules are violated is with "A1 Sync".  On a disk formatted in an IBM PC or compatible using DOS, before each address and data field there will be three A1 bytes written.  In order to alert the disk controller that there is going to be a new address or data field coming up, the three A1 bytes will be written in a way which does not conform to the MFM rules.

A1 is hex 1010 0001

In MFM it should encode as 01000100 10101001, and if it is an ordinary data byte, it will.

But an A1 Sync byte encodes as follows : 01000100 10001001.  The missing clock bit is in bold.  However, because MFM can accept three 0 fluxes in a row, it will not cause the disk controller to lose sync.  For normal disk usage, these A1 Sync bytes should be the only non-conforming MFM data on the disk that matters.  Weak bit disk protections usually violate the MFM rules by introducing for four or more 0 fluxes in a row, which will induce randomness into the fluxes read and therefore the data decoded.

Group Code Recording - Single Density Data Rates at Double Density Capacities

Apple and Commodore each took a different approach to getting more capacity out of the same disks.  The Apple II's drives use single density speeds and a flux transition measurement time of 4us.  Even though it is not official, a 4us sample time gives a 250kbps clock rate.  In Group Code Recording, each eight periods of a single density pulse is encoded into an intermediate value.  Apple and Commodore use different GCR methods.

Apple's disk non-HD controllers used three GCR schemes, 4-and-4, 5-and-3 and 6-and-2.  The first two methods imposed a FM-like rule that a 1 bit must precede and follow a 0 bit.  Additionally, each 8-byte must start with a 1 bit.

In the 4-and-4 scheme, which permits 4 bits of storage for every 8 bits of flux reversal, you get this :

HI is broken down in to nibbles, 4, 8, 4, 9 and then becomes 10101110 11101010 10101110 11101011, 21 pulses for 16-bits of data.  As 4-and-4 is no better than FM encoding, the data rate is effectively 125kbps.  

As you can see, this scheme is not particularly efficient but it is quick to decode.  It was only used to encode sector address IDs, which are short sections preceding and identifying a data sector on a track.

Here is where the code in "Group Code Recording" comes into play.  The flux reversals are grouped into 8-bits, but as only 4-bits of data is encoded into those 8-bits and those 8-bits must behave according to those rules, only 16 unique byte codes will be seen from this type of GCR encoding : AA, AB, AE, AF, BA, BB, BE, BF, EA, EB, EE, EF, FA, FB, FE, FF.  In other words, you can encode the hex for HI, 48 49, as AE EA AE EB.  

The 5-and-3 scheme, which permits 5 bits of storage for every 8 bits of flux reversal data and improves on things a little :

HI is (almost) now 10111101 10101101 10110101, BD AB BE, 16 pulses for 15-bits of data

The rules have not changed, you can still only have a single 0 on either side of a 1, but with the addition of the extra bit to hold data you can now have 32 unique codes instead of 16 : AB, AD, AE, AF, B5, B6, B7, BA, BB, BD, BE, BF, D6, D7, DA, DB, DD, DE, DF, EA, ED, EE, EF, F5, F6, F7, FA, FB, FD, FE, FF. 

The 5-and-3 scheme was only used for data sectors on the Disk II Controller card with 13-sector firmware and Apple DOS 3.0-3.2.1.  

Now we come to the 6-and-2 scheme, which permits 6 bits of storage for every 8 bits of flux reversal by allowing a maximum of two 0 bits per 1 bit of flux reversal, this is the result :

HI is finally 10110110 10011101, B6 9D, 10 pulses for 12-bits of data.

Because the rule about how many 0s can be together has changed, the 6-and-2 scheme permits 64 unique codes to be used.  

The 6-and-2 scheme was used for data sectors on the Disk II Controller card with 16-sector firmware, Apple DOS 3.3, ProDOS and Apple 3.5" "double density" disk drives and controllers for the Apple II, Apple //gs and early Apple Macintosh.  The older 5-and-3 scheme no longer used after DOS 3.3's release, but the 4-and-4 scheme was still used for sector address ID encoding.  

Had the 4-and-4 scheme been used with data sectors, it would have permitted only 10 sectors with 256 bytes per sector.  The 5-and-3 scheme permitted 13 x 256 byte sectors and the 6-and-2 scheme allowed for 16 x 256 byte sectors.  Apple's original 5.25" disk drives were reliable up to 35 sectors per track.

If you consider 5-and-3 to be 25% and 6-and-2 to be 50% more efficient at storing data than 4-and-4 encoding, then that would give you rough data speeds of 156.25kbps and 187.5kbps, respectively.

Commodore's drives vary speed by tracks, the clock speed is 250kpbs but with the inner tracks use a data rate of 200kbps and the outer tracks use a data rate of 246kbps.  Commodore used a system where the rule was that no more than two 0 bits could be together and allowing for four bits of useful data to be encoded with five bits.

HI is 01110010010111011001 11 pulses for 16-bits of data. 

Commodore used this scheme for all its 8-bit computers, from the PET, the VIC-20, the C64 and C128.

Flux, bitcells and bytes

There are some devices which can capture flux-level transitions such as Kryoflux, Super Card Pro, Applesauce and Greaseweazle.  They tend to generate very large files, they capture every flux transition they can find on every track the drive is told to read.  This includes flux that has no value, such as the flux after the last sector of the track.  There may be noise and jitter in the flux capture that should be normally be excluded from a proper disk image, except when a protection method relies on it.  Multiple reads of a track are also often utilized to reconstruct data or protection schemes that rely on "breaking the encoding rules", which introduces randomness into the bit stream ("weak bit protections").  Files in the .raw format, .scp and .a2r are flux level formats/

Flux level captures can often be 20MiB for a single sided disk.  They are simply not suitable to use in emulators because they are too raw.  Some programs may be able to convert raw flux captures into a bitcell capture, but sometimes human expertise in knowing how the protection works is needed.  Bitcell captures reduce a raw capture into the data an emulator needs to reconstruct a disk, gap bytes, sector address IDs, data sector bytes.  This is where data is stored in the FM, MFM or GCR encodings.  The IBM PC world has several such formats including .86F, .PSI, .MFM, .TC.  Each PC emulator that supports copy protected floppy images has a favorite format.  86Box, PC-PCE, MAME, DOSBox-TC.  prefer .86F, .PSI, .MFM, .TC, respectively.  Other protection-preserving formats include .g64 for the C64, .stx for the Atari ST, .atx for the Atari 8-bit, .woz for the Apple II, .ipf for Commodore Amiga and .dmk for TRS-80.  Not all these formats store information in the bitcell format (.stx, .atx and .dmk are not).

The final level of disk imaging is the data sector dump.  These are files of all the data sectors on a disk and nothing else.  Boot sectors and operating system files are maintained, but when it comes to preserving a disk's protection, they're almost as worthless as the Apple II's COPYA or IBM PC's DISKCOPY commands.  .img/.ima/.imz/.dsk/.dmf are common extensions for PC disk images data sector dumps.  .dsk/.do/.po are used for Apple IIs, .d64 for Commodore 64, .at for Atari 8-bit, .st for Atari ST, .adf for Amiga. Some formats, like .2mg for the Apple II and Apple IIgs, are essentially sector dumps with some extra file system data (like Volume number) not preserved in a dsk image.  There are formats for the PC like .td0 and .imd which add some additional information.