Friday, July 1, 2022

Mass Storage and the Apple II - Conquering the Final Frontier

When we think about storage devices and the Apple II, everyone's first thought is usually the same, the 5.25" floppy disk.  Maybe the more historically informed of us may think of cassette tape or 3.5" disks or even the Apple Hard Disk 20SC, but those devices come a distant second to the mainstay of Apple II program and data storage, the 5.25" floppy disk.  When one is introduced to the Apple II, one must at least understand how floppies work on the basic level.  But what can you do when you want to go beyond the standard floppy disk to explore faster, higher capacity storage solutions.  If all software of any note was originally released on copy protected floppy disks, why bother looking for something else?  And if you want to explore, what will you need?  In this blog entry, I will try to answer these questions.

The Drawbacks of the 5.25" Floppy

The 5.25" double density floppy drive is the lowest common denominator of the Apple II world.  Every system, from the Apple II to the Apple //gs and Apple //c+ supports reading, writing and booting 5.25" floppy disks.  Booting an Apple II floppy is extremely simple, insert a disk, turn the machine on or perform a full reset and wait for a program to load.  Some people may need to appreciate that Apple 5.25" drives are single sided and you can only access one side of a disk's data at any time.  Flipping disks was well-known to users of 8-bit machines but unknown to users of more advanced systems.  Users will also come to learn that Apple II floppy disks are not generous with size, allowing only 143KiB of GCR encoded data to be stored on a disk side.  Many programs take up both sides of a disk or use multiple disks.  The 143KiB limit does not change regardless of whether you are using either of the two major disk operating systems for the 8-bit Apple IIs, DOS 3.3 or ProDOS.  

Eventually, all that disk flipping and swapping and waiting for disks to load, which can be a tad long with software using DOS 3.3 or DOS 3.3-like disk operating systems, tends to wear on one.  ProDOS can speed disk access a little, but it can only do so much.  Unlike DOS 3.3, which was designed to operate with exactly one type of disk drive, ProDOS is far more flexible.  ProDOS does not really care if the disk is a floppy disk, a hard disk, a CD-ROM or some other kind of storage device so long as there is a software driver to present blocks of 512 bytes to ProDOS on command.  ProDOS can boot from 5.25" floppy disks on any Apple II (some versions of ProDOS may not boot with some Apple II models) without any hardware beyond the Disk II Drive (or the compatible drives: DuoDisk, Disk //c, UniDisk 5.25 and Apple 5.25 Drive) and the standard Disk II Controller (later slightly modified as the Apple I/O Controller and renamed to the Apple 5.25 Drive Controller).  ProDOS is not directly compatible with DOS 3.3 disks, they have to be converted.  

In addition to ProDOS, one other element was required to obtain seamless support of larger capacity storage devices on the Apple II line, Smartport.  Smartport is a set of firmware routines which allow the system to interface with generic storage devices via ProDOS and its block storage device system.  Smartport was introduced on the Apple //c in its first major firmware update, ROM version 0.  Smartport was not brought to the Apple IIe systems via a firmware update but is present on the Apple //gs and Apple //c+.  Smartport functionality can be added to the Apple IIe and even the Apple II+ via certain add-on cards.

The Official Higher Capacity Solutions

When it comes to booting ProDOS with 3.5" drives, the situation becomes a minefield of compatibility.  There are three 3.5" drives officially compatible with the Apple II line, the UniDisk 3.5, the Apple 3.5 Drive and the Apple FDHD, a.k.a. the Apple SuperDrive.  The UniDisk 3.5 and Apple 3.5 Drives support 400K single sided and 800K double sided GCR disks, while the SuperDrive supports those disks as well as standard IBM MFM formats up to and including 1.44MiB.  

Due to the advancements provided by these drives, none of them were compatible with the standard Disk II Controller.  The UniDisk 3.5 "Liron" Controller Card was intended to be used with the UniDisk 3.5 and Apple II/II+/IIe systems.  The built in disk controllers of the Apple //c, //gs and //c+ had some support for 3.5" drives built in.  The Apple II 3.5 Disk Controller Card was required to use the SuperDrive to its fullest extent.  

Official 3.5" drive support can be summarized as follows :

Apple II+ - UniDisk 3.5 with Liron Controller

Apple IIe - UniDisk 3.5 with Liron Controller, UniDisk 3.5, Apple 3.5 Drive or SuperDrive with Apple II 3.5 Disk Controller Card

Apple //c - UniDisk 3.5 with ROM version 0 or better

Apple //gs - UniDisk 3.5, Apple 3.5 Drive, SuperDrive up to 800KiB, SuperDrive with full 1.44MiB support with Apple II 3.5 Disk Controller Card

Apple //c+ - UniDisk 3.5, Apple 3.5 Drive, SuperDrive up to 800KiB

You can use up two 3.5" drives via any of these methods except for the Apple //c (which has one internal 5.25" drive) and the Apple //c+ (which has one internal 3.5" drive)

The UniDisk 3.5 and the Apple 3.5 Drive have one fundamental difference, the UniDisk 3.5 drive has electronic circuitry designed, in conjunction with the Liron Controller to maintain a proper pace for a 1MHz 6502 or 65C02.  The Apple 3.5 Drive does not have this circuitry and is designed to work in a faster computer such as the 2.8MHz 65816 CPU found in the Apple //gs and the 4MHz 65C02 with cache found in the Apple //c+.  It works with the Apple II+ and IIe with the Apple II 3.5 Disk Controller Card because that card has its own 65C02, ROM and RAM to manage the data transfer speeds to something the 1MHz 6502 system CPU can handle.  

If your Apple //c has ROM Version 255, then you will need to upgrade the ROM on your system's mainboard to add the Smartport functionality to its firmware.  The procedure is about as easy as it gets with a soldering iron and is described here.  The Apple //c cannot handle the Apple 3.5 Drive because its CPU runs at 1MHz and has no room for a coprocessor.  The Apple //c+ runs at 4MHz and therefore can easily handle the faster disk data transfer speeds of 3.5" drives.

Hard disk support was somewhat limited, as most Apple II software could not conceive of using an enormously costly hard drive with their modest capabilities.  The official solution was to install an Apple II SCSI Card or Apple II High Speed SCSI Card in your IIe or //gs.  These SCSI controllers were universal, so they could support any contemporaneous device, including CD-ROM drives (need a rev. C Apple II SCSI Card for the AppleCD SC drive).  The Apple IIe must be an Enhanced model with a 65C02 CPU and 128KiB of RAM.  Apple offered the Hard Disk 20SC, a 20MiB SCSI hard drive.  More modern SCSI devices like CF readers will work with the High Speed SCSI Card if they can communicate over the SCSI-1 protocol.  My friend Cloudschatze uses this card and a more modern CF to SCSI adapter for fast disk access.  A clone of the non-High Speed card exists.

Modern Day Solutions

Both the Liron and the Apple II 3.5 Disk Controller Cards are rather hard to find and expensive these days.  Fortunately you don't have to hunt one down if you are interested in going into higher capacity storage via the 3.5" route.  ReactiveMicro and Big Mess o' Wires both sell clones of the Apple II 3.5 Disk Controller Card.  ReactiveMicro's card is a strict clone, so it cannot be connected to 5.25" drives, but the BMOW clone, called Yellowstone, can support the 5.25" drives.  

There are many modern hard drive interfaces that can plug into an Apple II.  One of the best known is the CFFA3000 expansion card, it accepts CF cards and has been around for a long time.  The CFFA3000 maker used to only make a batch of cards every couple of years but now that ReactiveMicro handles the sales, its availability is more consistent.  The card can emulate a pair of 5.25" floppy drives or provide multiple ProDOS hard drive images.  Disk formats supported are .dsk, .do, .po, .nib and .2mg.  ProDOS 8 supports up to 32MiB ProDOS volumes.  Hard drive support requires an Apple IIe Enhanced or Apple //gs, but it is much faster than any floppy disk drive.  The Compact Flash card must be formatted as FAT16 or FAT32.  At $225 it is very expensive, and while it used to be the only game in town for mass storage, it now has very competitive competition.  

The Micro Drive/Turbo expansion card is a ReactiveMicro product which also provides hard disk image support via Compact Flash cards, but at $75.00 it is much cheaper than the CFFA3000.  It supports up to 32MiB hard drive disk images, and it use Apple II's DMA to improve transfer speeds over the CFFA3000.  There are two firmwares for the card, one that supports all Apple IIs and one that only supports the Apple //gs.  The Apple //gs-specific firmware is much faster than the Apple II universal firmware on the //gs.  You can swap out firmware flash chips for the appropriate system.  Unlike the CFFA3000, this card offers no floppy drive emulation.  It also can support real IDE hard drives in addition to Compact Flash cards.  The Micro Drive/Turbo will work with an Unenhanced Apple IIe or an Apple II+

Although not as fast as the Micro Drive/Turbo or CFFA3000, the Apple II Booti Card is slightly cheaper ($55) than the Micro Drive/Turbo and has the convenience of supporting USB thumb drives.  It supports regular hard disk images up to 32MiB in size but also has support for special 8GiB-sized images on the Apple IIe Enhanced or Apple //gs.  These use ISO or HFS formatted volumes.  The Micro Drive/Turbo also supports HFS images up to 8GiB.  These larger capacities are effectively limited to GS/OS.  

There are other modern mass storage solutions for the Apple II line, but these three are the most widely known and available at this time.  

The Budget Solution

If you have a wDrive and a standard Disk II Controller Card, you can get support for large capacity storage via a software driver loaded by wDrive or by a SoftSP card.  Both load Smartport routines, but they load them at different locations in the memory map.  The wDrive's software-only Smartport support loads these routines into a portion of RAM above 64KiB.  If software uses the upper 64KiB, then the SoftSP driver will conflict with the software and not work.  

The SoftSP card is a small and inexpensive ($19.90) card which can be inserted into any slot of an Apple II, II+ or IIe, but traditionally mass storage boot devices prefer Slot 7 and you should use that slot.  It provides a ROM chip with Smartport firmware in the area of the memory map reserved for the expansion slots like an Apple //c.  This improves compatibility with hard disk images substantially and it does not require 128KiB of RAM to use.  As you are using the comparatively slow Disk II Controller Card, you will not enjoy the faster loading speeds of the Booti, CFFA3000 or Micro Drive/Turbo.  

The wDrive and Floppy Emu can act as a Smartport device without any SoftSP driver or card in an Apple //c or later Apple II.  Note that the disk controller chip, the IWM, in the Apple //c and later consoles are faster than the discrete logic Disk II Controller Cards and you should see a speed improvement when using these floppy simulators as Smartport devices in these systems compared to the Apple IIe and older systems with the Disk II Controller Card.  You can see the first screens of the Total Replay hard drive image, described below, within five seconds of powering on the system.

You may use the SoftSP card with a FloppyEmu but only in conjunction with its 5.25" drive emulation modes.  There is one serious issue with this combination but it can be overcome.  Smartport devices ground Pin 12 while 5.25" Disk Drive Controllers provide +5v on Pin 12.  This can cause an electrical short circuit.  The wDrive does not connect this pin to anything or leaves it connected to +5v, but the Floppy Emu connects this pin to ground when in Smartport mode.  So you must cut a hole in the ribbon cable for pin 12 when using a Floppy Emu with a system with a SoftSP card to avoid damaging your Floppy Emu.  As the Floppy Emu creator explains, the result of connecting pin 12 may not be catastrophic immediately, but the Floppy Emu was not designed to handle this kind of shorting and its defenses will break down over time.  If you do not wish to do this, then you should remove the SoftSP card when using a Floppy Emu in your system.

What can you to do with a Hard Drive in an Apple II?

After you have found a way to introduce mass storage to your humble Apple II, what do you do with it?  Apple II games were designed for floppy disk only, they used custom or copy protected disk operating systems and were simply not meant to be put on a larger disk with other programs.  Some games, mainly educational games from MECC and other companies, did release their most popular games onto 800K disks.  Prince of Persia, The Bard's Tale III and Rampage are three known non-educational games to be  officially released on the 3.5" disk format.  These disks eliminated disk swapping and cut down on loading times, but they were limited to one program per disk.   

With 32MiB hard drive volumes to play with, you can put a lot of Apple II games onto a single hard drive volume.  I do not own an Apple //gs, so I cannot speak as to hard disk volume collections for games for that system but there are several images located here.  This is what the people behind the Total Replay collection have done.  Total Replay is a hard drive image containing 431 games modified to work with a hard disk without exclusive access to the disk drive.  The game list is here.  Total Replay is focused on arcade, action and platform adventure games and contains almost all of the popular Apple II games in those categories.  All games are taken from "clean sources" (no hacker intro screens or credits) and should work without any known bugs.  Reported bugs tend to be resolved very quickly.

When you boot the image, you get an introductory screen.  From this screen you can type in the letters of the name of the game you want to play and the screen will change to show a screenshot of the game whose name meets the letter combination you typed.  So if you want to play Karateka, you start by typing K-A-R.  You can also cycle through the games using the Left and Right cursor keys.  Most games can be reached by the third letter.  Pressing Space allows you to preview the game, pressing Return starts the game.

When you are feeling indecisive or just away from your computer and have not started a game, Total Replay has a screensaver function where it will cycle through screenshots of the games included on the image.  This is extremely helpful in case you are using a display susceptible to burn-in, (CRTs, OLEDs, Plasma).  Total Replay also offers cheats for many games via Ctrl-C and a joystick calibration utility via Ctrl-P.  

Another unique Total Replay feature is that it tailors your game selection to the hardware you have inside your computer.  Total Replay requires 64KiB of RAM to run and if it only detects 64KiB, it will only show you games that will run with 64KiB.  It will run on an Apple II+ or Unenhanced Apple IIe.  If you have 128KiB, it add those games which require 128KiB to the available games.  It will also detect if you have a joystick installed because many games in the archive only work with a joystick.  Mockingboards and their speech chips will also be detected, even though this will only mean that games which use them will be automatically configured to enable them.

Perhaps you are looking for more cerebral entertainment?  Then you may wish to try the Pitch Dark image.  Pitch Dark is a compilation of every Infocom text-based adventure game released for the Apple II.  Every known revision of every game is included and they run in 80-column text mode (there is an option for 40-column text mode, but it is glitchy).  There is a more traditional menu system used to select a game.  This disk image requires a 128K Enhanced Apple IIe or better as it comes, but it can be modified to run on a 128K Unenhanced Apple IIe by opening the image with CiderPress and renaming the file named ONBEYOND.SYSTEM to BASIC.SYSTEM.  You lose the custom menu functionality and the ability to display box graphics, but you can run any revision of any game from the Bitsy Bye menu.

Or maybe you wish to revisit your youth at your school's computer lab, where rows of Apple IIe computers were bought by the school to teach its students about computers.  These schools would typically have educational programs and games to run on these systems.  Most of these software titles, like Number Munchers and The Oregon Trail would come from MECC.  There is an 8-bit MECC Collection featuring over 160 programs on two hard drive disk volumes available to make loading and playing your childhood memories faster and more convenient. 

There are several hacks of individual games, those generally which would take up many disk sides like Maniac Mansion, Neuromancer, Ultima V or Pool of Radiance, to fit into one disk image.  These disk images are either 800KiB or 1.4MiB in size.  Their size notwithstanding, they will work with a normal Disk II Controller, Smartport and wDrive or Floppy Emu.  Most of these are made by qkumba and can be found on the Asimov FTP site under images\games\collections\san_inc_prodos.  Some may require an Enhanced IIe, but most should work with an Unenhanced IIe.  These hacked disk images are ProDOS based and they have viewable file directories now.  

I tried the conversion of Ultima IV to the ProDOS format with my SoftSP card and wDrive via a Disk II Controller Card.  I was most curious to see if there would be any speed improvement because the original game had some annoying instances of disk loading.  Principally, whenever the game would have to load a new screen during the introduction or a new section of the map when walking on the Britannia map, it would pause for 2 seconds and the music hold a note during the load.  Similarly when you wanted to talk to someone in town, the game would pause to load the dialogue for that person.  Now because I was still using the same Disk II Controller regardless of whether I was using a .woz original dump or a hack, my expectations were kept in check.  Fortunately I was pleasantly surprised when the game booted quickly, had no appreciable slowdown of music and essentially instantaneous loading for those things of which I had previously complained.  I found a bug with the dungeons, but it has been fixed so you can enjoy Ultima IV as best as the Apple II can play it.    

1 comment:

  1. Awsome articel, was looking for this, because I am getting SCSI card with Bluescsi for IIe

    ReplyDelete