Sunday, November 15, 2015

Will the Real Successor to the Pitfall Legacy on the NES Stand Up?

There never was a game called Pitfall III.  David Crane programmed the original Pitfall for the Atari 2600 in 1982 while working for Activision, a company he co-founded.  Two years later he released Pitfall II: Lost Caverns for the Atari 2600.  Thereafter came the great Video Game Crash and Crane left Activision in 1986 to co-found Absolute Entertainment.  While there would be other games in the Pitfall series, this blog entry will discuss three games released or were to be released during the third generation of home video game consoles would could compete for the right to be called the successor to the Pitfall Legacy.

In the Beginning: Pitfall and Pitfall II: Lost Caverns

Pitfall essentially made David Crane as close to a household name as video games got before the Crash and cemented his reputation as a game designer worthy to be inducted into The Academy of Interactive Arts & Sciences' Hall of Fame.  This game was a massive success in its day, selling over four million copies and ported over to all the major consoles and many of the popular home computing platforms of the day.

Starting Screen with Stationary Log
Pitfall is one of the first platforming and side-scrolling games, even though the Atari 2600 hardware did not accommodate scrolling and this game only used static screens.  This game was inspired by Raiders of the Lost Ark and invited players to explore a large, non-randomized world.  In fact, Pitfall is actually far more popular than the official 2600 Raiders adaptation because Crane was able to pare the exploration concept to the basics while still maintaining a sense of fun.  You can just pick up and play Pitfall, Raiders requires reading the manual and a lot of trial and error in order to make any sense out of it.

Swinging over a crocodile lake
There are many dangers in Pitfall : tar pits, lakes (both of which can expand and retract from what appears to be solid ground), crocodiles, scorpions, snakes and campfires that can kill your character, Pitfall Harry.  Logs can steal points if you collide with them, vines can allow you to cross over dangers, you can jump on the crocodile's heads to get over water, ladders and holes allow you to go underground and the underground passages allow you to "warp" a few screens, if there is no brick wall in the way to block you.  Multi-colored "sprites", a relatively detailed background and catchy sound effects and jingles make the game a visual and aural treat for the 2600.

Swinging over a tar pit and rolling logs
Finally, there is plenty of challenge in that you have to locate 20 treasures strewn across 256 screens in 20 minutes with 3 lives.  You will need to map and plan carefully and identify useful warp passages if you want a good chance of finishing the game.  Important to note that unlike most other Atari 2600 games this game will end (it just stops) if you find the twentieth treasure.

Camping out
One other important theme to note here is that Pitfall Harry has no way to kill any of the enemies which he encounters.  In Pitfall, you have to jump over or otherwise avoid all obstacles, including animals.  This aspect will be maintained in the other games Crane designed and discussed in this article.  Ultimately finding non-violent solutions is a recurrent theme throughout almost all the games Crane has designed.

I forgot to bring my tambourine
Two years after Pitfall, Crane released his sequel, Pitfall II: Lost Caverns.  He did not reinvent the wheel or take the series into a radical new direction.  Some sprites and mechanics were taken from the previous game and the two games are visually similar.  However, he expanded the concept by taking the basic exploration theme of the original game (which only used the horizontal plane) and expanded it into the vertical plane.  There are eight horizontal screens and twenty-seven vertical depth levels which you can explore.  In other words instead of having 255 screens stacked together horizontally, you have a grid of 8x~8 screens, adding an extra dimension to the world.

Starting Screen with Checkpoint and Eventual Goal
Now Pitfall Harry can swim in water and has checkpoints to start at if he is "killed". You do not die in this game, you just get sent back to the last checkpoint.  Nor is there a time limit.  There are several new enemies, like electric eels, frogs, vultures, bats, the rat and the return of the scorpions.  The  vultures and bats travel in a repeating pattern across the screen.  In addition, each time you enter the screen they may start from a different position than the previous time you entered the screen.  This makes it hard judge when you can run underneath them, but you have to run past them to make progress in many areas.  At one point you will need to catch a balloon to progress further in the game.  

Bat and Rat
You can find typical gold brick treasures for points, but the object of the game is to find your pet Quickclaw, your girlfriend Rhonda and a diamond ring.  The game ends when you obtain all three with Harry doing some victory jumps before the game stops.  You loose points by colliding with enemies or falling and landing after a certain height.  The screen scrolls vertically as you descend and explore the "lost caverns".  The look of the game is still very much in line with Pitfall, and like Pitfall, the only thing you can do is climb and descend ladders, walk left and right or jump.

Scorpion, Swimming and Treasure
Crane designed a special hardware chip for the cartridge called the DPC.  Because of this chip, Pitfall II was undoubtedly the most complex cartridge ever to be released on the Atari 2600 during its lifetime.  The DPC chip allows bankswitching up to 10KB of ROM (Atari carts are only 2K or 4K without bankswitching; Pitfall is a 4K game), a hardware random number generator, hardware to assist with graphics processing and hardware to generate the data to feed into the internal 2600 audio to create three additional channels of sound.  There was not enough space to put the Adventurer's Edition second quest into the 2600 original that was present in the Atari 5200, 8-bit and Commodore 64 versions of the game.

Watertfall and Electric Eel
Frogs, Ladders and Multiple Levels
Thanks to the DPC chip, Pitfall 2 has music playing throughout.  There are four pieces of music, the treasure theme, the return to checkpoint theme, the default theme and the balloon theme.  This music plays according to what happens during the game.  Multi-channel music as heard in Pitfall 2 is difficult to pull off on the 2600, but when coupled with an advanced game engine like Pitfall 2's, it needed hardware assistance.

Multi-level Madness
Vultures
A Boy and his Blob


Title Screen
Start Screen
The 2600 was too long in the tooth when David Crane was developing A Boy and his Blob, the 7800 and the Sega Master System did not have the market share and the Commodore 64 was dying in the US.  Having left Activision, he had no more right to create a Pitfall sequel than I would.  He ended up creating A Boy and his Blob for the dominant platform of the day, the NES.  A Boy and His Blob to me feels like Pitfall III and in addition to having its creator at the helm, let me explain the design similarities between this game and its Pitfall predecessors.

No money, no healthy food

A subtle way to inject the production team into the game
The Boy in the title is less capable than Pitfall Harry, he can only move/run left and right, he cannot jump unaided.  However, he has a companion, the Blob, which functions as a puzzle solving device.  Using different flavored jelly beans, the Blob can do different things.  Need a ladder, feed him the licorice jelly bean.  Want to drop down to the next level, turn the Blob into a hole with the punch jelly bean (you can do this multiple times for one bean).  While you have 15 flavors to choose from, your supply of jelly beans is limited and you need to use them wisely.  The Blob and jelly bean mechanic represents an evolution over using the character's own abilities to get past obstacles.

Treasure and Subway Serpent
Don't try this in real life
Like Pitfall 2, this game requires exploration of caverns beneath the surface.  Similarly, the game is structured in a grid of non-scrolling horizontal screens.  Unlike Pitfall 2, the screens do not scroll vertically, despite the superior NES hardware.  I do not believe this was done because David Crane could not figure out NES scrolling but because the levels were so vast and less structured than Pitfall 2 that it would have made map making much more difficult.

Cinnamon & Spice
The caverns have many dangers, including subway serpents, falling rocks, spider webs and stalactites and stalagmites in the water.  However, unlike Pitfall 2, the omnipresent danger is death by falling.  If you fall more than one and a half screens (without a device that can ease your fall), you die.  You start off with five lives in this game and no continues.  Also, your Boy moves a lot looser than Pitfall Harry, so you may run into something you cannot avoid and cannot turn back quickly enough to lose a life.  You cannot swim but the Blob can help you with that.  Finally, if you lose a life you will start at the point where you found your last treasure similar to Pitfall 2's checkpoints.

That Blob is very strong
But running under those subway serpents is still tricky
Mind your height
The goal of the first half of the game is to find treasures and escape the caverns.  There are 22 treasures located in different areas of the underground caverns, similar to Pitfall 2.  At the bottom of the caverns you will find an underground lake just like in Pitfall 2.  These treasures allow you to buy vitamins that you will need on the Blob's home planet, Blobolonia.  The second half of the game is a shorter experience where you have to dodge more enemies and defeat the evil, sweets-loving emperor.  Ultimately, gameplay is the key focus of this game.  The graphics are drab and backgrounds are mostly black underneath the subway.  It is quite easy to get lost because screens often look alike.  Music is limited to one basic piece of music for the game (in addition to the title screen music) and a few sound effects and ditties that play at certain times.

You will need to do this very often
Super Pitfall

Title Screen
I am aware that there is a game for the NES called Super Pitfall, but that is an abomination that David Crane had nothing to do with.  It is a port of a Japanese PC-8801 game by Pony Canon where is was called Super Pit Fall. Post-founders-era Activision licensed the title and published it as one of its first NES games. Activision may hold the rights to the Pitfall brand, but that does not mean it earns a place here considering its pedigree.

Spider and Bat Enemies and Water
Super Pitfall is a stripped down port from programmers who did not quite get the NES.  Scrolling is choppy, graphics are barely NES-worthy, the music is a 15-second piece intended to give you an idea of what hell feels like and there is quite a pause as the level loads.  You can now duck and shoot a gun with limited ammo and there are a couple of bosses, which pretty much sums up the evolution of this game over Pitfall 2.  Hit detection is unfair.  Even the manual states that the gun will feel useless at times, which is refreshingly honest. You can see glitches with sprites not infrequently.  The gun mechanic does introduce an element foreign to Crane's Pitfall games.  Also, items are invisible, so you have to jump in certain places to make them appear.  The game is filled with cheap deaths, in fact if you go down the first ladder in the game you will almost certainly die.

Bats and Frogs
Like Pitfall 2, you have to obtain the diamond, the pet and the girlfriend to win the game, but there are more obstacles to overcome. The world is huge (270+ screens) but you only have three lives and no continues, so it will be difficult to get a feel for it.

Waterfalls, Spikes and Treasure
Super Pitfall 2


Title Screen
There was going to be a game called Super Pitfall 2 released for the NES, but was canceled.  There is a prototype ROM floating about.  Super Pitfall 2 was a port of another Japanese game, this one a Famicom game called Atlantis no Nazo (Mystery of Atlantis) by Sunsoft.  The prototype appears to differ from the Japanese original only in its title screen.

1st Zone
Unlike Super Pitfall, Super Pitfall 2 is competently programmed.  Sunsoft was willing to hire competent programmers, Pony not so much.  The graphics are still bland but the music is pretty decent and the control is not quite as frustrating.  Your weapon is a bomb that you can throw, but it detonates when it wants to and can kill you if you are in its (short) blast radius.  The jumping could use more polish (you cannot change your trajectory in mid jump like Castlevania) and the game will get frustrating very quickly.  The bats dropping guano as a weapon is a nice touch.  The shell crabs will hide in their shells if they perceive a bomb coming, so killing them is a bit tricky.  Still, the violent solution is out of place in Crane's Pitfall games.  

2nd Zone
Ultimately, Super Pitfall 2 is really a horizontal side scroller.  The level progression is fairly linear and it does not feel like a Pitfall game at all.  You can find doors that will take you to a small portion of a later zone, but the game is still linear.  Even Super Pitfall was better at conveying an open world.  Pitfall's influence on the game is still present.  The enemies appear to be mostly of the natural variety and there are treasure chests you can open for points.  Between its lack of a real Pitfall feel and its rather long-in-the-tooth status for 1989 when Activision was considering porting it, it is no surprise that it eventually went unreleased.  An optimist may wish to believe that Activision saw that its former founder had released a Pitfall-like game in the same year and did not want to compete with him, but my idea is probably the more likely reason.

3rd Zone

Wednesday, November 11, 2015

IBM PC Floppy Disks - A Deeper Look at Disk Formats and Copy Protection

I.  Overview of Low-Level Floppy Disk Structure

I am going to be using a standard 360KB floppy disk as an example here.  A standard MS-DOS format command formats a 360KB floppy disk with 40 cylinders, 9 sectors per cylinder and 512 bytes per sector.  I distinguish between cylinders, which use both sides of the disk, from tracks, which only use one side of the disk.  Thus you get a total disk size of 368,640 bytes.  However, that is the space available for data bytes and a standard disk dump just dumps the data sectors.  In between the data there is extra data allowing the disk drive to find the right data.

A track's data starts with the Index Pulse which is sent when a disk's index hole passes over the index sensor. Then follow various kinds of fields on the disk which provide information about the disk's position, the location of the data and the data itself. There are three types of Fields found on a disk, the Index Field, the Sector ID Fields and the Sector Data Fields.

A. Gap & Sync Bytes

Gap Bytes precede and follow all three Fields and are first seen immediately following the Index Pulse. These Gap bytes, typically hex 4E, are used to compensate between slight differences between the drive that originally wrote the disk and the drive that is currently writing the disk. They separate the three types of Fields found a disk. Without some tolerance, the Fields may overwrite each other.  

After the Gap bytes come the Sync bytes, which help tell the drive that there will be location information or data to follow. Sync bytes follow a typical pattern of twelve 00s followed by three A1s.  These A1 bytes are missing the normal clocks bits and do not follow proper MFM encoding rules so the drive controller can figure out that this is not real data and the real data is coming up.  The A1s are also called the Sync Mark or just "A1 Sync".

B. The Index Field

There is only one Index Field on a track and it is the first Field which is present after an Index Pulse. The Index Field is not normally used by PC compatible disks and IBM did not even define the area in its manuals. Many disks do not have a distinguishable Index Field but others do. If a disk has no Index Field there will just be GAP bytes until the first Sector ID Field. The Index Field came from Shugart, which invented the floppy disk drive. There are protection schemes which rely on the presence of an Index Field.

The Index Field is a little different from other two Fields. Instead of having Sync bytes of twelve 00s followed by three A1s, the Index Field has twelve 00s followed by three C2 bytes. These C2 bytes also are not valid MFM bytes (see the discussion for A1 Sync below). Following the C2 bytes there is a single byte of FC, called the Index Address Mark (IAM).

C. Sector ID Fields

Each sector on the disk has a Sector ID Field and a corresponding Sector Data Field.

After the A1 Sync Bytes, a Sector ID Field begins with an FE byte as an ID Address Mark (IDAM). Then the Sector ID field continues with four bytes: the track/cylinder number (00-27), the side/head number (00-01), the sector number (01-09) and finally the sector size ID byte (00-06).  Unlike everything else, sector numbers use a convention starting with 01, not 00.  The size ID byte tells the system how large the following Sector Data field is and is interpreted as follows :

Size ID Byte   Actual Size in Bytes
00             128
01             256
02             512
03             1024
04             2048
05             4096
06             8192

The only sector size the standard DOS FORMAT command will use is a 512 byte sector size.  This applies to all standard 5.25" and 3.5" disks and drives, whether double density (160KB, 180KB, 320KB, 360KB & 720KB), high density (1.2MB and 1.44MB) or extra high density (2.88MB).  You can use third party utilities to format disks with other sector sizes, but DOS will not read them.  Note that a track on a 360KB disk cannot really fit 8192 data bytes.  I have never come across a 5.25" double density disk that has had more than 10 x 512 byte sectors.

The next two bytes form a 16-bit Cyclic Redundancy Check (CRC) value. The CRC value verifies that the information contained in the Sector ID field is correct. The bytes calculated include the A1 Sync bytes and the IDAM byte with a Polynomial of 0x1021 and an Initial Value of 0xFFFF. The CRC value is given in big-endian on the disk.

D. Sector Data Fields

As stated above, after the next series of Sync bytes comes a Sector Data Field.  The Sector Data Field begins with an FB byte as a Data Address Mark (DAM).  Then comes the actual data, the bytes you see with a standard disk dump.  Finally comes the 16-bit CRC value, calculated as with the Sector ID Field but with the whole 512 bytes of data.

A Sector ID Field, without the ID Address Mark and CRC Value, is 4 bytes large.  A Sector Data Field without the Data Address Mark and CRC Value, is typically 512 bytes large, otherwise the size is whatever the Size ID Byte indicates. 

For standard DOS compatible 160KB, 180KB, 320KB and 720KB disks, the only thing that changes is the track number, side number and sector number.  160KB and 180KB disks only use 1 side of the disk, 160KB and 320KB disks only use 8 sectors, and 720KB and larger disks use 80 tracks.  High density disks use 15 (1.2MB), 18 (1.44MB) or 36 (2.88MB) sectors per track.  

In a hex viewer, a full dump of the start of a disk's track plus its first sector from the index pulse might look like this :

4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 00 00 00 00 00 
00 00 00 00 00 00 00 C2 C2 C2 FC 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 00 00 00 00 00 00 00 
00 00 00 00 00 A1 A1 A1 FE 01 00 01 02 E1 07 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 00 00 00 00 00 00 00 00 00 00 00 00 A1 A1 
A1 FB 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 
41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 
54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 
41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 
54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 
41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 
54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 71 12 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E

Green = Gap Bytes
Red = Sync Bytes
Blue = Index, ID and Data Address Mark Bytes
Magenta = CRC Bytes
Black = ID and Data Fields

The hex values in the data field repeat the ASCII for USERDATA over 64 times.  

E. Gap Length Conventions

The number of bytes for any given gap, and the value of the gap byte itself, are usually standardized. There are 80 Gap Bytes before the IAM (Post-Index Gap Bytes), 50 Gap Bytes after the IAM, 22 Gap Bytes between an ID Sector Field and a Data Sector Field and 80 Gap Bytes between a preceding Data Sector Field and the following ID Sector Field. There is also a variable number of Gap Bytes following the last Data Sector on the disk. Following the last data sector, there are Gap Bytes until the Index Pulse is sensed (Pre-Index Gap Bytes). These various Gap Byte areas have been given certain identifiers in historical data sheets and manuals:

Pre-Index Gap Bytes/GAP 4B  = variable, normally 218 bytes (4E)

Index Pulse

Post-Index Gap Bytes/GAP 4A = 80 bytes (4E)
Index Field Sync = 12 bytes (00)  + 3 bytes (C2)*
Index Address Mark = 1 byte (FC)*
Post Index Field Gap Byte/GAP 1 = 50 bytes (4E)

ID Field Sync  = 12 bytes (00)  + 3 bytes (A1)
ID Address Mark = 1 byte (FE)
ID Area = 4 bytes
CRC = 2 bytes
Post-ID Field Gap Bytes = GAP 2 (22 x 4E)

Data Field Sync  = 12 bytes (00)  + 3 bytes (A1)
Data Address Mark = 1 byte (FB)
Data Area = 512 bytes
CRC = 2 bytes
Post-Data Field Gap Bytes = GAP 3 (80 x 4E)

* - optional, if not present Gap bytes of 4E are typically present.

This article shows the layout in a visual format with remarkable clarity in Fig 4 with the exception of the position of GAP 4B. The box for GAP 4B should be at the other end of the image, just before the red dot identifying the index hole.

II.  Floppy Disk Encoding

A floppy disk is a magnetic storage mechanism.  Data is encoded through a change in the polarity of a magnetic field from north to south and south to north. The direction of the change, called a flux reversal, does not matter. Only the fact that the polarity has changed is relevant. 

A flux reversal, or a flux pulse, takes a certain amount of time as the drive head passes over the surface of a disk. What distinguishes MFM encoding from its predecessor, FM encoding, is the amount of time required for a flux pulse.

A flux reversal could signify a 0 or a 1 bit. We start with the proposition that a 0 is represented by no flux reversal and a 1 is represented by a flux pulse over a specified length of time.  However, this scheme is not workable because it becomes too difficult to keep track of the bits when there is a prolonged period with no flux reversals.  

Thus the Frequency Modulation (FM) method was first used to encode data onto floppy disks flux reversals.  In this method, a 0 bit is encoded by a flux pulse followed by a no flux pulse , and a 1 bit is encoded by two flux pulses. In order to avoid desynchronization, a clock bit is inserted for every data bit, either 1 or 0. FM uses 4μs for timing to encode a single data bit. In other words, in FM you must have a flux bit every 4μs. There are 50,000 bits of 4μs clock periods on an FM-formatted track and with half of those being clock bits, the maximum data capacity of a track is 3,125 bits.

This method is used in the earliest floppy disk controllers, but for home computers only the Atari 8-bit computer disk drives used it.  These disk drives were called single density.  The IBM PC floppy disk controllers that use an NEC uDP765 or 100% compatible like the Intel 8272 support FM encoding.  Later floppy disk controllers lose that functionality.

This system was easy to implement, but not very efficient. The next improvement was the Modified Frequency Modulation (MFM) method, which is what PC disk drives use.  The fundamental distinction of MFM over FM is that fewer bits are needed to encode the same data bytes as FM. The fewer the flux reversals needed, the more data can be stored on a disk.  The disks that supported this system were called double density.  All IBM PC compatible floppy disk controllers use the MFM encoding scheme, as did the earliest hard disk drives. 

In the MFM encoding method a 0 bit preceded by a 0 bit would be encoded by flux reversal then no flux reversal.  A 0 bit preceded by a 1 bit would be two clocks of no flux reversal.  A 1 bit would be no flux pulse followed by a flux pulse.  No more than three clock periods of no flux reversals are permitted at a time with MFM encoding. On average this only requires half the flux reversals per bit as in the FM scheme.  In MFM encoding, there will never be more than three 0 bits between a 1 bit. The effective timing rate for an MFM flux reversal is 2μs versus 4μs for FM flux reversals. MFM still has a clock bit for every data bit, but because there are fewer clock bits required in MFM, the pulses can be half as long than they are in FM. There are 100,000 bits of 2μs clock periods on an MFM-formatted track and with half of those being clock bits, the maximum data capacity of a track is 6,250 bits. MFM could be said that a flux reversal is required every 8μs.

To summarize how flux bits are encoded into FM and MFM bitcells:

FM  = FLUX

0 = 10
1 = 11

MFM = FLUX

0 preceded by 0 = 10
0 preceded by 1 = 00
1 = 01

Let's try to describe the hex value AA, which is binary 1010 1010. 

 1 0 1 0 1 0 1 0 - Data Bits
1110111011101110 - FM Encoding, 12 bits
0100010001000100 - MFM Encoding, 4 bits

The inversion of AA is 55:

 0 1 0 1 0 1 0 1 - Data Bits
1011101110111011 - FM Encoding, 12 bits
?001000100010001 - MFM Encoding, 4-5 bits

Now consider 00:

 0 0 0 0 0 0 0 0 - Data Bits
1010101010101010 - FM Encoding, 8 bits
?010101010101010 - MFM Encoding, 7-8 bits

And at the other extreme, FF:

 1 1 1 1 1 1 1 1 - Data Bits
1111111111111111 - FM Encoding, 16 bits
0101010101010101 - MFM Encoding, 8 bits

The MFM first bit for hex 55 and 00 is unknown because its value depends on the last data bit of the previous byte.

The data lines of a floppy disk drive do not operate in a vacuum. An FM or MFM bitstream includes clocking bits instead of a separate clock line. Normally the two are in sync, meaning that the clock bits correspond exactly to the MFM data bits.  But there is one time when they intentionally are not by design, the Sync bytes.  How this happens is as follows :

The A1 in the Sync bytes does not follow the appropriate clock and data bit pattern. A1 should be encoded in MFM as:

 1 0 1 0 0 0 0 1 - Data Bits
0100010010101001 - MFM Encoding
0100010010001001 - MFM A1 Sync Encoding

A1 is encoded the proper way, as shown on the second line anywhere else an A1 byte might be stored on the track, such as in a Sector Data Field.  However, to put the A1 out of sync, a clocking bit is skipped, as underlined as shown on the third line above. This intentional error notifies the floppy disk controller that a Sync Mark has been identified a Sector ID or a Sector Data field is about to be read. And for purposes of comprehensiveness, C2 Sync in the Index Field looks like this:

 1 1 0 0 0 0 1 0 - Data Bits
0101001010100100 - MFM Encoding
0101001000100100 - MFM C2 Sync Encoding

III.  Copy Protection

Copy protection methods on the IBM PC platform ultimately generally work within this scheme.  Other systems like the Apple II, Commodore 64, Atari ST and Commodore Amiga have controller hardware that allows far more sophisticated protection methods to be used.  The PC disk controller will not tolerate the more bizarre methods like half tracks, fat tracks, spiral streams, offset tracks or varying bitrate speeds.  

The simplest methods are to use non-standard sector sizes, sector numbers or tracks.  This is good enough to bypass DISKCOPY and FORMAT, but any utility that uses the PC BIOS can format a disk with a nonstandard number of tracks, sector sizes or sector numbers.  Each track and sector within the track can have differing sector sizes.  

Another simple method is to use the write protection tab/notch on a disk.  If a program expects the disk to be write protected, it may try to write a destructive sequence of bytes to the disk.  Of course, the original disk should have no tab or notch.  Be wary if you see a disk notched roughly via scissors or with a hole punch.

A common method is to intentionally write a disk with erroneous CRC bytes.  A standard IBM PC Diskette Adapter can read, but not write erroneous CRCs.  Of course, sectors with data that cannot be reliably read will also return CRC errors.  It is then up to the person de-protecting the diskette to figure out whether the error is intentional or unintentional.  

Another method is to check that the exact number of Gap and Sync Bytes are on the disk.  If the original disk was written with 16 of each and the copy only has 15, that is a way to detect the copy.  The program could instead read data from the Gap bytes, like an encryption key.  The PC disk controller's read track command would read the Gap bytes.

A method used the Sync bytes to let sectors overlap.  The disk can play with the sector ID sizes to make no sense physically, like having sector 1 with an ID of 8192 bytes and then fitting the other sectors "within" sector 1.  This protection can often be fooled by writing the image to a high density disk, which allows for 15 or 18 sectors per track.  Of course, high density disk controllers were far from ubiquitous in the 1980s when most of these games were released.

A devious method used weak bits.  In this case, the game would read an area on the disk insufficient flux transitions have been written, making the read coils increase the gain to try to find transitions. If there were sufficiently lengthy periods without a flux transition (greater than three 0 flux bits) the read head would send back random data on different track rotations as the head tried to guess where one bitcell began and the last one ended. The game would not expect the same result to be returned from multiple reads of a sector.  If the same results were always read back, the game would fail to work, thinking it was being run on a copy. Unfortunately, if the disk was damaged unintentionally or the flux transitions have lost their force with age, a flux copier like an Option Board, Catweasel, Kryoflux, Super Card Pro  or a Greaseweazle may misinterpret this and make a bad copy.

IV.  Known Protection Schemes used in Games :

Cops Copylock II (HD Media Supported)

Legend of Mystic Zone, The
Rising Dynasty, The

Electronic Arts IBM Interlock

Used by Electronic Arts booters, involves 96 interleaved sector IDs, typically at track 15, side 0.  They also use 200KB and 400KB disk sizes with 10 sectors per track.

Amnesia
Archon: The Light and the Dark
Chuck Yeager's Advanced Flight Trainer
Chuck Yeager's Advanced Flight Simulator
Hard Hat Mack
Julius Erving vs. Larry Bird Go One-on-One
Marble Madness
Murder on the Zinderneuf
Pinball Construction Set
Seven Cities of Gold, The
Super Boulder Dash
Timothy Leary's Mind Mirror
Will Harvey's Music Construction Set
World Tour Golf

Formaster Copylock

Contains a sector with a Data Field CRC error, usually at at track 20, side 0, sector 5 or track 6, side 0, sector 1. The sector ID is marked as having 256 bytes, but actually has 512 bytes, fooling copiers like TeleDisk. Used mostly by Sierra. This protection will probably fail at speeds greater than an AT (8MHz 80286)

B.C.'s Quest for Tires (Sierra release)
Crossfire (IBM PCjr floppy disk versions)
Gato (versions 1.0-1.4)
Great Escape, The (Thunder Mountain)
Hardball (Disk 2/EGA Disk, & Tandy Release)
King's Quest (IBM PC-CGA booter rev0, rev1)
Mr. Cool
Pinball Wizard
Oil's Well
Quink (Thunder Mountain)
Sierra Championship Boxing (1984 version)
Top Gun (Thunder Mountain rev0)
Troll's Tale
Ultima II (rev0 & PCjr. version)
Ultima III (rev0, rev1, rev2)
Wizard and the Princess, The (IBM PCjr exclusive version)

H.L.S. Duplication

Accolade and Epyx used this type of protection.  There is an additional sector ID field in one of the sectors.

Ancient Land of Ys
Apollo 18
Ajax
Bad Dudes
Boot Camp
Batman - The Caped Crusader
California Games
Crossword Magic 
Destroyer 
Final Assault 
Gauntlet (rev1
Grand Prix Circuit (early version only)
Hardball (Disk 1 only)
Jr. Pac-Man 
Karateka 
L.A. Crackdown 
Mean 18
Mini-Putt
Rack'em
Rush 'n' Attack
RoboCop (Data East)
Serve and Volley
Star Wars (Broderbund)
Street Sports Basketball 
Sub Battle Simulator 
Super Pac-Man (rev0) 
Test Drive
Test Drive II (including Expansion Disks
TKO

Microprose Protection 1

Protection is found on track 4, side 0.  Uses overlapping sectors.

Gunship (certain versions)
Sid Meier's Pirates

Microprose Protection 2

Protection is found on tracks 38-39, side 0.  Uses overlapping sectors.

Airborne Ranger
The Amazing Spider-Man and Captain America in Dr. Doom's Revenge!
F-15 Strike Eagle II
F-19 Stealth Fighter
M1 Tank Platoon
Megatraveller
Microprose Pro Soccer
Sid Meier's Pirates (432.02 and above)
Red Storm Rising
Rick Dangerous
Savage
Stunt Track Racer
Sword of the Samurai
X-Men - Madness in Murderworld

Mindscape DEM

This protection uses weak bits and will probably fail at 386 speeds (16MHz and above)

Blockbuster!
Captain Blood
Contra 
Defender of the Crown (rev0)
Forgotten Worlds
Gauntlet (rev0) 
Infiltrator 2
Italy 1990
Jackal 
Moonwalker
Outrun
Paperboy (Elite)
Paperboy (Mindscape)
Road Runner
Shadowgate
Space Harrier
Street Fighter
Strider
Super Pac-Man (rev1) 
Top Gun (Thunder Mountain rev1)

Minder03
Used by Prism Software

Pipe Dream (European version, U.S. version by LucasFilm Games used a Codewheel)

On-Line Systems Protection #1

This protection will fail in a PCjr. or DMA-less Tandy 1000 and does not work with the 2nd and 3rd IBM PC/XT BIOS.

Crossfire (PC) 
Frogger
Ulysses and the Golden Fleece

Origin Systems OSI-1

Used by all Origin Systems games with an EXE executable.

2400 A.D. 
Bionic Commando
Ultima I (rev1)
Ultima II (Origin Ultima Trilogy Release)
Ultima IV
Ultima V

Atarisoft

Battlezone 
Defender 
Ms. Pac-Man (Atarisoft &  Thunder Mountain)
Stargate 

Datasoft

Uses weak bit protection

Bruce Lee
Dallas Quest, The
Dig Dug (Datasoft) 
Juno First 
Lost Tomb 

Rob Northern Copylock

Uses "strong bits", a long string of 1bits, and calculates the entropy expected.  Encrypts the actual executable and stores a 32-bit decryption number on a bad sector.

Back to the Future Part III
Blade Warrior
Blasteroids  
Blood Money
Borodino
Carrier Command 
Castle Master
Cisco Heat
Frank Bruno's Boxing (Finale)
Kick Off 2
Klax  
Lemmings (1991-07-19)
Oh No! More Lemmings (comes in disk-based and document-based versions)
Lemmings 2: The Tribes (uses High Density disks)
Midwinter
Mighty Bomb Jack
Nebulus
Onslaught
Paperboy (Finale)
Pit Fighter (Domark)
Predator 2
Pro Tennis Tour  
Resolution 101
The Sentinel 
Stormlord
Thunderstrike  
Toobin'  
Weird Dreams  
Xenomorph
Xenon II

Sector ID Duplication

Sierra used this for the versions of their games distributed under Tandy's label prior to 1985.  In this scheme, there are multiple sectors with identical Sector IDs, but different data.  Subsequent read sector commands will read the later sectors.

BC's Quest for Tires (Tandy release)
King's Quest (Tandy version & PCjr./Tandy version)
Wizard of Ids WizType (Tandy release)

Softguard Superlok v3.0 with original loader

Found mostly in Taito games.  Works like Sierra's version, but with a different loader that is harder to decrypt.  The file CMLxxxx.FCL will be present.

Arkanoid
Arkanoid II: Revenge of DOH
Bubble Bobble
Operation Wolf
Puzznic
QIX
Rambo III
Rastan
Renegade
Sky Shark

Softguard Superlok v2.3 with Sierra's Loader

This protection was used in Sierra's games.  It relied on a track with non-standard sector sizes, overlapping sectors, CRC errors in the sector data field   There is a hidden file called CPC.COM on the first disk that checks for the protection.  All these games could be installed to a hard drive, but required the first disk, the key disk, to be inserted in a floppy drive.  Police Quest: In Pursuit of the Death Angel and Mixed Up Mother Goose, were the only AGI version 2 games that never seemed to be protected. This protection was applied to both booter and DOS games but was no found on AGI version 3 games.

3-D Helicopter Simulator (versions 1.00 & 1.10)
Black Cauldron, The (AGI versions 1.1J, 1.1K, Tandy & 2.00)
Donald Duck's Playground
King's Quest I: Quest for the Crown (AGI versions 1.0U & 2.0F)
King's Quest II:  Romancing the Throne (AGI versions 1.0W, 1.1H, Tandy, 2.1 & 2.2)
King's Quest III: To Heir is Human
Leisure Suit Larry in the Land of the Lounge Lizards (AGI version)
Mickey's Space Adventure (3.5" disks only)
Sierra Championship Boxing (1985 version)
Space Quest: The Sarien Encounter (AGI version)
Space Quest II: Vohaul's Revenge
Thexder
Ultima II (rev1)

Weak Bits Generic Implementation

Typically done by using more than three 0 bits following a 1 bit in violation of MFM encoding rules.

Green Beret  
RoboCop (Ocean)
Starglider II

After Burner
Bop'N' Wresting
Harrier Combat
Horror Zombies From The Crypt
Rick Dangerous II
Shinobi

POLA106

Uses weak bits

Ghosts 'n Goblins (rev1)

U.S. Gold

Uses weak bits

Solomon's Key 

Waydisk Minder
This protection will probably fail at greater than IBM AT (8MHz 286) speeds. Most games protected by this protection use weak bits

4 Soccer Simulators
AM, FM Trivia Vol. 2
Arac
Boulder Dash (Prism Leisure, no weak bit)
Boulder Dash 2 (Prism Leisure Release)
Circus Games
Football Manager
Football Manager 2
Frank Bruno's Boxing
Hotshot 
Ninja Rabbits
The King of Chicago
Mayday Squad Heroes
Speedball 
Super Sunday
World Championship Soccer

Xidex Magnetics XEMAG / XELOK

Used mainly by Broderbund.  Has 16 sectors instead of 8 on track 9, side 0.

Bank Street Writer
Championship Lode Runner
F-15 Strike Eagle 
Friendlyware PC Arcade (v3.0) 
GFL Championship Football
Inca 
The Last Ninja
Lode Runner (rev0, rev1, Tandy)
Quink (CBS Software) 
Rampage 
Sargon 3
Spitfire Ace 
Super Hang-On 
Transylvania 
Trivia Fever 
Ultima I (rev0) 
Ultima III (rev3) 
Word Challenge

Titus

Baby Jo in ''Going Home'
The Blues Brothers
Crazy Cars 2 
Dick Tracy
Les Aventures de Moktar (rev1) 
Fire and Forget (Titus Gap Count v1)
Fire and Forget II
Fire and Forget II
Highway Patrol II  
Knight Force
Off Shore Warrior
Panza Kick Boxing  
Pinball Magic 
Prehistorik (Titus Gap Count v1)

RKO

Agent U.S.A

Data Encore

Crime and Punishment 

Sector overlap

Commando (rev0) 
Ghostbusters 
Ikari Warriors  
Might and Magic (rev1)  
Might and Magic II   
Oo-Topos 
Sherlock Holmes - Another Bow 
World Games 

Sonopress

Grand Monster Slam

RPL

Ken Uston's Professional Blackjack
MicroLeague Baseball 

Infogrames

Safari Guns

Insoft

Wordtrix

Miscellaneous Protections

These disks contain unusual but valid MFM Data. Most use simple methods like non-standard sector seizes, which are probably the earliest method of copy protection and can easily be copied with TeleDisk.

Adventure in Serenia
Algebra Arcade
Alpine Tram Ride 
Alleycat 
Arkanoid (Imagine)+Super Tennis 
Below the Root 
Bivouac 
Centipede 
Changes 
Crypto Cube 
Cyrus 
Czorian Siege (requires DMA controller)
Dig Dug (Atarisoft) 
Electric Crayon Deluxe - TMNT - World Tour (v3.0) 
Facemaker (Gold Edition) 
Galaxian 
Gremlins 
Gryzor 
IDSI's Rack 'em Up
In Search of the Most Amazing Thing (v2.0)
Jumpman 
Jungle Hunt 
King's Quest (128k IBM)
Microsoft Flight Simulator (v1.x)
Microsoft Flight Simulator (v2.x) 
Moon Patrol 
Pitstop II 
Planetfall
Sid Meiers' Pirates! v432.03
Sorcerer (r4)
Touchdown Football (IBM) 
Trains (DOS version)
Trap-A-Zoid
Wizardry
Wizardry II
Wizardry III
Wizardry IV
Wizardry V
ZorkQuest II 

V.  Hardware Based Backup Methods

There are several programs that rely purely on the floppy disk controller to try to make a working backup, including Teledisk, ImageDisk, CopyIIPC & Snatchit.  None of them is a perfect solution.  TeleDisk did not take every PC protection method into account, but its great to write images with non-standard sector sizes.  ImageDisk is a newer program and doesn't have that many disk images available for it, but it too is bound by the limitations of the PC disk controller.  CopyIIPC and Snatchit only work on 100% compatible IBM PCs (no Tandy 1000s or PCjr.s), do not work at high 386 speeds or faster and potentially will modify the data on the disk image to crack the program.

The closest thing that was available back in the 1980s that could copy anything was Central Point Software's Deluxe Option Board.  This was a hardware ISA card that intercepted the signals between the disk drive and disk controller.  Because it operated on the flux transition level, it could theoretically copy anything from a double density disk.   It could write bad CRCs, overlapping sectors, interleaved sectors and data in the gap bytes.  This board used a program called TransCopy.  TransCopy could save an image of a disk for future rewriting.  360KB disks would have an image that would be 1,088KB in size.

TransCopy was not without its flaws.  The non-Deluxe Option Board could only work with TransCopy versions 4.x or earlier.  TransCopy versions below 5.x would not copy 80 track disks, inlcuding 720KB disks.  TransCopy images made with version 5.x would not work with earlier TransCopy versions.  TransCopy would also be crippled so that it would not work with certain protections, depending on which company was threatening CPS with litigation at the time.  You would need to hope that an earlier version of TransCopy would work.  Copy Copylock II will not be able to written back with an Option Board or Deluxe Option Board.

The modern equivalent of the TransCopy board are the Kryoflux and SuperCard Pro.  In the PC context, KryoFlux will create raw stream files of each track.  There are utilities that can convert these raw stream files to TransCopy or TeleDisk formats.  It can now write back raw stream files, so you no longer need to send your disk to the SPS and get an IPF file.  However, writing back raw stream files is not fully reliable because it does not account for fluctuations in drive rotation speed and imperfections in the signal, possibly leading to signal loss similar to copying analog tapes.  There are unsupported utilities to make your own IPF files, but it appears that unless you know the stream format well, you may not be able to write back a working disk. SuperCard Pro does the same thing but contains everything in one file.  The two formats require conversion. 

HDMIfy your Old Consoles - Console Specific HDMI Upgrades

If you want high quality input from classic consoles to a modern flat screen TV, you need to use an HDMI converter like the X-RGB Mini Framemeister.  This converter will take high quality RGB or Component video signals and convert them to HDMI-compatible 720p or 1080p with excellent results and minimal lag. I played with a Framemeister some months ago, and wrote up my impressions of the device here : http://nerdlypleasures.blogspot.com/2015/05/fun-with-framemeister.html  Unfortunately, some classic consoles have difficulty displaying high quality video signals due to their lack of high quality video outputs.  Some enterprising individuals have made mod boards that will directly convert the video to HDMI with less lag than a Framemeister, more features and more reliable results.  Here are the projects that, as of the date of this writing, actually have been released in some form to the public.

NES - HiDef NES Mod & the AVS

The NES is one of the most important of the post-crash consoles and one of the few without a native RGB solution.  There has been a NESRGB mod board released for two years now, which works wonders with a Framemeister.  Before that people harvested 2C03 chips from Playchoice-10 PCBs and made do with the differences in the color palette, compatibility issues with color emphasis and sometimes video jailbars.

Kevtris' HiDefNES mod has already been mentioned in this blog, and it adds a whole host of features over the NESRGB + Framemeister solution.  I lay them out in detail here : http://nerdlypleasures.blogspot.com/2015/07/analogue-nt-nes-as-luxury-retro-console.html
It is the board found in the Analogue Nt HDMI version and has been sold in pre-modded systems on ebay and on the http://www.game-tech.us/ website.  The kit will be available for purchase so you can attempt to mod it yourself.  However, it requires desoldering both the CPU and PPU without destroying either chip or the PCB, so it is not a beginner mod.  Virtually all the compatibility issues with the HiDefNES mod and certain MMC5 games and the EverDrive N8 have been eliminated through a firmware update, so now is the time to consider taking the plunge.

The only obvious issue is that its FDS audio emulation is less than perfect.  The mod will not work in an original Famicom, a Famicom Twin or an AV Famicom with laser-marked CPU and PPU chips.

The basic principle of how the HiDefNES obtains the color values of each background and sprite pixel is the same as used in the NESRGB.  See here for my explanation : http://nerdlypleasures.blogspot.com/2015/06/nes-hardware-explained.html
There is now a 2600RGB board for the Atari 2600.  The 2600RGB board uses similar principles to discover the colors of 2600 pixels as the NESRGB board does for NES pixels.  Theoretically, it is quite possible for someone to develop an HDMI mod for the 2600.

bunnyboy, who runs the RetroUSB site, has displayed his AVS replica console at the 2015 Portland Retro Gaming Expo. His AVS has a 72-pin and a 60-pin connector for NES and Famicom games, and is designed as a front loader for the former and a top loader for the latter.  It uses a completely new board and an FPGA chip to provide hardware emulation for the NES.  It only outputs HDMI video and has separate power and reset buttons in the shape of NES front loader buttons.  It has built in four NES controller ports that can be set to function as a NES Four Score or Famicom 4-player adapter.  It also has a 15-pin Famicom expansion port for Famicom peripherals.

Because the console only supports HDMI output, it will not work with any Zapper or compatible Light Gun device and it will not work with the Famicom 3-D Glasses.  The FPGA inside the console does not emulate any cartridge hardware or expansion audio.  Expansion audio from Famicom cartridges is digitized and then mixed with the internal audio.  However, because the AVS uses an FPGA, it can load updated firmware to fix any compatibility issues.  Reports from the Portland Retro Gaming Expo were very positive, one person said the system ran Micro Machines correctly, which is a hard game to get right.  bunnyboy is also designing wireless RF controller without lag to go with the console via an controller port adapter.

The console can only be powered through its USB port.  The console has a scoreboard reporting function via the USB port, it will save high scores for certain known games and report them to NintendoAge.  The idea is that you play a supported game, the high score is saved and then you plug in the system to your PC and it send the score to NintendoAge.  The AVS only supports 720p while the HiDefNES can do either 720p or 1080p.  The AVS is not for sale at the moment, but bunnyboy is aiming for a just under $200 price point.  The best place for updates seems to be here : http://nintendoage.com/forum/messageview.cfm?catid=7&threadid=92557

N64 - Ultra HDMI

No N64 natively outputs RGB, but some early consoles can be modded with a simple amplifier board. Most consoles, however, have a video encoder chip that only outputs composite and s-video.  These consoles require the N64RGB board.  This board acts like a custom DAC, taking the digital signals from the N64 and turning them into analog RGB video.  This will make it truly Framemeister worthy, although the s-video output from the N64 is pretty good compared to composite video.  PAL N64s need a special cable to boost the S-Video signal from the console, NTSC N64s can use standard Nintendo or 3rd Party cables.

Today, if you want to bypass the Framemeister option, you can have an UltraHDMI board installed in your system.  It works with NTSC and PAL framerates (up to 1080p for both) and not unlike the HiDefNES it comes with filters.  The Ultra HDMI filters can simulate a CRT TV.  Without the filters the lag is truly negligible, with them there is about a frame of lag.  It supports all standard digital SD and HD resolutions, as shown here : http://ultrahdmi.retroactive.be/

The mod is very involved because you have to solder a flexible mylar-like ribbon cable to the finely spaced pins of the surface mounted graphics chip, the Reality Co-Processor (RCP).  The RCP sends out 7-bit digital RGB values along with sync information.  This allows for direct digital to digital conversion to the 8-bit RGB values HDMI supports without quality loss.    The kit uses a mini-HDMI cable and only needs a small hole cut into the back of the N64.  Because the mod is very tricky and not for the beginner, the board and kit is being sold only experienced mod kit installers.  If you want your N64 modded with the Ultra HDMI, you need to send it to someone.  This site is authorized to perform the mod : http://www.badassconsoles.com/ultrahdmi/

Game Boy - hdmyboy

In one sense, it is not too difficult to obtain a good quality HDMI image from a Gameboy.  You use a Super Gameboy or a Super Gameboy 2 (for accurate speed) on a SNES that supports RGB and send the video and audio through a Framemeister.  Alternatively, you can use a Game Boy Player and a Gamecube with the component video cables through the Framemeister, but that requires running custom software to make the Game Boy Player output at a true 240p speed.

Some time ago, there was a Kickstarter for a product called the hdmyboy.  The campaign is archived here : https://www.kickstarter.com/projects/507669971/hdmyboy-a-full-hd-power-up-for-your-game-boy-class
This is a fully assembled kit that you insert between the two halves of a DMG-01 Game Boy.  It adds half an inch or so to the height of the console when laid flat. It has an HDMI cable port and can support 720p or (preferred) 1080p.

Unlike the NES and N64 mods, this mod is easy to install.  It requires no soldering.  All you have to do is unscrew the shell, remove the ribbon cable that attaches the LCD PCB to the CPU PCB and insert that ribbon cable into the hdmyboy.  The hdmyboy comes with a NES controller attached internally so you can control your Game Boy.  It essentially becomes a consolized Game Boy at this point.

The Game Boy's LCD can only support four shades of one color, its video signal is fundamentally 2-bit digital monochrome signal with sync information.  This information, along with mono audio, is available through the ribbon cable.  The hdmyboy converts the digital video and the analog audio and sends it through the HDMI cable to the TV.  The hdmyboy only supports mono audio because only a mono audio signal is necessary for the single speaker connected to the LCD PCB.

Unfortunately the Kickstarter campaign was not successful.  However, the hdmyboy team still has some prototypes left here : http://www.hdmyboy.com/play/.  They are very expensive however, (a bit too expensive for what it does) but should represent the best solution to obtain true Game Boy output without using an emulator.  If they try another Kickstarter campaign again, then hopefully more people would be willing to contribute and get the price of this fine idea down.

Given a 1080p display, the hdmyboy can by default do 7x nearest neighbor scaling to give razor sharp graphics at the Game Boy's original aspect ratio.  It can also do 12:7 scaling to give razor sharp widescreen graphics and occupy almost the whole screen if you prefer.  Like the Super Game Boy it has 32 color palettes available to colorize the graphics.  While the Game Boy's sprite and background tiles have 10 valid palette selections to choose from, the Super Game Boy (unless the game is enhanced) and hdmyboy only apply colors based on the actual color value of the outputted pixel.

Sunday, November 1, 2015

Weapons in Mega Man 1, 2 & 3

The classic Mega Man games have always had a non-linear progression as a hallmark of the series.  Instead of following a set path from beginning to end, you begin the game by choosing which level you will tackle in any order.  Some levels may be easier than others.  Each level has a Robot Master at the end which you must fight and defeat to beat the stage.  When you beat a Robot Master, you will acquire his weapon for use in other stages.  Each Robot Master weapon has its own energy meter and its own attack.  One or more of the other Robot Masters will be weak to that attack.

Prior to Mega Man 4 (and Mega Man III for the Game Boy), which added the Mega Buster charged shot to the series, the weapons were very important in the Mega Man series.  However, not all were created equal.  In this blog entry, I would like to give my impressions of the weapons in each of the first three Mega Man games for the NES and discuss the Robot Master weaknesses.  Let's delve into the central design mechanic of the Mega Man series from the beginning :


The designers of Mega Man intended the Robot Masters to each be weak against a specific weapon taken from another Robot Master and stronger or invulnerable to others.  This rock-paper-scissors design would remain throughout the series.  Mega Man and each Robot Master has 28 units of life energy.  A weak attack may only take one unit, but a strong attack could take four or ten or fourteen.  Mega Man's Arm Cannon would always take at least one unit for a successful hit, regardless of enemy.


In the original Mega Man, each weakness had an easy to understand rationale.

Cut Man is weak against Guts Man's weapon*
Rationale : Rock blunts scissors

Guts Man is weak against Bomb Man's weapon
Rationale : Bomb blasts rock

Bomb Man is weak against Fire Man's weapon
Rationale : Fire detonates explosives

Fire Man is weak against Ice Man's weapon
Rationale : Ice/water extinguishes fire

Ice Man is weak against Elec Man's weapon
Rationale : Ice/water conducts electricity

Elec Man is weak against Cut Man's weapon
Rationale : Electricity cannot flow once its circuit is cut

* - Unlike every other canonical Mega Man game, the Robot Masters weapons are not named in the game or manual.  The manual also refers to the Robot Masters as one word, "Cutman", and identifies their and Mega Man's creator as "Dr. Wright."  In the sequel it was firmly established as Dr. Light.  Sometimes, the games refers to "Dr. Wiley" or "DR. WILeY", but the accepted spelling of the villain's last name is "Dr. Wily."

Not all of these weapons are equally useful.  Guts Man's weapon only works when there are rocks around to pick up.  Bomb Man's weapon does not travel far and does not deal damage instantly, so it is usually limited in its application.  Is it any wonder that these two weapons are nowhere to be found in the Gameboy game? Ice Man's weapon usually only freezes enemies and is useless against most Robot Masters.  Fire Man, Cut Man and Elec Man's weapons are much more general purpose weapons.  The latter two can cause even more damage to a boss by the select/pause button exploit.

Some Robot Masters were made easier to defeat with the Arm Cannon than others, after all, the player had to find a way to break into this weakness loop.  Most players usually start with Bomb Man or Cut Man.  The Arm Cannon takes two units from Bomb Man and three from Cut Man, but Cut Man has to be fought in closer quarters and is more aggressive of the two Robot Masters.  Guts Man and Fire Man can also be beaten with a little practice and a full health meter.  Ice Man is more difficult because you have to follow his pattern very precisely and his projectile attack hits for a lot of damage.  Elec Man is by far the hardest of the Robot Masters to defeat because he is aggressive and so fast that his patterns are harder to memorize.

There is a special item you can obtain in this game called the Magnet Beam.  It allows you to construct temporary platforms which you can use to walk or ascend the screen.  It is behind three moveable blocks in Elec Man's stage.  It is necessary in Dr. Wily's stages and extremely useful in Ice Man's stage.  It is also very useful in Guts Man's stage, but you need Guts Man's weapon to move the blocks.  Alternatively, you can defeat Elec Man and go back to his stage to retrieve the Magnet Beam using Elec Man's weapon to destroy the blocks.  Mega Man allows you to revisit stages, but Mega Man 2 and 3 do not.


Now let us turn to Mega Man 2.  Mega Man 2 introduced the standard eight basic Robot Masters. However, they no longer are as grounded in basic concepts like their predecessors.


Metal Man is weak against Quick Boomerang
Wood Man is weak against Atomic Fire
Bubble Man is weak against Metal Blade
Air Man is weak against Leaf Shield
Crash Man is weak against Air Shooter
Heat Man is weak against Bubble Lead
Flash Man is weak against Metal Blade
Quick Man is weak against Time Stopper

The following assumes you are playing on the "Difficult" difficulty level, which is the only difficulty level in the original Japanese version.  In addition to the above, Metal Man is even weaker to his own weapon than to anything else.  The Time Stopper will only take half of Quick Man's life and thereafter the Crash Bomber is the weapon that will damage him the most.  As you may have noticed, Flash Man and Bubble Man share the same weakness. The Atomic Fire weapon can damage many of the Robot Masters more than the other weapons, but the fully charged shot takes half of its energy meter.  You can only defeat Wood Man using the Atomic Fire weapon alone.

Some of these pairings make some sense.  Time Stopper is a natural assumption to use against Quick Man.  Bubble Lead, a water-type attack, makes sense against Heat Man.  Moreover, Heat Man's attack pattern practically invites you to use Bubble Lead whereas Crash Man's pattern does not.  Fire burns wood, so Wood Man's weakness makes sense.  If you want to go out on a limb, I suppose you could say that the Leaf Shield can block the air intake of Air Man, who has a big fan in his chest.  The Metal Blades can cut through the swim gear of Bubble Man.  But why should Metal Man be weak against the Quick Boomerangs?  Why should Crash Man be so weak to the Air Shooter?  I don't really see the particular connection between Flash Man and the Metal Blades.

You may note some similarities between some of the weapons in Mega Man and Mega Man II.  The Quick Boomerang has an attack pattern nearly identical to Cut Man's weapon.  Fire Man and Atomic Fire have similar weapon properties, as do Bubble Lead and Ice Man's weapon.  Crash Man's bombs serve a similar function to Guts Man and Elec Man's weapons.

Weapons are more unbalanced in Mega Man 2 than in Mega Man.  The Metal Blade is easily the best weapon in the game, its fast, can shoot in eight directions, damage most enemies and takes quite a while to run out of energy.  The Quick Boomerang can be useful and lasts a while, but its range is short.  The Bubble Lead weapon has occasional uses.  The Time Stopper runs down its energy quickly and is best suited to Quick Man's stage and Quick Man itself.  Crash Bomber is best used for destroying barriers and is the only weapon that will work against the boss in Dr. Wily's castle, stage 4.  Heat Man's weapon is slow to fire and a fully charged shot takes half the energy meter, so it is usually reserved for Wood Man and Dr. Wily's first form.  Air Man's weapon is usually useless except against Crash Man and drains quickly.  Wood Man's weapon is almost certainly the most useless in the game, when does a skilled player need its protection and it is a pretty weak attack.  Even Air Man may go down faster to the Arm Cannon than to the Leaf Shield because his air jets can block the shield.

There are three acquirable Items in this game.  You acquire an Item by beating one of the Robot Masters.  Item-1 creates up to three temporary platforms which hover upwards for a short distance.  Item-2 allows you to travel across the screen in a flying platform.  Item-3 allows you to climb walls.  All three are necessary for Dr. Wily's Castle.  Items 1 and 2 are especially useful in Heat Man's stage to pass the areas with the disappearing blocks easily.

Although I do not find the original Mega Man to be overly difficult (anymore), Mega Man 2 added a password system and you could collect up to four disposable Energy Tanks that could refill your health meter at any time.  These two additions made the game much more forgiving.  For non-Japanese audiences, a Normal mode was added to make the game much easier to beat.  In Normal mode, everything takes twice as much damage from your weapons and the enemies give better item drops.  Mega Man 2's Difficult mode is the only mode in the Japanese version.


Finally, I will talk about Mega Man 3.  Mega Man 3 built on Mega Man 2's success and is a great game in its own right.  Nothing has changed on the weapons front.  Its weapons, however, leave even more to be desired.


Top Man is weak against Hard Knuckle
Shadow Man is weak against Top Spin
Gemini Man is weak against Search Snake
Needle Man is weak against Gemini Laser
Snake Man is weak against Needle Cannon
Spark Man is weak against Shadow Blade
Magnet Man is weak against Spark Shock
Hard Man is weak against Magnet Missile

Magnet Man is equally weak against Shadow Blade.  Shadow Man is also very weak against his own weapon.  Except for Hard Man and Spark Man, the logic of the weaknesses of the Robot Masters here is beyond me.

The best weapon in the game is the Shadow Blade.  It can fire in five directions, has good speed, decent range and comes back to you.  Essentially it is a more balanced version of the Metal Blade.  Needle Cannon is also pretty good, it fires fast and lasts a while.  Spark Shock is also fast and travels far.  Search Snake is pretty limited in its usefulness and often really weak, the Gemini Laser is slow to travel, you can only have one shot out at a time and it takes several ricochets off walls for each shot to expire.  Note that Search Snake has a slightly improved attack pattern over Bubble Lead. The Magnet Missiles exhaust themselves too quickly to be of much use.  Hard Knuckle is so slow that it is really only useful to break down barriers and in boss fights, like Crash Bomber.

Top Spin takes the prize here as the worst weapon in the first three Mega Man games.  It is worse than the Leaf Shield, the Air Cannon, Bomb Man and Guts Man's weapons.  There are three issues with the Top Spin.  First, you have to be right next to the enemy to register a hit, leaving you prone to being hit in return.  Second, the amount of damage it does is very variable.  Third, it drains really quickly.  You can spin against Shadow Man until Top Spin's energy is depleted and still not defeat it.  The second time around, you will almost certainly use another weapon against Shadow Man.  While there is one boss in Dr. Wily's castle that is vulnerable to the Top Spin, it is also vulnerable to another weapon, fortunately.

Mega Man's companion/helper Rush is introduced in this game.  Rush is a robot dog who can perform three abilities.  The first ability is the Rush Coil and is present when you begin the game.  It acts like a spring to allow you to jump high. The second is the Rush Marine and the third is the Rush Jet.  The Rush Marine allows you to travel freely in water and is useful in Gemini Man's stage.  The Rush Jet allows you to travel freely in the air and makes Magnet Man's stage a piece of cake as well as any other area with a lot of pits.

Mega Man 3 allows you to collect up to nine Energy Tanks.  There are some levels where you can get more than one Energy Tank or get one early in the level so you can collect them quickly.  Additionally, the slide move makes its first appearance.  With it, you can usually slide under any Robot Master who jumps and get away from them quickly.  This makes using a Robot Master weapon a little less necessary in many instances.

Dr. Wily's Revenge

The Game Boy game is something like a remake or adaptation of the original game with elements of the sequel thrown in.  The original Mega Man had six stages and four smaller parts to Dr. Wily's stage.  For the small screen, some cuts had to be made.  Gone are Bomb Man and Guts Man and their stages.  Dr. Wily's Castle and Space Ship are longer than any Robot Master's stage.  The game is six stages long, but the last two stages are quite difficult.  The health and energy meters were decreased from 28 to 19 units.


Cut Man, Fire Man, Ice Man and Elec Man are the Robot Masters you can choose from in this game.  Fire Man's attack has been simplified.  Elec Man has become much less aggressive and his attacks do far less damage, making him the usual Robot Master to start out with.  Their weakness have not changed from the NES game, but the best weapon to use against Cut Man is Fire Storm.  Their weapons are essentially the same as in the NES game.

When you get to the end of Dr. Wily's Castle, you will see teleporters you can take in any order to fight Bubble Man, Heat Man, Flash Man and Quick Man from Mega Man 2.  Their weaknesses have not changed, but the Metal Blade does not exist in this game, so you are intended use Atomic Fire against Flash Man.  They really are not particularly weak to any of the weapons from the first set of Robot Masters.  The only major change is that Heat Man's weapon does not take up as much energy for a charged shot.  It can also destroy the barriers on Dr. Wily's Space Ship.

Once you defeat the second four Robot Masters, you must fight Enker, who only is affected by the Arm Cannon. Enker gives up the Mirror Buster, which reflects enemy shots. Unfortunately, once you obtain their weapons, you have only one stage to use them.  The final password only allows you to continue before Dr. Wily's Castle, so if you want to try to beat the game again you have to fight the Mega Man 2 Robot Masters again.

There is one item that can be acquired in this game, the Carry item.  You obtain it after you defeat the final Robot Master of the initial four.  It creates a platform to the side or underneath you for a short period of time.  Only one can be active at a time, but you can use it to save yourself from a deadly fall or spikes or to reach areas where you cannot jump to unaided.  You cannot change to a weapon while it is active, but you can conserve its energy by leaving the screen it is on.


Capcom used Arabic numerals to designate the NES games and Roman numerals (and the subtitle above) to designate Game Boy games on their respective boxes in order to tell them apart.  I will do the same for this article.


Mega Man II gives the other half of the Robot Master contingent from Mega Man 2: Air Man, Crash Man, Metal Man and Wood Man.  After you get to the end of Dr. Wily's Castle, you will be able to go through teleporters for new stages for half of the contingent of Mega Man 3: Needle Man, Top Man, Magnet Man and Hard Man.  After you beat them and acquire their powers, you will fight Quint and obtain his power, the Sakugarne, which is a pogo-stick.  Then it is onto the final stage, Dr. Wily's Space Ship.  Other than actual stages instead of arenas for the second set of Robot Masters, not much has changed from its predecessor.

Rush, Energy Tanks and the slide move make their Game Boy introduction.  Rush transforms into the Coil, Jet and Marine.  You can obtain up to four Energy Tanks in Mega Man II.

I should talk about Robot Master weaknesses in this game, but this game is so poorly done that you can beat them all with the Arm Cannon.  The Robot Master's attacks and patterns are fairly simplistic and do not usually do much damage.  Quint is a joke and Dr. Wily's final forms won't cause you grief, unlike the previous game.  The music stinks as well.  Considering that the engine from the previous game was quite robust, the lack of inspiration in this game is appalling.

Okay, so in this game, Metal Man is now vulnerable to Crash Bomber.  On the other hand, Crash Man is not nearly as vulnerable to the Air Shooter, but it remains the best weapon to use against it.  Neither Shadow Man nor Spark Man are present, so Magnet Man is now vulnerable to Needle Cannon.  Similarly, Gemini Man is not present, so Needle Man is now vulnerable to Top Spin.