Sunday, November 15, 2015

Will the Real Successor to the Pitfall Legacy on the NES Stand Up?

There never was a game called Pitfall III.  David Crane programmed the original Pitfall for the Atari 2600 in 1982 while working for Activision, a company he co-founded.  Two years later he released Pitfall II: Lost Caverns for the Atari 2600.  Thereafter came the great Video Game Crash and Crane left Activision in 1986 to co-found Absolute Entertainment.  While there would be other games in the Pitfall series, this blog entry will discuss three games released or were to be released during the third generation of home video game consoles would could compete for the right to be called the successor to the Pitfall Legacy.

In the Beginning: Pitfall and Pitfall II: Lost Caverns

Pitfall essentially made David Crane as close to a household name as video games got before the Crash and cemented his reputation as a game designer worthy to be inducted into The Academy of Interactive Arts & Sciences' Hall of Fame.  This game was a massive success in its day, selling over four million copies and ported over to all the major consoles and many of the popular home computing platforms of the day.

Starting Screen with Stationary Log
Pitfall is one of the first platforming and side-scrolling games, even though the Atari 2600 hardware did not accommodate scrolling and this game only used static screens.  This game was inspired by Raiders of the Lost Ark and invited players to explore a large, non-randomized world.  In fact, Pitfall is actually far more popular than the official 2600 Raiders adaptation because Crane was able to pare the exploration concept to the basics while still maintaining a sense of fun.  You can just pick up and play Pitfall, Raiders requires reading the manual and a lot of trial and error in order to make any sense out of it.

Swinging over a crocodile lake
There are many dangers in Pitfall : tar pits, lakes (both of which can expand and retract from what appears to be solid ground), crocodiles, scorpions, snakes and campfires that can kill your character, Pitfall Harry.  Logs can steal points if you collide with them, vines can allow you to cross over dangers, you can jump on the crocodile's heads to get over water, ladders and holes allow you to go underground and the underground passages allow you to "warp" a few screens, if there is no brick wall in the way to block you.  Multi-colored "sprites", a relatively detailed background and catchy sound effects and jingles make the game a visual and aural treat for the 2600.

Swinging over a tar pit and rolling logs
Finally, there is plenty of challenge in that you have to locate 20 treasures strewn across 256 screens in 20 minutes with 3 lives.  You will need to map and plan carefully and identify useful warp passages if you want a good chance of finishing the game.  Important to note that unlike most other Atari 2600 games this game will end (it just stops) if you find the twentieth treasure.

Camping out
One other important theme to note here is that Pitfall Harry has no way to kill any of the enemies which he encounters.  In Pitfall, you have to jump over or otherwise avoid all obstacles, including animals.  This aspect will be maintained in the other games Crane designed and discussed in this article.  Ultimately finding non-violent solutions is a recurrent theme throughout almost all the games Crane has designed.

I forgot to bring my tambourine
Two years after Pitfall, Crane released his sequel, Pitfall II: Lost Caverns.  He did not reinvent the wheel or take the series into a radical new direction.  Some sprites and mechanics were taken from the previous game and the two games are visually similar.  However, he expanded the concept by taking the basic exploration theme of the original game (which only used the horizontal plane) and expanded it into the vertical plane.  There are eight horizontal screens and twenty-seven vertical depth levels which you can explore.  In other words instead of having 255 screens stacked together horizontally, you have a grid of 8x~8 screens, adding an extra dimension to the world.

Starting Screen with Checkpoint and Eventual Goal
Now Pitfall Harry can swim in water and has checkpoints to start at if he is "killed". You do not die in this game, you just get sent back to the last checkpoint.  Nor is there a time limit.  There are several new enemies, like electric eels, frogs, vultures, bats, the rat and the return of the scorpions.  The  vultures and bats travel in a repeating pattern across the screen.  In addition, each time you enter the screen they may start from a different position than the previous time you entered the screen.  This makes it hard judge when you can run underneath them, but you have to run past them to make progress in many areas.  At one point you will need to catch a balloon to progress further in the game.  

Bat and Rat
You can find typical gold brick treasures for points, but the object of the game is to find your pet Quickclaw, your girlfriend Rhonda and a diamond ring.  The game ends when you obtain all three with Harry doing some victory jumps before the game stops.  You loose points by colliding with enemies or falling and landing after a certain height.  The screen scrolls vertically as you descend and explore the "lost caverns".  The look of the game is still very much in line with Pitfall, and like Pitfall, the only thing you can do is climb and descend ladders, walk left and right or jump.

Scorpion, Swimming and Treasure
Crane designed a special hardware chip for the cartridge called the DPC.  Because of this chip, Pitfall II was undoubtedly the most complex cartridge ever to be released on the Atari 2600 during its lifetime.  The DPC chip allows bankswitching up to 10KB of ROM (Atari carts are only 2K or 4K without bankswitching; Pitfall is a 4K game), a hardware random number generator, hardware to assist with graphics processing and hardware to generate the data to feed into the internal 2600 audio to create three additional channels of sound.  There was not enough space to put the Adventurer's Edition second quest into the 2600 original that was present in the Atari 5200, 8-bit and Commodore 64 versions of the game.

Watertfall and Electric Eel
Frogs, Ladders and Multiple Levels
Thanks to the DPC chip, Pitfall 2 has music playing throughout.  There are four pieces of music, the treasure theme, the return to checkpoint theme, the default theme and the balloon theme.  This music plays according to what happens during the game.  Multi-channel music as heard in Pitfall 2 is difficult to pull off on the 2600, but when coupled with an advanced game engine like Pitfall 2's, it needed hardware assistance.

Multi-level Madness
Vultures
A Boy and his Blob


Title Screen
Start Screen
The 2600 was too long in the tooth when David Crane was developing A Boy and his Blob, the 7800 and the Sega Master System did not have the market share and the Commodore 64 was dying in the US.  Having left Activision, he had no more right to create a Pitfall sequel than I would.  He ended up creating A Boy and his Blob for the dominant platform of the day, the NES.  A Boy and His Blob to me feels like Pitfall III and in addition to having its creator at the helm, let me explain the design similarities between this game and its Pitfall predecessors.

No money, no healthy food

A subtle way to inject the production team into the game
The Boy in the title is less capable than Pitfall Harry, he can only move/run left and right, he cannot jump unaided.  However, he has a companion, the Blob, which functions as a puzzle solving device.  Using different flavored jelly beans, the Blob can do different things.  Need a ladder, feed him the licorice jelly bean.  Want to drop down to the next level, turn the Blob into a hole with the punch jelly bean (you can do this multiple times for one bean).  While you have 15 flavors to choose from, your supply of jelly beans is limited and you need to use them wisely.  The Blob and jelly bean mechanic represents an evolution over using the character's own abilities to get past obstacles.

Treasure and Subway Serpent
Don't try this in real life
Like Pitfall 2, this game requires exploration of caverns beneath the surface.  Similarly, the game is structured in a grid of non-scrolling horizontal screens.  Unlike Pitfall 2, the screens do not scroll vertically, despite the superior NES hardware.  I do not believe this was done because David Crane could not figure out NES scrolling but because the levels were so vast and less structured than Pitfall 2 that it would have made map making much more difficult.

Cinnamon & Spice
The caverns have many dangers, including subway serpents, falling rocks, spider webs and stalactites and stalagmites in the water.  However, unlike Pitfall 2, the omnipresent danger is death by falling.  If you fall more than one and a half screens (without a device that can ease your fall), you die.  You start off with five lives in this game and no continues.  Also, your Boy moves a lot looser than Pitfall Harry, so you may run into something you cannot avoid and cannot turn back quickly enough to lose a life.  You cannot swim but the Blob can help you with that.  Finally, if you lose a life you will start at the point where you found your last treasure similar to Pitfall 2's checkpoints.

That Blob is very strong
But running under those subway serpents is still tricky
Mind your height
The goal of the first half of the game is to find treasures and escape the caverns.  There are 22 treasures located in different areas of the underground caverns, similar to Pitfall 2.  At the bottom of the caverns you will find an underground lake just like in Pitfall 2.  These treasures allow you to buy vitamins that you will need on the Blob's home planet, Blobolonia.  The second half of the game is a shorter experience where you have to dodge more enemies and defeat the evil, sweets-loving emperor.  Ultimately, gameplay is the key focus of this game.  The graphics are drab and backgrounds are mostly black underneath the subway.  It is quite easy to get lost because screens often look alike.  Music is limited to one basic piece of music for the game (in addition to the title screen music) and a few sound effects and ditties that play at certain times.

You will need to do this very often
Super Pitfall

Title Screen
I am aware that there is a game for the NES called Super Pitfall, but that is an abomination that David Crane had nothing to do with.  It is a port of a Japanese PC-8801 game by Pony Canon where is was called Super Pit Fall. Post-founders-era Activision licensed the title and published it as one of its first NES games. Activision may hold the rights to the Pitfall brand, but that does not mean it earns a place here considering its pedigree.

Spider and Bat Enemies and Water
Super Pitfall is a stripped down port from programmers who did not quite get the NES.  Scrolling is choppy, graphics are barely NES-worthy, the music is a 15-second piece intended to give you an idea of what hell feels like and there is quite a pause as the level loads.  You can now duck and shoot a gun with limited ammo and there are a couple of bosses, which pretty much sums up the evolution of this game over Pitfall 2.  Hit detection is unfair.  Even the manual states that the gun will feel useless at times, which is refreshingly honest. You can see glitches with sprites not infrequently.  The gun mechanic does introduce an element foreign to Crane's Pitfall games.  Also, items are invisible, so you have to jump in certain places to make them appear.  The game is filled with cheap deaths, in fact if you go down the first ladder in the game you will almost certainly die.

Bats and Frogs
Like Pitfall 2, you have to obtain the diamond, the pet and the girlfriend to win the game, but there are more obstacles to overcome. The world is huge (270+ screens) but you only have three lives and no continues, so it will be difficult to get a feel for it.

Waterfalls, Spikes and Treasure
Super Pitfall 2


Title Screen
There was going to be a game called Super Pitfall 2 released for the NES, but was canceled.  There is a prototype ROM floating about.  Super Pitfall 2 was a port of another Japanese game, this one a Famicom game called Atlantis no Nazo (Mystery of Atlantis) by Sunsoft.  The prototype appears to differ from the Japanese original only in its title screen.

1st Zone
Unlike Super Pitfall, Super Pitfall 2 is competently programmed.  Sunsoft was willing to hire competent programmers, Pony not so much.  The graphics are still bland but the music is pretty decent and the control is not quite as frustrating.  Your weapon is a bomb that you can throw, but it detonates when it wants to and can kill you if you are in its (short) blast radius.  The jumping could use more polish (you cannot change your trajectory in mid jump like Castlevania) and the game will get frustrating very quickly.  The bats dropping guano as a weapon is a nice touch.  The shell crabs will hide in their shells if they perceive a bomb coming, so killing them is a bit tricky.  Still, the violent solution is out of place in Crane's Pitfall games.  

2nd Zone
Ultimately, Super Pitfall 2 is really a horizontal side scroller.  The level progression is fairly linear and it does not feel like a Pitfall game at all.  You can find doors that will take you to a small portion of a later zone, but the game is still linear.  Even Super Pitfall was better at conveying an open world.  Pitfall's influence on the game is still present.  The enemies appear to be mostly of the natural variety and there are treasure chests you can open for points.  Between its lack of a real Pitfall feel and its rather long-in-the-tooth status for 1989 when Activision was considering porting it, it is no surprise that it eventually went unreleased.  An optimist may wish to believe that Activision saw that its former founder had released a Pitfall-like game in the same year and did not want to compete with him, but my idea is probably the more likely reason.

3rd Zone

Wednesday, November 11, 2015

IBM PC Floppy Disks - A Deeper Look at Disk Formats and Copy Protection

I.  Overview of Low-Level Floppy Disk Structure

I am going to be using a standard 360KB floppy disk as an example here.  A standard MS-DOS format command formats a 360KB floppy disk with 40 cylinders, 9 sectors per cylinder and 512 bytes per sector.  I distinguish between cylinders, which use both sides of the disk, from tracks, which only use one side of the disk.  Thus you get a total disk size of 368,640 bytes.  However, that is the space available for data bytes and a standard disk dump just dumps the data sectors.  In between the data there is extra data allowing the disk drive to find the right data.

A track's data starts with the Index Pulse which is sent when a disk's index hole passes over the index sensor. Then follow various kinds of fields on the disk which provide information about the disk's position, the location of the data and the data itself. There are three types of Fields found on a disk, the Index Field, the Sector ID Fields and the Sector Data Fields.

A. Gap & Sync Bytes

Gap Bytes precede and follow all three Fields and are first seen immediately following the Index Pulse. These Gap bytes, typically hex 4E, are used to compensate between slight differences between the drive that originally wrote the disk and the drive that is currently writing the disk. They separate the three types of Fields found a disk. Without some tolerance, the Fields may overwrite each other.  

After the Gap bytes come the Sync bytes, which help tell the drive that there will be location information or data to follow. Sync bytes follow a typical pattern of twelve 00s followed by three A1s.  These A1 bytes are missing the normal clocks bits and do not follow proper MFM encoding rules so the drive controller can figure out that this is not real data and the real data is coming up.  The A1s are also called the Sync Mark or just "A1 Sync".

B. The Index Field

There is only one Index Field on a track and it is the first Field which is present after an Index Pulse. The Index Field is not normally used by PC compatible disks and IBM did not even define the area in its manuals. Many disks do not have a distinguishable Index Field but others do. If a disk has no Index Field there will just be GAP bytes until the first Sector ID Field. The Index Field came from Shugart, which invented the floppy disk drive. There are protection schemes which rely on the presence of an Index Field.

The Index Field is a little different from other two Fields. Instead of having Sync bytes of twelve 00s followed by three A1s, the Index Field has twelve 00s followed by three C2 bytes. These C2 bytes also are not valid MFM bytes (see the discussion for A1 Sync below). Following the C2 bytes there is a single byte of FC, called the Index Address Mark (IAM).

C. Sector ID Fields

Each sector on the disk has a Sector ID Field and a corresponding Sector Data Field.

After the A1 Sync Bytes, a Sector ID Field begins with an FE byte as an ID Address Mark (IDAM). Then the Sector ID field continues with four bytes: the track/cylinder number (00-27), the side/head number (00-01), the sector number (01-09) and finally the sector size ID byte (00-06).  Unlike everything else, sector numbers use a convention starting with 01, not 00.  The size ID byte tells the system how large the following Sector Data field is and is interpreted as follows :

Size ID Byte   Actual Size in Bytes
00             128
01             256
02             512
03             1024
04             2048
05             4096
06             8192

The only sector size the standard DOS FORMAT command will use is a 512 byte sector size.  This applies to all standard 5.25" and 3.5" disks and drives, whether double density (160KB, 180KB, 320KB, 360KB & 720KB), high density (1.2MB and 1.44MB) or extra high density (2.88MB).  You can use third party utilities to format disks with other sector sizes, but DOS will not read them.  Note that a track on a 360KB disk cannot really fit 8192 data bytes.  I have never come across a 5.25" double density disk that has had more than 10 x 512 byte sectors.

The next two bytes form a 16-bit Cyclic Redundancy Check (CRC) value. The CRC value verifies that the information contained in the Sector ID field is correct. The bytes calculated include the A1 Sync bytes and the IDAM byte with a Polynomial of 0x1021 and an Initial Value of 0xFFFF. The CRC value is given in big-endian on the disk.

D. Sector Data Fields

As stated above, after the next series of Sync bytes comes a Sector Data Field.  The Sector Data Field begins with an FB byte as a Data Address Mark (DAM).  Then comes the actual data, the bytes you see with a standard disk dump.  Finally comes the 16-bit CRC value, calculated as with the Sector ID Field but with the whole 512 bytes of data.

A Sector ID Field, without the ID Address Mark and CRC Value, is 4 bytes large.  A Sector Data Field without the Data Address Mark and CRC Value, is typically 512 bytes large, otherwise the size is whatever the Size ID Byte indicates. 

For standard DOS compatible 160KB, 180KB, 320KB and 720KB disks, the only thing that changes is the track number, side number and sector number.  160KB and 180KB disks only use 1 side of the disk, 160KB and 320KB disks only use 8 sectors, and 720KB and larger disks use 80 tracks.  High density disks use 15 (1.2MB), 18 (1.44MB) or 36 (2.88MB) sectors per track.  

In a hex viewer, a full dump of the start of a disk's track plus its first sector from the index pulse might look like this :

4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 00 00 00 00 00 
00 00 00 00 00 00 00 C2 C2 C2 FC 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 00 00 00 00 00 00 00 
00 00 00 00 00 A1 A1 A1 FE 01 00 01 02 E1 07 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 00 00 00 00 00 00 00 00 00 00 00 00 A1 A1 
A1 FB 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 
41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 
54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 
41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 
54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 
41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 
54 41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 
41 55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 
55 53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 
53 45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 
45 52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 
52 44 41 54 41 55 53 45 52 44 41 54 41 55 53 45 52 
44 41 54 41 71 12 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 4E 
4E

Green = Gap Bytes
Red = Sync Bytes
Blue = Index, ID and Data Address Mark Bytes
Magenta = CRC Bytes
Black = ID and Data Fields

The hex values in the data field repeat the ASCII for USERDATA over 64 times.  

E. Gap Length Conventions

The number of bytes for any given gap, and the value of the gap byte itself, are usually standardized. There are 80 Gap Bytes before the IAM (Post-Index Gap Bytes), 50 Gap Bytes after the IAM, 22 Gap Bytes between an ID Sector Field and a Data Sector Field and 80 Gap Bytes between a preceding Data Sector Field and the following ID Sector Field. There is also a variable number of Gap Bytes following the last Data Sector on the disk. Following the last data sector, there are Gap Bytes until the Index Pulse is sensed (Pre-Index Gap Bytes). These various Gap Byte areas have been given certain identifiers in historical data sheets and manuals:

Pre-Index Gap Bytes/GAP 4B  = variable, normally 218 bytes (4E)

Index Pulse

Post-Index Gap Bytes/GAP 4A = 80 bytes (4E)
Index Field Sync = 12 bytes (00)  + 3 bytes (C2)*
Index Address Mark = 1 byte (FC)*
Post Index Field Gap Byte/GAP 1 = 50 bytes (4E)

ID Field Sync  = 12 bytes (00)  + 3 bytes (A1)
ID Address Mark = 1 byte (FE)
ID Area = 4 bytes
CRC = 2 bytes
Post-ID Field Gap Bytes = GAP 2 (22 x 4E)

Data Field Sync  = 12 bytes (00)  + 3 bytes (A1)
Data Address Mark = 1 byte (FB)
Data Area = 512 bytes
CRC = 2 bytes
Post-Data Field Gap Bytes = GAP 3 (80 x 4E)

* - optional, if not present Gap bytes of 4E are typically present.

This article shows the layout in a visual format with remarkable clarity in Fig 4 with the exception of the position of GAP 4B. The box for GAP 4B should be at the other end of the image, just before the red dot identifying the index hole.

II.  Floppy Disk Encoding

A floppy disk is a magnetic storage mechanism.  Data is encoded through a change in the polarity of a magnetic field from north to south and south to north. The direction of the change, called a flux reversal, does not matter. Only the fact that the polarity has changed is relevant. 

A flux reversal, or a flux pulse, takes a certain amount of time as the drive head passes over the surface of a disk. What distinguishes MFM encoding from its predecessor, FM encoding, is the amount of time required for a flux pulse.

A flux reversal could signify a 0 or a 1 bit. We start with the proposition that a 0 is represented by no flux reversal and a 1 is represented by a flux pulse over a specified length of time.  However, this scheme is not workable because it becomes too difficult to keep track of the bits when there is a prolonged period with no flux reversals.  

Thus the Frequency Modulation (FM) method was first used to encode data onto floppy disks flux reversals.  In this method, a 0 bit is encoded by a flux pulse followed by a no flux pulse , and a 1 bit is encoded by two flux pulses. In order to avoid desynchronization, a clock bit is inserted for every data bit, either 1 or 0. FM uses 4μs for timing to encode a single data bit. In other words, in FM you must have a flux bit every 4μs. There are 50,000 bits of 4μs clock periods on an FM-formatted track and with half of those being clock bits, the maximum data capacity of a track is 3,125 bits.

This method is used in the earliest floppy disk controllers, but for home computers only the Atari 8-bit computer disk drives used it.  These disk drives were called single density.  The IBM PC floppy disk controllers that use an NEC uDP765 or 100% compatible like the Intel 8272 support FM encoding.  Later floppy disk controllers lose that functionality.

This system was easy to implement, but not very efficient. The next improvement was the Modified Frequency Modulation (MFM) method, which is what PC disk drives use.  The fundamental distinction of MFM over FM is that fewer bits are needed to encode the same data bytes as FM. The fewer the flux reversals needed, the more data can be stored on a disk.  The disks that supported this system were called double density.  All IBM PC compatible floppy disk controllers use the MFM encoding scheme, as did the earliest hard disk drives. 

In the MFM encoding method a 0 bit preceded by a 0 bit would be encoded by flux reversal then no flux reversal.  A 0 bit preceded by a 1 bit would be two clocks of no flux reversal.  A 1 bit would be no flux pulse followed by a flux pulse.  No more than three clock periods of no flux reversals are permitted at a time with MFM encoding. On average this only requires half the flux reversals per bit as in the FM scheme.  In MFM encoding, there will never be more than three 0 bits between a 1 bit. The effective timing rate for an MFM flux reversal is 2μs versus 4μs for FM flux reversals. MFM still has a clock bit for every data bit, but because there are fewer clock bits required in MFM, the pulses can be half as long than they are in FM. There are 100,000 bits of 2μs clock periods on an MFM-formatted track and with half of those being clock bits, the maximum data capacity of a track is 6,250 bits. MFM could be said that a flux reversal is required every 8μs.

To summarize how flux bits are encoded into FM and MFM bitcells:

FM  = FLUX

0 = 10
1 = 11

MFM = FLUX

0 preceded by 0 = 10
0 preceded by 1 = 00
1 = 01

Let's try to describe the hex value AA, which is binary 1010 1010. 

 1 0 1 0 1 0 1 0 - Data Bits
1110111011101110 - FM Encoding, 12 bits
0100010001000100 - MFM Encoding, 4 bits

The inversion of AA is 55:

 0 1 0 1 0 1 0 1 - Data Bits
1011101110111011 - FM Encoding, 12 bits
?001000100010001 - MFM Encoding, 4-5 bits

Now consider 00:

 0 0 0 0 0 0 0 0 - Data Bits
1010101010101010 - FM Encoding, 8 bits
?010101010101010 - MFM Encoding, 7-8 bits

And at the other extreme, FF:

 1 1 1 1 1 1 1 1 - Data Bits
1111111111111111 - FM Encoding, 16 bits
0101010101010101 - MFM Encoding, 8 bits

The MFM first bit for hex 55 and 00 is unknown because its value depends on the last data bit of the previous byte.

The data lines of a floppy disk drive do not operate in a vacuum. An FM or MFM bitstream includes clocking bits instead of a separate clock line. Normally the two are in sync, meaning that the clock bits correspond exactly to the MFM data bits.  But there is one time when they intentionally are not by design, the Sync bytes.  How this happens is as follows :

The A1 in the Sync bytes does not follow the appropriate clock and data bit pattern. A1 should be encoded in MFM as:

 1 0 1 0 0 0 0 1 - Data Bits
0100010010101001 - MFM Encoding
0100010010001001 - MFM A1 Sync Encoding

A1 is encoded the proper way, as shown on the second line anywhere else an A1 byte might be stored on the track, such as in a Sector Data Field.  However, to put the A1 out of sync, a clocking bit is skipped, as underlined as shown on the third line above. This intentional error notifies the floppy disk controller that a Sync Mark has been identified a Sector ID or a Sector Data field is about to be read. And for purposes of comprehensiveness, C2 Sync in the Index Field looks like this:

 1 1 0 0 0 0 1 0 - Data Bits
0101001010100100 - MFM Encoding
0101001000100100 - MFM C2 Sync Encoding

III.  Copy Protection

Copy protection methods on the IBM PC platform ultimately generally work within this scheme.  Other systems like the Apple II, Commodore 64, Atari ST and Commodore Amiga have controller hardware that allows far more sophisticated protection methods to be used.  The PC disk controller will not tolerate the more bizarre methods like half tracks, fat tracks, spiral streams, offset tracks or varying bitrate speeds.  

The simplest methods are to use non-standard sector sizes, sector numbers or tracks.  This is good enough to bypass DISKCOPY and FORMAT, but any utility that uses the PC BIOS can format a disk with a nonstandard number of tracks, sector sizes or sector numbers.  Each track and sector within the track can have differing sector sizes.  

Another simple method is to use the write protection tab/notch on a disk.  If a program expects the disk to be write protected, it may try to write a destructive sequence of bytes to the disk.  Of course, the original disk should have no tab or notch.  Be wary if you see a disk notched roughly via scissors or with a hole punch.

A common method is to intentionally write a disk with erroneous CRC bytes.  A standard IBM PC Diskette Adapter can read, but not write erroneous CRCs.  Of course, sectors with data that cannot be reliably read will also return CRC errors.  It is then up to the person de-protecting the diskette to figure out whether the error is intentional or unintentional.  

Another method is to check that the exact number of Gap and Sync Bytes are on the disk.  If the original disk was written with 16 of each and the copy only has 15, that is a way to detect the copy.  The program could instead read data from the Gap bytes, like an encryption key.  The PC disk controller's read track command would read the Gap bytes.

A method used the Sync bytes to let sectors overlap.  The disk can play with the sector ID sizes to make no sense physically, like having sector 1 with an ID of 8192 bytes and then fitting the other sectors "within" sector 1.  This protection can often be fooled by writing the image to a high density disk, which allows for 15 or 18 sectors per track.  Of course, high density disk controllers were far from ubiquitous in the 1980s when most of these games were released.

A devious method used weak bits.  In this case, the game would read an area on the disk insufficient flux transitions have been written, making the read coils increase the gain to try to find transitions. If there were sufficiently lengthy periods without a flux transition (greater than three 0 flux bits) the read head would send back random data on different track rotations as the head tried to guess where one bitcell began and the last one ended. The game would not expect the same result to be returned from multiple reads of a sector.  If the same results were always read back, the game would fail to work, thinking it was being run on a copy. Unfortunately, if the disk was damaged unintentionally or the flux transitions have lost their force with age, a flux copier like an Option Board, Catweasel, Kryoflux, Super Card Pro  or a Greaseweazle may misinterpret this and make a bad copy.

IV.  Known Protection Schemes used in Games :

Cops Copylock II (HD Media Supported)

Legend of Mystic Zone, The
Rising Dynasty, The

Electronic Arts IBM Interlock

Used by Electronic Arts booters, involves 96 interleaved sector IDs, typically at track 15, side 0.  They also use 200KB and 400KB disk sizes with 10 sectors per track.

Amnesia
Archon: The Light and the Dark
Chuck Yeager's Advanced Flight Trainer
Chuck Yeager's Advanced Flight Simulator
Hard Hat Mack
Julius Erving vs. Larry Bird Go One-on-One
Marble Madness
Murder on the Zinderneuf
Pinball Construction Set
Seven Cities of Gold, The
Super Boulder Dash
Timothy Leary's Mind Mirror
Will Harvey's Music Construction Set
World Tour Golf

Formaster Copylock

Contains a sector with a Data Field CRC error, usually at at track 20, side 0, sector 5 or track 6, side 0, sector 1. The sector ID is marked as having 256 bytes, but actually has 512 bytes, fooling copiers like TeleDisk. Used mostly by Sierra. This protection will probably fail at speeds greater than an AT (8MHz 80286)

B.C.'s Quest for Tires (Sierra release)
Crossfire (IBM PCjr floppy disk versions)
Gato (versions 1.0-1.4)
Great Escape, The (Thunder Mountain)
Hardball (Disk 2/EGA Disk, & Tandy Release)
King's Quest (IBM PC-CGA booter rev0, rev1)
Mr. Cool
Pinball Wizard
Oil's Well
Quink (Thunder Mountain)
Sierra Championship Boxing (1984 version)
Top Gun (Thunder Mountain rev0)
Troll's Tale
Ultima II (rev0 & PCjr. version)
Ultima III (rev0, rev1, rev2)
Wizard and the Princess, The (IBM PCjr exclusive version)

H.L.S. Duplication

Accolade and Epyx used this type of protection.  There is an additional sector ID field in one of the sectors.

Ancient Land of Ys
Apollo 18
Ajax
Bad Dudes
Boot Camp
Batman - The Caped Crusader
California Games
Crossword Magic 
Destroyer 
Final Assault 
Gauntlet (rev1
Grand Prix Circuit (early version only)
Hardball (Disk 1 only)
Jr. Pac-Man 
Karateka 
L.A. Crackdown 
Mean 18
Mini-Putt
Rack'em
Rush 'n' Attack
RoboCop (Data East)
Serve and Volley
Star Wars (Broderbund)
Street Sports Basketball 
Sub Battle Simulator 
Super Pac-Man (rev0) 
Test Drive
Test Drive II (including Expansion Disks
TKO

Microprose Protection 1

Protection is found on track 4, side 0.  Uses overlapping sectors.

Gunship (certain versions)
Sid Meier's Pirates

Microprose Protection 2

Protection is found on tracks 38-39, side 0.  Uses overlapping sectors.

Airborne Ranger
The Amazing Spider-Man and Captain America in Dr. Doom's Revenge!
F-15 Strike Eagle II
F-19 Stealth Fighter
M1 Tank Platoon
Megatraveller
Microprose Pro Soccer
Sid Meier's Pirates (432.02 and above)
Red Storm Rising
Rick Dangerous
Savage
Stunt Track Racer
Sword of the Samurai
X-Men - Madness in Murderworld

Mindscape DEM

This protection uses weak bits and will probably fail at 386 speeds (16MHz and above)

Blockbuster!
Captain Blood
Contra 
Defender of the Crown (rev0)
Forgotten Worlds
Gauntlet (rev0) 
Infiltrator 2
Italy 1990
Jackal 
Moonwalker
Outrun
Paperboy (Elite)
Paperboy (Mindscape)
Road Runner
Shadowgate
Space Harrier
Street Fighter
Strider
Super Pac-Man (rev1) 
Top Gun (Thunder Mountain rev1)

Minder03
Used by Prism Software

Pipe Dream (European version, U.S. version by LucasFilm Games used a Codewheel)

On-Line Systems Protection #1

This protection will fail in a PCjr. or DMA-less Tandy 1000 and does not work with the 2nd and 3rd IBM PC/XT BIOS.

Crossfire (PC) 
Frogger
Ulysses and the Golden Fleece

Origin Systems OSI-1

Used by all Origin Systems games with an EXE executable.

2400 A.D. 
Bionic Commando
Ultima I (rev1)
Ultima II (Origin Ultima Trilogy Release)
Ultima IV
Ultima V

Atarisoft

Battlezone 
Defender 
Ms. Pac-Man (Atarisoft &  Thunder Mountain)
Stargate 

Datasoft

Uses weak bit protection

Bruce Lee
Dallas Quest, The
Dig Dug (Datasoft) 
Juno First 
Lost Tomb 

Rob Northern Copylock

Uses "strong bits", a long string of 1bits, and calculates the entropy expected.  Encrypts the actual executable and stores a 32-bit decryption number on a bad sector.

Back to the Future Part III
Blade Warrior
Blasteroids  
Blood Money
Borodino
Carrier Command 
Castle Master
Cisco Heat
Frank Bruno's Boxing (Finale)
Kick Off 2
Klax  
Lemmings (1991-07-19)
Oh No! More Lemmings (comes in disk-based and document-based versions)
Lemmings 2: The Tribes (uses High Density disks)
Midwinter
Mighty Bomb Jack
Nebulus
Onslaught
Paperboy (Finale)
Pit Fighter (Domark)
Predator 2
Pro Tennis Tour  
Resolution 101
The Sentinel 
Stormlord
Thunderstrike  
Toobin'  
Weird Dreams  
Xenomorph
Xenon II

Sector ID Duplication

Sierra used this for the versions of their games distributed under Tandy's label prior to 1985.  In this scheme, there are multiple sectors with identical Sector IDs, but different data.  Subsequent read sector commands will read the later sectors.

BC's Quest for Tires (Tandy release)
King's Quest (Tandy version & PCjr./Tandy version)
Wizard of Ids WizType (Tandy release)

Softguard Superlok v3.0 with original loader

Found mostly in Taito games.  Works like Sierra's version, but with a different loader that is harder to decrypt.  The file CMLxxxx.FCL will be present.

Arkanoid
Arkanoid II: Revenge of DOH
Bubble Bobble
Operation Wolf
Puzznic
QIX
Rambo III
Rastan
Renegade
Sky Shark

Softguard Superlok v2.3 with Sierra's Loader

This protection was used in Sierra's games.  It relied on a track with non-standard sector sizes, overlapping sectors, CRC errors in the sector data field   There is a hidden file called CPC.COM on the first disk that checks for the protection.  All these games could be installed to a hard drive, but required the first disk, the key disk, to be inserted in a floppy drive.  Police Quest: In Pursuit of the Death Angel and Mixed Up Mother Goose, were the only AGI version 2 games that never seemed to be protected. This protection was applied to both booter and DOS games but was no found on AGI version 3 games.

3-D Helicopter Simulator (versions 1.00 & 1.10)
Black Cauldron, The (AGI versions 1.1J, 1.1K, Tandy & 2.00)
Donald Duck's Playground
King's Quest I: Quest for the Crown (AGI versions 1.0U & 2.0F)
King's Quest II:  Romancing the Throne (AGI versions 1.0W, 1.1H, Tandy, 2.1 & 2.2)
King's Quest III: To Heir is Human
Leisure Suit Larry in the Land of the Lounge Lizards (AGI version)
Mickey's Space Adventure (3.5" disks only)
Sierra Championship Boxing (1985 version)
Space Quest: The Sarien Encounter (AGI version)
Space Quest II: Vohaul's Revenge
Thexder
Ultima II (rev1)

Weak Bits Generic Implementation

Typically done by using more than three 0 bits following a 1 bit in violation of MFM encoding rules.

Green Beret  
RoboCop (Ocean)
Starglider II

After Burner
Bop'N' Wresting
Harrier Combat
Horror Zombies From The Crypt
Rick Dangerous II
Shinobi

POLA106

Uses weak bits

Ghosts 'n Goblins (rev1)

U.S. Gold

Uses weak bits

Solomon's Key 

Waydisk Minder
This protection will probably fail at greater than IBM AT (8MHz 286) speeds. Most games protected by this protection use weak bits

4 Soccer Simulators
AM, FM Trivia Vol. 2
Arac
Boulder Dash (Prism Leisure, no weak bit)
Boulder Dash 2 (Prism Leisure Release)
Circus Games
Football Manager
Football Manager 2
Frank Bruno's Boxing
Hotshot 
Ninja Rabbits
The King of Chicago
Mayday Squad Heroes
Speedball 
Super Sunday
World Championship Soccer

Xidex Magnetics XEMAG / XELOK

Used mainly by Broderbund.  Has 16 sectors instead of 8 on track 9, side 0.

Bank Street Writer
Championship Lode Runner
F-15 Strike Eagle 
Friendlyware PC Arcade (v3.0) 
GFL Championship Football
Inca 
The Last Ninja
Lode Runner (rev0, rev1, Tandy)
Quink (CBS Software) 
Rampage 
Sargon 3
Spitfire Ace 
Super Hang-On 
Transylvania 
Trivia Fever 
Ultima I (rev0) 
Ultima III (rev3) 
Word Challenge

Titus

Baby Jo in ''Going Home'
The Blues Brothers
Crazy Cars 2 
Dick Tracy
Les Aventures de Moktar (rev1) 
Fire and Forget (Titus Gap Count v1)
Fire and Forget II
Fire and Forget II
Highway Patrol II  
Knight Force
Off Shore Warrior
Panza Kick Boxing  
Pinball Magic 
Prehistorik (Titus Gap Count v1)

RKO

Agent U.S.A

Data Encore

Crime and Punishment 

Sector overlap

Commando (rev0) 
Ghostbusters 
Ikari Warriors  
Might and Magic (rev1)  
Might and Magic II   
Oo-Topos 
Sherlock Holmes - Another Bow 
World Games 

Sonopress

Grand Monster Slam

RPL

Ken Uston's Professional Blackjack
MicroLeague Baseball 

Infogrames

Safari Guns

Insoft

Wordtrix

Miscellaneous Protections

These disks contain unusual but valid MFM Data. Most use simple methods like non-standard sector seizes, which are probably the earliest method of copy protection and can easily be copied with TeleDisk.

Adventure in Serenia
Algebra Arcade
Alpine Tram Ride 
Alleycat 
Arkanoid (Imagine)+Super Tennis 
Below the Root 
Bivouac 
Centipede 
Changes 
Crypto Cube 
Cyrus 
Czorian Siege (requires DMA controller)
Dig Dug (Atarisoft) 
Electric Crayon Deluxe - TMNT - World Tour (v3.0) 
Facemaker (Gold Edition) 
Galaxian 
Gremlins 
Gryzor 
IDSI's Rack 'em Up
In Search of the Most Amazing Thing (v2.0)
Jumpman 
Jungle Hunt 
King's Quest (128k IBM)
Microsoft Flight Simulator (v1.x)
Microsoft Flight Simulator (v2.x) 
Moon Patrol 
Pitstop II 
Planetfall
Sid Meiers' Pirates! v432.03
Sorcerer (r4)
Touchdown Football (IBM) 
Trains (DOS version)
Trap-A-Zoid
Wizardry
Wizardry II
Wizardry III
Wizardry IV
Wizardry V
ZorkQuest II 

V.  Hardware Based Backup Methods

There are several programs that rely purely on the floppy disk controller to try to make a working backup, including Teledisk, ImageDisk, CopyIIPC & Snatchit.  None of them is a perfect solution.  TeleDisk did not take every PC protection method into account, but its great to write images with non-standard sector sizes.  ImageDisk is a newer program and doesn't have that many disk images available for it, but it too is bound by the limitations of the PC disk controller.  CopyIIPC and Snatchit only work on 100% compatible IBM PCs (no Tandy 1000s or PCjr.s), do not work at high 386 speeds or faster and potentially will modify the data on the disk image to crack the program.

The closest thing that was available back in the 1980s that could copy anything was Central Point Software's Deluxe Option Board.  This was a hardware ISA card that intercepted the signals between the disk drive and disk controller.  Because it operated on the flux transition level, it could theoretically copy anything from a double density disk.   It could write bad CRCs, overlapping sectors, interleaved sectors and data in the gap bytes.  This board used a program called TransCopy.  TransCopy could save an image of a disk for future rewriting.  360KB disks would have an image that would be 1,088KB in size.

TransCopy was not without its flaws.  The non-Deluxe Option Board could only work with TransCopy versions 4.x or earlier.  TransCopy versions below 5.x would not copy 80 track disks, inlcuding 720KB disks.  TransCopy images made with version 5.x would not work with earlier TransCopy versions.  TransCopy would also be crippled so that it would not work with certain protections, depending on which company was threatening CPS with litigation at the time.  You would need to hope that an earlier version of TransCopy would work.  Copy Copylock II will not be able to written back with an Option Board or Deluxe Option Board.

The modern equivalent of the TransCopy board are the Kryoflux and SuperCard Pro.  In the PC context, KryoFlux will create raw stream files of each track.  There are utilities that can convert these raw stream files to TransCopy or TeleDisk formats.  It can now write back raw stream files, so you no longer need to send your disk to the SPS and get an IPF file.  However, writing back raw stream files is not fully reliable because it does not account for fluctuations in drive rotation speed and imperfections in the signal, possibly leading to signal loss similar to copying analog tapes.  There are unsupported utilities to make your own IPF files, but it appears that unless you know the stream format well, you may not be able to write back a working disk. SuperCard Pro does the same thing but contains everything in one file.  The two formats require conversion. 

HDMIfy your Old Consoles - Console Specific HDMI Upgrades

If you want high quality input from classic consoles to a modern flat screen TV, you need to use an HDMI converter like the X-RGB Mini Framemeister.  This converter will take high quality RGB or Component video signals and convert them to HDMI-compatible 720p or 1080p with excellent results and minimal lag. I played with a Framemeister some months ago, and wrote up my impressions of the device here : http://nerdlypleasures.blogspot.com/2015/05/fun-with-framemeister.html  Unfortunately, some classic consoles have difficulty displaying high quality video signals due to their lack of high quality video outputs.  Some enterprising individuals have made mod boards that will directly convert the video to HDMI with less lag than a Framemeister, more features and more reliable results.  Here are the projects that, as of the date of this writing, actually have been released in some form to the public.

NES - HiDef NES Mod & the AVS

The NES is one of the most important of the post-crash consoles and one of the few without a native RGB solution.  There has been a NESRGB mod board released for two years now, which works wonders with a Framemeister.  Before that people harvested 2C03 chips from Playchoice-10 PCBs and made do with the differences in the color palette, compatibility issues with color emphasis and sometimes video jailbars.

Kevtris' HiDefNES mod has already been mentioned in this blog, and it adds a whole host of features over the NESRGB + Framemeister solution.  I lay them out in detail here : http://nerdlypleasures.blogspot.com/2015/07/analogue-nt-nes-as-luxury-retro-console.html
It is the board found in the Analogue Nt HDMI version and has been sold in pre-modded systems on ebay and on the http://www.game-tech.us/ website.  The kit will be available for purchase so you can attempt to mod it yourself.  However, it requires desoldering both the CPU and PPU without destroying either chip or the PCB, so it is not a beginner mod.  Virtually all the compatibility issues with the HiDefNES mod and certain MMC5 games and the EverDrive N8 have been eliminated through a firmware update, so now is the time to consider taking the plunge.

The only obvious issue is that its FDS audio emulation is less than perfect.  The mod will not work in an original Famicom, a Famicom Twin or an AV Famicom with laser-marked CPU and PPU chips.

The basic principle of how the HiDefNES obtains the color values of each background and sprite pixel is the same as used in the NESRGB.  See here for my explanation : http://nerdlypleasures.blogspot.com/2015/06/nes-hardware-explained.html
There is now a 2600RGB board for the Atari 2600.  The 2600RGB board uses similar principles to discover the colors of 2600 pixels as the NESRGB board does for NES pixels.  Theoretically, it is quite possible for someone to develop an HDMI mod for the 2600.

bunnyboy, who runs the RetroUSB site, has displayed his AVS replica console at the 2015 Portland Retro Gaming Expo. His AVS has a 72-pin and a 60-pin connector for NES and Famicom games, and is designed as a front loader for the former and a top loader for the latter.  It uses a completely new board and an FPGA chip to provide hardware emulation for the NES.  It only outputs HDMI video and has separate power and reset buttons in the shape of NES front loader buttons.  It has built in four NES controller ports that can be set to function as a NES Four Score or Famicom 4-player adapter.  It also has a 15-pin Famicom expansion port for Famicom peripherals.

Because the console only supports HDMI output, it will not work with any Zapper or compatible Light Gun device and it will not work with the Famicom 3-D Glasses.  The FPGA inside the console does not emulate any cartridge hardware or expansion audio.  Expansion audio from Famicom cartridges is digitized and then mixed with the internal audio.  However, because the AVS uses an FPGA, it can load updated firmware to fix any compatibility issues.  Reports from the Portland Retro Gaming Expo were very positive, one person said the system ran Micro Machines correctly, which is a hard game to get right.  bunnyboy is also designing wireless RF controller without lag to go with the console via an controller port adapter.

The console can only be powered through its USB port.  The console has a scoreboard reporting function via the USB port, it will save high scores for certain known games and report them to NintendoAge.  The idea is that you play a supported game, the high score is saved and then you plug in the system to your PC and it send the score to NintendoAge.  The AVS only supports 720p while the HiDefNES can do either 720p or 1080p.  The AVS is not for sale at the moment, but bunnyboy is aiming for a just under $200 price point.  The best place for updates seems to be here : http://nintendoage.com/forum/messageview.cfm?catid=7&threadid=92557

N64 - Ultra HDMI

No N64 natively outputs RGB, but some early consoles can be modded with a simple amplifier board. Most consoles, however, have a video encoder chip that only outputs composite and s-video.  These consoles require the N64RGB board.  This board acts like a custom DAC, taking the digital signals from the N64 and turning them into analog RGB video.  This will make it truly Framemeister worthy, although the s-video output from the N64 is pretty good compared to composite video.  PAL N64s need a special cable to boost the S-Video signal from the console, NTSC N64s can use standard Nintendo or 3rd Party cables.

Today, if you want to bypass the Framemeister option, you can have an UltraHDMI board installed in your system.  It works with NTSC and PAL framerates (up to 1080p for both) and not unlike the HiDefNES it comes with filters.  The Ultra HDMI filters can simulate a CRT TV.  Without the filters the lag is truly negligible, with them there is about a frame of lag.  It supports all standard digital SD and HD resolutions, as shown here : http://ultrahdmi.retroactive.be/

The mod is very involved because you have to solder a flexible mylar-like ribbon cable to the finely spaced pins of the surface mounted graphics chip, the Reality Co-Processor (RCP).  The RCP sends out 7-bit digital RGB values along with sync information.  This allows for direct digital to digital conversion to the 8-bit RGB values HDMI supports without quality loss.    The kit uses a mini-HDMI cable and only needs a small hole cut into the back of the N64.  Because the mod is very tricky and not for the beginner, the board and kit is being sold only experienced mod kit installers.  If you want your N64 modded with the Ultra HDMI, you need to send it to someone.  This site is authorized to perform the mod : http://www.badassconsoles.com/ultrahdmi/

Game Boy - hdmyboy

In one sense, it is not too difficult to obtain a good quality HDMI image from a Gameboy.  You use a Super Gameboy or a Super Gameboy 2 (for accurate speed) on a SNES that supports RGB and send the video and audio through a Framemeister.  Alternatively, you can use a Game Boy Player and a Gamecube with the component video cables through the Framemeister, but that requires running custom software to make the Game Boy Player output at a true 240p speed.

Some time ago, there was a Kickstarter for a product called the hdmyboy.  The campaign is archived here : https://www.kickstarter.com/projects/507669971/hdmyboy-a-full-hd-power-up-for-your-game-boy-class
This is a fully assembled kit that you insert between the two halves of a DMG-01 Game Boy.  It adds half an inch or so to the height of the console when laid flat. It has an HDMI cable port and can support 720p or (preferred) 1080p.

Unlike the NES and N64 mods, this mod is easy to install.  It requires no soldering.  All you have to do is unscrew the shell, remove the ribbon cable that attaches the LCD PCB to the CPU PCB and insert that ribbon cable into the hdmyboy.  The hdmyboy comes with a NES controller attached internally so you can control your Game Boy.  It essentially becomes a consolized Game Boy at this point.

The Game Boy's LCD can only support four shades of one color, its video signal is fundamentally 2-bit digital monochrome signal with sync information.  This information, along with mono audio, is available through the ribbon cable.  The hdmyboy converts the digital video and the analog audio and sends it through the HDMI cable to the TV.  The hdmyboy only supports mono audio because only a mono audio signal is necessary for the single speaker connected to the LCD PCB.

Unfortunately the Kickstarter campaign was not successful.  However, the hdmyboy team still has some prototypes left here : http://www.hdmyboy.com/play/.  They are very expensive however, (a bit too expensive for what it does) but should represent the best solution to obtain true Game Boy output without using an emulator.  If they try another Kickstarter campaign again, then hopefully more people would be willing to contribute and get the price of this fine idea down.

Given a 1080p display, the hdmyboy can by default do 7x nearest neighbor scaling to give razor sharp graphics at the Game Boy's original aspect ratio.  It can also do 12:7 scaling to give razor sharp widescreen graphics and occupy almost the whole screen if you prefer.  Like the Super Game Boy it has 32 color palettes available to colorize the graphics.  While the Game Boy's sprite and background tiles have 10 valid palette selections to choose from, the Super Game Boy (unless the game is enhanced) and hdmyboy only apply colors based on the actual color value of the outputted pixel.