Thursday, June 18, 2015

Cleaning Saved Information from Famicom Disk System Game Dumps

A lot of Famicom Disk System games save some information to disk.   Often this information may be custom levels designed with an in-game level editor, the top scores or the fastest times or the levels beaten.  The more ambitious adventure and RPG games would typically offer three save slots to store the player's progress during the game.  The use of magnetic media, the Disk Cards, allowed any Famicom Disk System game to save information to the disk.  When games are dumped, any information saved will come with them. While this will not diminish the playability of a game, it still cosmetically very unappealing to see SHITHEAD all over the high score lists or hacked games on the file select screen.

Unfortunately, for most games there is no way to restore the game to a pristine, never-played state within the game itself. I have always wanted a tool or utility that could take these disk images and do just that, but since none exists I decided to investigate the images and try to fix them myself. Restoring a Famicom Disk System game to a pristine state is often times more involved than just deleting or zeroing out the last file. The resulting disk image has to work after it has been "cleansed". I used Nestopia UE 1.46 and FDSExplorer 1.63 and my favorite hex editor.  A hex to decimal calculator will also come in handy.

Famicom Disk System games come on one or two disks.  A game on a single disk can use one or both sides of the disk, two disk games always use four sides.  The sides are labeled Side A and Side B.  The Disk System's drive was single sided, so the user had to physically eject the disk, remove it from the drive, turn it over and insert the disk.  The disk drive can automatically sense when a disk has been inserted and inform the rest of the system.  

A straight dump of each side of a Famicom Disk Card will have a size of exactly 65500 bytes.  Each subsequent disk side dump is appended to the previous side, so two disk side games will be 131,000 and four disk side games 262,000.   Modern Famicom Disk System game dumps use an .fds file extension.  They can come with an extra 16-byte header for emulators and flash carts.  This header looks like this :

46 44 53 1A 0x 00 00 00 00 00 00 00 00 00 00 00

x = number of disk sides, typically 1, 2 or 4.

The first three bytes show FDS in ASCII and the fourth byte is also used as the MS-DOS End of File byte.  The header is extremely basic and is designed to inform an emulator that the file loaded is a true FDS disk image and how many disk sides the game has.  While this could have been discerned by the file extension and file size, the FDS image format was created in the days of Nesticle, an MS-DOS based NES emulator.  MS-DOS files used 8.3 character/extension file names, so the likelihood that files could be confused was much greater.

Famicom Disk System games have a simple file system.  Each file on the disk has a name using up to eight characters, a file number (0-x), an ID number (ditto), a size (up to 32KB) and a type.  The three types are Nametable, CHR and PRG.  CHR stores graphics tiles, PRG stores code and the Nametable is always the first file on Side A and is used for the Copyright text at the beginning.  If the text is not exact, the disk will load no further.  Save games are always stored in a PRG type file.  

What made this much easier was the fact that Famicom Disk System games always seem to save to the last file on a disk's side. Later in the Disk System's life, (1988) Nintendo used a new drive controller chip that limited the disk drive to writing only to the last file on a disk. The earlier convention became a rule. 

Sometimes the save file has an obvious name and is so small it could be nothing else, but for other games it is not so obvious. So the first thing I had to do was to observe which side was being written. Nestopia tells you when a disk is being read or written, so I just played until there was a save. Metroid was almost unique among the games I looked at because it saves information to both sides. 

Once I figured out which side was being saved, I opened the disk image in FDSExplorer and looked at the file. For games with three save slots, I simply deleted two and made a third with a file name that was easy to find like ABCDEFGH. With a little deductive reasoning (i.e. looking at the hex patterns and logical reasoning) I could figure out how the file could be corrected so that no save games would be present at startup. Dr. Chaos did not follow the typical rule about writing changes to all files after starting a new game, but its save file is really simple and my image came with one unused file.

On a couple of occasions, I discovered that my image was pristine but the game actually created a file once there was saving. FDSExplorer makes it easy to delete the last file in these cases, but you have to verify that the game creates a file before you delete it. I made it a point to alter no more bytes than was necessary to make the game appear to look pristine. Whether it would actually correspond byte by byte to a pristine, error-less dump of a sealed Disk Card is an issue that may never be known for some games, but for other games the changes are so obvious and minor that it can hardly be doubted that this is how they should have appeared. 

Nintendo, for its own first-party games, eventually began to store a backup of a pristine save file elsewhere on the disk.  That way, if the save file became corrupted after a write, the game could try to restore the save file so the user could continue to play the game.  Hikari Shinwa - Palutena no Kagami is an example of such a game.

Some games will allow you to restore the game to a pristine state via an in-game function.  They are detailed below.

Here is a list of the games and what I did to make them "pure". 

Akumajou Dracula (all versions) (Konami, 1986)
Information Saved - Three slots, last stage completed and number of continues
Fix : Side A - Zero out last file
NES/Famicom Port : Castlevania/Akumajou Dracula

All Night Nippon Super Mario Bros. (Nintendo 1986)
Information Saved - Number of times you have beaten the game
Fix : Side A - Zero out last file
NES/Famicom Original : Super Mario Bros.

Bubble Bobble (Taito, 1987)
Information Saved - Allows you to select a starting level up to the last level you completed
Fix : Side A - Game creates a save file when you save your level progress, delete file with FDS Explorer
NES Port : Bubble Bobble

Clu Clu Land - Welcome to New Clu Clu Land (Nintendo, 1992)
Information Saved - High scores
Fix : Side A - In the last file, this pattern must repeat ten times : 14 EF EF EF 24 00 00 00
NES/Famicom Connection : Clu Clu Land

Dr. Chaos (Pony Canon, 1987)
Information Saved - Three slots, progress through game
Fix : Side A - Zero out last file, except for the last eight bytes, should say DR.CHAOS in ASCII
NES Port : Dr. Chaos (NTSC Only)

Dracula II - Noroi no Fuuin (Konami, 1987)
Information Saved - Allows you to select a starting level up to the last level you completedtime taken in game
Fix : Side A - Zero out last file
NES Port : Castlevania II : Simon's Quest

Eggerland (HAL Laboratory, 1987)
Information Saved - Rooms completed
Fix : Side B - Start a new game, commit suicide until you see the Game Over screen, then save.
NES/Famicom Connection : Adventures of Lolo 1, 2 & 3, Eggerland - Meikyuu no Fukkatsu

Eggerland - Souzouhe no Tabidachi (HAL Laboratory, 1988)
Information Saved - Allows you to select a starting level up to the last level you completed, saves levels you build with the construction editor in the game.
Fix : Side B - Zero out the last byte in the last file and remove all tiles for all construction levels using the in-game utility, then save.
NES/Famicom Connection : Adventures of Lolo 1, 2 & 3, Eggerland - Meikyuu no Fukkatsu

Hao-Kun no Fushigina Tabi (DOG, 1987)
Information Saved - Three slots, progress through game
Fix : Side A - Delete all three save files in the main menu, then create file one. Immediately close the FDS image in Nestopia. Replace the first seven bytes in the last file with 00 5F 5F 5F 5F 5F 5F.
NES Port : Mystery Quest (NTSC Only)

Hikari Shinwa - Palutena no Kagami (Nintendo, 1986)
Information Saved - Three slots, progress through game, time taken to beat game, high scores
Fix : Side A - Copy hex from REG BACK and paste into REG.SAVE
NES Port : Kid Icarus

Kaettekita Mario Bros. (Nintendo, 1988)
Information Saved - Name, age and gender of players controlling Mario and Luigi, High Scores
Fix : Side A - Rename characters to the default, use hiragana for Mario and Luigi, set the ages to 00 and the gender to male (おとこ). Mario is まりお and Luigi is るいーじ in Nintendo's preferred hiragana spelling.
Side B - Copy over all bytes in file 9 to file 10
NES Port : Mario Bros. Classic Series (PAL Only)
NES/Famicom Connection : Mario Bros.

Knight Move (Nintendo, 1990)
Information Saved - High Scores
Fix : Side A - Save file is split into four sections of 48 bytes. The game uses the first and last section for saving the scores, so copy the second or third section to the first and fourth section. 
NES/Famicom Connection : Tetris/Tetris (designed by Alexey Pajitnov)

Legend of Zelda 2, The - Link no Bouken (Nintendo, 1987)
Information Saved - Three slots, progress through game, number of deaths
Fix : Side A - Kill all save files on the file select screen, the game will reset each slot as after it is killed.
NES Port : Zelda II -  The Adventure of Link

Metroid (all versions) (Nintendo, 1986)
Information Saved - Three slots, progress through game, time taken in game, ending earned for last game.
Fix : Side B - Create a new character for file one, then start the game. Immediately close the FDS image in Nestopia. Do the same for file two and three. Do NOT save the game. This causes the game to reset the save information for the world for the three files because the game will write to side B when you create a new character.  Side A - Overwrite the first three bytes with 80 and the next forty eight bytes with FF. Zero out the rest of the file. This deletes character information.
NES Port : Metroid

Nazo no Kabe - Block Kuzushi (Konami, 1986)
Information Saved - Allows you to select a starting level up to the last level you completed
Fix : Side A - Delete all three save files, then create file one. Start the game, lose all your balls, then save and quit. In the last file, replace the first sixteen bytes with the second sixteen bytes. These two sets of bytes should be identical to the third set of sixteen bytes. The last sixteen bytes of the file should be 00.
NES Port : Crackout (PAL Only)

Nazo no Murasamejou (Nintendo, 1986)
Information Saved - Three slots, progress through game
Fix : Side A - Kill all save files on the file select screen, the game will reset each slot as after it is killed.
NES/Famicom Connection : The Legend of Zelda/Zelda no Densetsu 1 - The Hyrule Fantasy (engine)

Section-Z (Capcom, 1987)
Information Saved - Three slots, progress through game
Fix : Side A - Delete all three character saves in the game menu, then create file one. Start the game, get killed, then save and quit. In the last file, replace the first twenty three bytes with the second twenty three bytes. These two sets of bytes should be identical to the third set of twenty three bytes. The last four bytes of the file should be 00.
NES Port : Section Z

Super Mario Bros. 2 (Nintendo 1986)
Information Saved - Number of times you have beaten the game
Fix : Side A - Zero out last file
NES/Famicom Connection : Super Mario Bros. 1, 3

Vs. Excitebike (Nintendo, 1988)
Information Saved - Race times and data for custom tracks 4-7 designed with built in construction editor.
Fix : Side A - Delete the last file
NES/Famicom Connection : Excitebike

Winter Games (Pony Canon, 1987)
Information Saved - Records for each event
Fix : Side B - Zero out last file
NES Port : Winter Games (NTSC Only)

Wrecking Crew (Nintendo, 1989)
Information Saved - Custom level designs for phases 1-4 designed with built in construction editor.
Fix : Side A - Delete all objects on each of the four phases ofthe design part, then save.
NES/Famicom Original : Wrecking Crew

Yume Koujou Doki Doki Panic (Nintendo/Fuji Television, 1987)
Information Saved - Worlds completed by each character.
Fix : Side A - Zero out first four bytes in last file
NES/Famicom Port : Super Mario Bros. 2/Super Mario Bros. USA

Zelda no Densetsu - The Hyrule Fantasy (Prototype and final) (Nintendo, 1986)
Information Saved - Three slots, progress through game, second quest, number of deaths
Fix : Side B - Kill all save files on the file select screen, the game will reset each slot as after it is killed.
NES/Famicom Port : The Legend of Zelda/Zelda no Densetsu 1 - The Hyrule Fantasy

Further notes :

There are many, many other FDS games that save information to disk.  I have only given a sample of the games which either have a NES port, are part of an NES series or substantially related to an NES game. Many Famicom Disk System games are Japanese-text heavy, which can make it more difficult to purify the image.

Nestopia does not save to the disk image by default, but can in its options menu. You should also change the option so that you can see the status Reading Disk/Writing Disk in Nestopia's status bar.  Always make backups when experimenting. Nestopia saves to an image when the image is closed or the user quits Nestopia. 

Unfortunately, FDSExplorer does not allow you to edit files inside a disk image directly, it is easiest to use a Hex Editor and search for the bytes you wish to alter. 

This list really only scratches the surface, but I wanted to focus on games that were ported in some way or form on or related to a NES cartridge. Since my Japanese is "a little rusty", I did not want to tackle the more Japanese text-heavy games.

2 comments:

bleuge* said...

this a very interesting idea i've toyed many times (and i suppose a lot of NES emu lovers)

years ago someone tried to created a redumping project (maybe the people in no-intro, i can't remember)

as you know find FDS games in pristine conditions is very hard (i mean never played, so the disk has never been writeen and is as exit from factory), but i think this is a thing sooner or later must be done

as we are talking about fds preserving, maybe the unit could be used with Kryoflux or SCP?
someone must be tried this...

keep the good articles man! i follow you via feedly!

Great Hierophant said...

The No-Intro group are generally to be commended when it comes to ROM dumping, but when it comes to NES and Famicom cartridge and disk dumps, their previous policy on headerless dumps caused no amount of difficulties for emulators and flash carts, which often rely on this crucial (at least for cartridges) data.

The FDS format is not perfect by any means. FDS-format disk dumps use a very simple file system where data is organized into blocks. There are at least eight blocks on any disk and the FDS format captures only that data. The resulting dump is essentially equivalent to an MS-DOS disk sector dump. No in-between data like GAP bytes and block start marks or error correction data like CRC bytes is captured. Also, depending on the dumping hardware, the dump may not be complete as in "hidden files" (files with file numbers above the file count number written to the disk) are not dumped.

Getting a Kyroflux or SCP to work with a Quick Disk drive, which is not like a typical 3.5" or 5.25" PC drive, would be a very interesting project, but not a simple one.