Sunday, October 16, 2016

Video Potpourri II

Sometimes, one needs to discuss related subjects that do not by themselves warrant a full blog entry.  Hence the video potpourri series, in which I can talk about aspects of display technology.

I.  Composite Artifact Color Emulation

Composite artifact color emulation has been around for a long time.  Unfortunately, in many cases it just isn't very good at trying to replicate the look of a composite monitor.  Simple schemes just assign a color arbitrarily to a group of two or four monochrome pixels.  Mainline SVN DOSBox tries to simulate CGA composite color as if it were a VGA mode.  The colors are reasonably accurate to the real IBM CGA cards, but the text color fringing is rather blocky.  There are custom builds that use filtering and 16-bit color to give a more subtle impression of the composite color, but it still looks much, much cleaner than the real image.  You can obtain a custom build with better composite color emulation here : http://www.vogons.org/viewtopic.php?f=41&t=12319&start=660#p501453




While CGA emulation was late in coming to PC emulators, Apple II emulators have always supported some kind of composite artifact color, because that was the only way most games displayed color.  The most popular Apple II emulator is AppleWin, and as of 1.25.03 it used to behave like DOSBox in that it only really displayed 16 colors.  AIPC and JACE are other emulators that try to emulate color clashing on pixel borders, but it the the striping it adds to the graphics is just too rigid.  Today there is a new release of AppleWin, 1.26.06, that does an admirable job of getting the graphics to look close to how they look on a real composite monitor.  This release represents the incorporation of the development of unofficial patches to improve the NTSC composite color emulation over the very basic emulation of 1.25.03.

AppleWin 1.26.06 represents the pinnacle of composite artifact color emulation available today.  Obtain it here : https://github.com/AppleWin/AppleWin/releases/tag/v1.26.0.6

The Black Cauldron - Old IBM CGA
While other systems like the Atari 8-bit computers and the Tandy CoCo support composite artifact color, their color choices were much more limited compared to IBM CGA and Apple II Double High Resolution graphics.  They also supported direct color modes which did not support artifact color modes.  Their emulators give less sophisticated composite artifact color emulation.

In order to show the differences between real hardware and emulators, I have taken a photo of my IBM PC/XT running The Black Cauldron with an old IBM CGA card.  Although I do not have an Apple II just yet, the composite colors shown by an Apple II and an old IBM CGA card are almost identical.  The Apple II image will look very similar to the IBM CGA image except for the obvious difference of the white vs. black text area underneath the graphics.

DOSBox Mainline SVN
DOSBox Custom Build Composite Color Optimized

AppleWin 1.26.06
JACE (note the incorrect lack of text fringing)
AppleWin 1.25.03
AIPC


II. NES Palette Choices

The NES's PPU was designed in Japan in 1982-1983 and only generates composite video colors, not RGB colors.  The 48 rainbow colors it can display are divided into 12 hues, each with four brightness levels.   The other 16 colors are variations of black, white and gray (4-6 unique choices depending on your TV),  Without special tricks, the NES can display 25 of the 64 colors on the screen at a time.  There are three color tint bits that can emphasize red, green and blue colors, combinations of these bit can give you eight times the colors, but they were not often used in commercial NES games.  The color tint bits will also tint white, gray and black.  For this article I will concentrate on the standard 48 colors.  

Colorimetry, which is the mathematical method to reproduce the colors the human eye can perceive, was first standardized in 1931.  In 1931 TVs were not ready for the market, and only by the end of the decade the first B&W TV sets were being introduced into the market.  WWII put a halt to TV production, but afterward B&W TV took off and development really started to begin on color TV systems,

The first color TV system, NTSC, introduced a color standard.  Original (1953) NTSC colors were very lush and are called the FCC 1953 standard.  Given that 3-strip Technicolor was still the dominant film color technology at the time NTSC was developed, these colors make sense.  Original NTSC TVs tend to give a very Technicolor-like image.  However, over time (by the late 60s), broadcasters were broadcasting less saturated color images to improve picture clarity.  TV display standards took quite a while to catch up to what was being broadcast.  In 1987, the less saturated and more subtle color standard, the SMPTE C standard, was officially adopted.  SMPTE C was the standard that TV manufacturers were moving toward during the 1980s.  However, not every manufacturer conformed rigidly to the SMPTE C specifications before or after the official standardization.  TV manufacturers were more concerned about their TVs standing out on the sales floor than with color accuracy.

(This has not really changed, for example today's TVs tend to show really bright images with the factory presets.  They also tend to have the Smooth or Natural Motion features on by default, which try to turn film images into TV video images by interpolating frames.  The effect is horrible, imagine if every film was made to look like The Hobbit.)

The Famicom and the NES were introduced during this transitional period.  A gamer may have connected his system to a TV made in 1980 or 1990.  Older TVs tend to display color by the FCC standard or some derivative of it, newer TVs by the SMPTE standard.  But for the reasons described above, actual TVs are likely not to rigidly adhere to any standard.  Moreover, Japanese TVs did not really adopt the SMPTE C standard.

The NES outputs an NTSC color signal, made up of phase shifted square waves (for hue) and voltage levels (for saturation and brightness).  It is very far from RGB model used by computers to define color.  Since the beginnings of emulation, emulator users have often wanted a "true" or "accurate" NES palette.  Good emulators like Nestopia allow the user to customize the palette to their tastes. These emulators can import custom palettes in a .pal file consisting of 192 bytes.  Each set of three bytes gives the RGB color value for one of the 64 NES color palette entries.  Outside of software emulation, hardware video upgrades like the NESRGB and the High-Def NES Mod must define their own palettes and allow the user to select from a couple of palette choices.  The NESRGB allows the user to insert his own custom palette in lieu of one of the built-in palettes NES/Famicom FPGA clones like the RetroUSB AVS and the Analogue Nt-mini also need to define their palettes using RGB color values.

Recently, an individual called FirebrandX has been developing his ideal, accurate palette.  His method was to use a composite video capture device, capture each palette color as displayed on the capture and then using averaging obtain the ideal RGB values for each color.  Over time he has tweaked the various entries and the current version, V6, is in high demand.

FirebrandX claims that he is trying to avoid a debate about what person X and person Y saw on their TV 30 years ago by displaying what the NES was intended to display.  In his palettes, the sky in Super Mario Bros has something of a lavender hue.  Some other palettes tend to display it with a sky blue to a bright blue hue.  This is what Nestopia shows with an "FCC Palette" on the left and FirebrandX's V6 palette on the right :


I have two NTSC CRT TVs, and both have controls for contrast, brightness, tint and color.  One TV is a cheap 19" Sylvania with a composite input, the other is a higher-end (for the time) Toshiba with a 27" "flat" tube with a component and an s-video input and three composite inputs.  Both TVs are almost 15 years old.  With the tint control at the middle position, my TVs show Super Mario Bros. sky with a light blue color, no lavender hints to be seen.  I see this color whether I use my Famicom, my front loading NES or my AV Famicom.  Each system has a different PPU revision (E, G & H, respectively).  If I set my TV halfway to the red end of the tint, I will get the lavender tint of FirebrandX's palettes.



(My camera shows somewhat more saturated colors than the tube does to my eyes.)

I also have a RCA Full HDTV LCD from 2008.  Being an RCA TV, it isn't exactly high end, but unlike my 4K TV, it will accept a 240p composite signal.  On this TV, if I set the all the picture control settings to neutral, I will see the lavender sky.  All of the preferences, sports, film, etc. and the default will give a blue hue for the sky.

Now there is historical precedent for both sky blue and lavender sky.  Compare these Japanese commercials for Super Mario Bros./Spartan X and Super Mario Bros. 2/Zelda no Densetsu :

https://www.youtube.com/watch?v=_A1CD4e3_8Q&list=PLtsZRFeG-MhKYFHr2MDnMIIXznW7XYeET&index=134

https://www.youtube.com/watch?v=3KGzcrJ8tDM&list=PLtsZRFeG-MhKYFHr2MDnMIIXznW7XYeET&index=124

You will note that the commercial for Super Mario Bros. gives a sky blue to light blue sky, but the commercial featuring Super Mario Bros 2 for the Famicom Disk System shows more of a lavender sky.   SMB and SMB2 use the same palette entry for the sky.  SMB and Spartan X (Kung Fu) also use the same palette entry for the sky/background.

Here are more contemporary commercials :

Light blue :
https://www.youtube.com/watch?v=jcyslT8U0Pw

Darker blue with a hint of purple :
https://www.youtube.com/watch?v=bzdyp9L7se8

From all this evidence, it is hard to draw a definite conclusion about what the true color was supposed to be.  NTSC systems used to vary quite widely in terms of color reproduction.  Printed material is also somewhat suspect, since most of Nintendo's screenshots (especially from their earlier publication) were taken with a 2C03 PPU.  The 2C03 PPU uses RGB colors, but unless you were playing at an arcade machine or had one of the expensive Sharp Famicom C-1 TVs or later the Sharp Famicom Titler, you would not have experienced it.  The color for the sky on a 2C03 or other RGB PPU has a touch of red in its blue.

The blue color is no less right, however, than the lavender color.  Moreover, if you wanted to ask what SMB's designers, Shigeru Miyamoto and Takashi Tezuka, had in mind, sky blue would be the rational answer.  Their original graph paper layouts for the levels shows that was the color they had in mind and their monitors must have shown it :


In conclusion, FirebrandX's methodology for proving what the "true" NTSC colors are is flawed.  He took his colors based on what his TV capture card reports to be the color.  All capture card come with certain assumptions on how to interpret the color, probably using the SMPTE C standard.  There is no guarantee that the captured colors from the card or device will reflect the colors you actually see from a TV screen.  I believe his capture card interpreted the colors using SMPTE C colorimetry.  

FirebrandX's palette and his justification can be found here :

http://www.firebrandx.com/nespalette.html

The "FCC palette" I used to generate the Nestopia screenshot to the left, above, can be generated by using this site's default setting :

http://drag.wootest.net/misc/palgen.html

10 comments:

  1. You bring an excellent point. My frontloader NES as well as AV Famicom both display true blue sky on all my CRT tube sets. On out LCD TVs, the skys are violet. The bricks are also more brown than red. It's something I've come to accept in recent years, though it shouldn't be. Even Nintendo's Virtual console to an extent does this.

    On my AVS, the FCEU has the bluest sky but the colors are over-saturated. Since all capture card devices use the new standard for color conversions, they will do the same hues as an LCD that we try and avoid.

    Perhaps if the square waves recorded from the NES PPU composite could be processed using a logic analyzer rather than a capture device, the precise timing offsets and voltage levels of the top and bottom waveform could be recorded. These offsets could then be converted to raw Hue-Saturation-Luminosity values using a software algorithm and finally to RGB.

    This process could be repeated 56 times for all color values, and finally a perfect pallet could be generated using said algorithm. For the emphasis bits, careful analysis into how these bits affect the raw output of the PPU would be necessary.

    ReplyDelete
  2. Assuming the PPU output a 50% duty cycle over composite:

    Absolute peak magnitude is Luminosity.
    Relative peak-to-peak magnitude is Saturation.
    Offset is Hue.

    The NES clock crystal is 21Mhz, six times the NTSC colorburst frequency. The NES PPU have 12 distinct hues of four shades excluding grayscale, presumably with evenly spaced offsets. This makes sense as the even divisions would match the native timing of the Crystal. Then the odd palettes could use the rising/falling edge of the crystal clock.

    Black and white voltage levels could be set according to NTSC standards and tweaked from there. I would be interested to see what a "pure" palette would look like based solely on exact measurements retrieved from a logic analyzer. Perhaps a guy like Kevtris could easily use such an approach to generate a flawless palette.

    ReplyDelete
  3. I am sure that several individuals have looked at the NES's video output through an oscilloscope, but the display device will interpret these signals how its video processing technology tells it to. The standards have changed over time, a NES displayed on a 1953 RCA CT-100 will look very different from a 2003 Sony KV-34XBR910. Even though Shigeru Miyamoto chose color $21 for his sky color, he was looking for one with a purple tint. Even on my monitors, $21 has a purple tint when the tint control is in the middle between red and green, compared to $22. $22 is a purer sky blue. If he wanted lavender or violet, he would have said so, but it would have looked rather weird for many people. I am not persuaded that the lavender color of the NES Classic or Firebrandx's palette is what the majority of NES and Famicom players in the 1980s and early 90s saw.

    ReplyDelete
  4. I'm not sure any search is going to be definitive for NTSC folks; those screens always had a tint control which offsets the colour subcarrier phase to account for phase shift during transmission. Which colours you got would depend on exactly where you set that dial.

    ReplyDelete
  5. FYI, FirebrandX has since then update his process, using a calibrated monitor. Also, many consumer decoders are biased, like the sony cxa2025, often touted as a good reference palette has documented decoding bias.

    ReplyDelete
  6. That actually makes 2 of us. But I'm now thinking later to buy second 808 for having both wide pan and close pan simultaneously. I have to play a little with other apps and time-laps to see how 4K plays out on 808. These examples what I aw just blew my mind. I must admit - I just love the dynamic range of 808.

    Not much. Most of the clips are untouched, for some I added contrast."Dancing reeds" part was done byt slowing the motion by 20%



    ReplyDelete
  7. lmao to the first comment lol i agree we don't want a sample, we want the whole thing ;}



    ReplyDelete
  8. Apparently, Miyamoto said that he specifically chose a purplish blue for the sky, so that Firebrand guy was indeed correct after all: https://i.imgur.com/tTRIe5r.jpg

    ReplyDelete
  9. The standard colour palette is defined by the following colours: 585858, 00237C, 0D1099, 300092, 4F006C, 600035, 5C0500, 461800, 272D00, 093E00, 004500, 004106, 003545, 000000, 000000, 000000, A1A1A1, 0B53D7, 3337FE, 6621F7, 9515BE, AC166E, A62721, 864300, 596200, 2D7A00, 0C8500, 007F2A, 006D85, 000000, 000000, 000000, FFFFFF, 51A5FE, 8084FE, BC6AFE, F15BFE, FE5EC4, FE7269, E19321, ADB600, 79D300, 51DF21, 3AD974, 39C3DF, 424242, 000000, 000000, FFFFFF, B5D9FE, CACAFE, E3BEFE, F9B8FE, FEBAE7, FEC3BC, F4D199, DEE086, C6EC87, B2F29D, A7F0C3, A8E7F0, ACACAC, 000000, 000000.

    ReplyDelete
  10. For what it is worth, the lavender color in the Nintendo's palette is actually outside the NTSC color gamut(as are several other entries). There is not, and CANNOT BE, a canonical correct rendering other than "what I saw on the screen"

    ReplyDelete