Wednesday, February 6, 2019

Proper Analog Retro Video Capture with the Datapath E1/E1s

Capturing analog video can be a difficult task.  Analog video follows rather imprecise standards and is increasingly being discarded in today's world where 100% digital video solutions like HDMI and DisplayPort rule.  Capturing a digital signal is often simply a matter of buying a capture card/box and plugging everything in.  But capturing audio signals, at least those signals that do not conform to the "broadcast standards of 525/625i", is not quite so easy.  But while there exist inexpensive devices that can handle low quality composite and medium quality s-video sources, what about high-end analog sources like component video, 15KHz RGB and 31KHz VGA signals?  Moreover, are any of them compatible with 240p signals put out by retro consoles and home computers?  While there are affordable devices that can sort of handle these signals like the Startech USB3HDCAP, the results are often second rate.  But what if there was a device that you can acquire for similar cost and provide truly first-rate capture?  Interested?  Well if you are, read on to discover the power and the caveats of the Datapath VisionRGB E1 and E1s.


Introducing Datapath and the E1/E1s

Datapath is a U.K. company that makes high-end video capture devices and wall display controllers.  They are not sold in stores, if you wish to purchase their products, you will have to deal with a sales representative.  The subject of this blog entry are its VisionRGB E1 and VisionRGB E1s PCI-e cards.  When they were released, these were very expensive cards, probably retailing for several hundred dollars each.  However, the E1 appears to have been discontinued and the E1s is the least featured product in Datapath's page showing its current video capture products.  As these cards are now being retired and replaced by the businesses and institutions which were able to afford them when new, they tend to show up on eBay for very reasonable prices.

The Datapath E1 and E1s are PCI-e cards requiring four PCI-e lanes and include one DVI connector supporting single-link DVI.  You will need to check that your motherboard not only has a 4x/16x slot available but you have enough lanes for the card after your video card(s) have taken their lanes . This connector will accept HDMI video signals with an adapter but it does not accept audio signals.  To record any kind audio you will need another device like the Line In on your motherboard's PC audio hardware or a USB audio device with an SPDIF input.  You will also need to strip any HDCP out of the signal as found on devices like the PlayStation 3.

The Datapath VisionRGB E1 and E1s can also capture DVI (up to 1920x1200), RGB/VGA (up to 2048x1536) and component video (up to 1080/60p).  They cannot capture composite or S-Video signals, you need their VisionSD cards for that.  The difference between the E1 and E1s is the amount of bandwidth they provide for video capture.  The E1 can transfer data to the host at 480MB/s while the E1s can transfer at 650MB/s.  I will discuss the practical difference later.

The Datapath E1 and E1s have several strengths for capturing analog sources.  They can capture in 24-bit RGB, so there will be no color loss or pixel blending as there is with cards that can only capture in nothing better than YUV 4:2:2.  They can accept horizontal sync signals down to 15KHz, which is necessary for capturing any retro video game console's output pre-Dreamcast/6th Gen.  This 15KHz capability is also necessary for capturing pre-VGA computer video output.  It also has plenty of bandwidth for a native 70Hz 320x200 VGA capture.  The Startech card I was using had nothing but issues with 70Hz.  Finally, the horizontal pixels can look very sharp and evenly sized due to the control the card's software provides you over the horizontal pixel capture.

The Datapath is only a DVI card, it cannot accept the input from an HDMI device that cannot send RGB video out when it detects being connected to the less-featured connector.  If your HDMI device only supports YCbCr output, either in 4:2:2: or 4:4:4, the resulting image on the Datapath will not look right at all.  Fortunately, the HDMI standard requires a source to configure it's color output to a format that the sink can accept, so if your sink is the Datapath and the device can output RGB, then you will be fine.

The Datapath accepts Separate Sync, Composite Sync and Sync on Green.  Console RGB sources typically output composite sync, but the Datapath will only accept TTL Sync, not 75 Ohm Sync, Sync-on-Luma or Composite Video Sync.  Computer display adapters (including Atari ST, Amiga and Apple //gs) almost invariably output TTL sync signals.  The Analogue Nt Mini outputs Separate Sync at TTL levels and Composite Sync at 75 Ohm levels, so capture that console's analog output via Separate Sync or Sync on Green.  Look here to see what kind of sync is available for your console : https://www.retrorgb.com/csync.html

Buying a Datapath Card - Where and What to Buy?

Prices on the Datapath cards on the second-hand market can fluctuate.  When their strength in capturing non-standard retro video sources was less well-known, they could sell for $70-80.  However now you will likely spend upwards of $100 to acquire one.  The E1 is usually priced a little less than the E1s.  The E1 will be quite sufficient for most uses, but it lacks the bandwidth to support two programs accepting video from it at 1080p at the same time.  So if you are using OBS to stream and FFMPEG to record video at 1080p, you will have to invest in an E1s.

Before you buy, be aware that these cards came sold standard with short card retention brackets.  The short brackets are designed for the low profile enclosure of a business workstation that your company would typically buy from Dell, HP or Lenovo.  They don't fit inside an ordinary PC case which uses a standard length bracket.  Datapath cards can be found on eBay with a long bracket, but those brackets command a premium.

My card came with a short bracket, but for $100 I was willing to look for an alternative bracket solution.  Unfortunately generic metal brackets do not work with the placement of the DVI connector on the Datapath cards.  I do not have access to a metal puncher or drill.  Fortunately one generous individual designed a 3-D printed bracket to function as a standard length bracket for the Datapath cards.  You can find it here : https://www.thingiverse.com/thing:2960059

Having long sought a Datapath card, I checked eBay for the available options.  The cheapest E1 card with a full-size bracket I could find was going for $132.99.  However there was one eBay seller who was selling a short bracket E1s for $100.  Was the hassle of obtaining the bracket worth the price on the higher-bandwidth card?

I remembered that my local UPS store had acquired a 3-D printer some time ago, so I sent the thingverse project file to them and asked how much it would cost for a bracket.  They replied $3.87 per bracket.  At that point I figured the 3-D printing route was worth taking a risk on the $100 card and I acquired the E1s.

When I got my E1s in the mail, I did not notice anything unusual about its short bracket at first.  Then I picked up my 3-D printed standard length bracket from the UPS store and when I compared it to the short bracket, I noticed that the notch for the retaining screw was on the opposite edge of the bracket!  Needless to say I was not happy, but I figured I could drill or dremel out a hole on the "right edge" of the bracket.  Before I did that, I decided to fit it in my PC anyway to make sure the fit was at least right.  I removed the screws for the DVI connector with a pair of pliers, inserted the 3-D printed connector and reattached the bracket to the DVI connector with the screws.  Imagine my delight when I saw that the notch lined up with the screw hole for the retaining screw perfectly.  I did not know that the short bracket had its notch opposite to the standard bracket.

Now you can print the bracket in metal, see here for more details.

Conquering the Datapath Drivers

By the time my Datapath had been delivered to my mailbox, I had downloaded the latest driver package.  At the time of this blog entry, that package is VisionInstall v7.19.0.  The first thing the driver install did when I ran it was to spontaneously reset my system!  The second run got the installer to update the firmware on my card and then finished installing the driver and the Vision program.

When I finished installing the drivers, I started up the Vision program and saw an image from the device I plugged into it.  Vision is an image viewing program, it will try to detect the resolution and refresh rate of the image but it allows the user to adjust the parameters of capture window.  Changes to the parameters can be viewed in the window, which will show the image Vision is trying to capture.  It can take screenshots of individual frames but cannot capture continuous video.

I used an HDMI device in order to ease my way into things.  I saw the image I was expecting and then I opened up OBS to try to capture it.  However, I did not see the Datapath listed in the devices I could use in the Video Capture function.  Next I tried VirtualDub's capturing program, which the Vision program's Help function specifically mentions as working as a capturing solution.  I could not select the Datapath card that program either.

Fortunately I am not the first to have walked the Datapath, so I sought help from the R3 RGB Retro Resource Discord server.  That server is dedicated to video capture, and one of the mods on that server told me that you must install the Datapath drivers in Kernel Mode in order to see it as a capture device in capturing programs like OBS and VirtualDub.  By default it installs in User Mode, which requires a program to specifically support the Datapath. I reinstalled, then did an uninstall/reinstall, but I could not see any option during the install to select Kernel Mode.  I even saw a picture of the box that allowed you to choose between Kernel and User Mode, but my installer showed it not.

Finally, I decided to drop back to the driver revision previous, v7.17.14, to the current driver revision.  That installer gave the option to select between Kernel Mode and User Mode.  After I installed the driver in Kernel Mode, then I could see the Datapath listed as a capture device in OBS and VirtualDub.  Capturing of digital sources worked, so I saw no need to go further back in the driver's past.

Balancing Vision and VCS

Super Mario Bros. - NES

The Datapath Vision program is designed to adjust the image settings for the Datapath card.  There is another program, however, called VCS, that performs a similar function but has its own strengths and is easier to use.  The current version, v1.2.5, is here : http://tarpeeksihyvaesoft.com/soft/  These guys also did a patch to eliminate the automatic CPU cache-renabling functions within Ultima VII, so they are pretty cool.  If you want to be able to get pixel-perfect captures, you really need both programs to fine-tune your settings to eliminate unsightly artifacts.

Super Mario Bros. - NES
To use a Datapath ideally, you must know the resolution you are trying to capture.  The NES is 256x240, IBM CGA is 320x200 and an Atari 2600 is typically 160x192.  Set the resolution you are trying to capture in Vision or VCS.  VCS has preset common resolutions you can select rather than typing in numbers. Here are some examples of resolutions you should use for common retro devices :

Atari w/ 2600RGB : 320x240
IBM CGA, PCjr. Tandy 1000 : 320x240, 640x240
Colecovision, NES w/NESRGB, SMS : 256x240
SNES : 256x240, 512x240
Genesis : 256x240 or 320x240
TG-16 : 256x240, 320x240, 336x240, 512x240

The Datapath is very good when it comes to counting scanlines, but sometimes it is not perfect.  Sometimes it sees 250 scanlines, which is found nowhere because the device you wish to capture is outputting more than 240 visible lines.  All resolutions that use less than 240 lines should be set to 240 scanlines, whether it is a Sega Master System (192 visible lines), an IBM EGA card (200 visible lines) or a SNES/Genesis (224 visible lines).  If you are capturing a 240p NTSC source, set the Datapath's resolution to 240 lines regardless of the number of active lines being output.  You can crop out anything displayed in the non-visible area with Avisynth later.

Phantasy Star - Sega Master System
The real tricky part about using a Datapath is that you must know or try to identify how many horizontal pixels a video source is actually outputting, not just the visible pixels.  For example, the NES has 256 active pixels but really has 341 pixel clocks per line.  Similarly, an IBM CGA card may display 320 active pixels but is really pumping out 456 pixels per line. Once you have that number (in decimal format), then you should be able to get a pixel perfect image.

You can find some of the more commonly captured console widths here (round up to next pixel for fractional pixel counts, and look to the sites linked too) : http://r3.fyi/Main/HomePage  That wiki has a lot of useful setup information for Vision.

You may see one or more shimmering columns of pixels in your capture at first.  This is normal and will be reduced to one column by the time you get the right number of horizontal pixels.  Once you are down to one you can adjust the phase to move that shimmering column off by one edge of the screen.  Track it as it leaves the screen on one edge, because it will reappear on the other edge if you still have room to play with on the phase slider.  Some video sources will show "mosquito noise" in areas of solid color, you may be able to mitigate this by applying a low pass filter, the Datapath cards do not have one.

There are times and signals that will display well in Vision but look corrupt in VCS and vice versa.  When that happens you may need to manually set the resolution and horizontal parameters in the program you wish to use.  VCS has presets you can save and automatically when it detects a video mode it recognizes, but it is not infallible.

When it comes to 240p analog sources, VirtualDub and OBS had great difficulty the signal properly for me using the Kernel Mode driver.  I used the latest driver which gave the Kernel Mode driver installation option,  v7.17.14, and I never got the same quality of captures or anywhere near it as I did from Vision or VCS.  What I did was to capture the Window using OBS, and at the native resolution of these consoles, the burden on my CPU was not particularly onerous.  For a 1080p HDMI source, this did not work, I needed the direct Video Capture provided by the Kernel Mode drivers to work to get a capture without excessive CPU usage.

The Vision software supports YUV Rec. 709, Rec. 601 and Rec. 2020, all in full and limited color range.  Component video sources will typically use Rec. 601, which is more of an SD standard whereas Rec. 709 is HD-preferred and should be used with HDMI.  Rec. 709 does not show substantially more color gamut than Rec. 609, but the UHDTV Rec. 2020 shows far more colors than either of the three-digit standards.  Computer output sources and all video game consoles prior to the PlayStation 2 use RGB and should be set to Rec. 709 in Vision.

My advice is to always use VCS when tweaking your captures and only use Vision when you need to take a screenshot.  You can use the scroll wheel on your mouse to enlarge the capture window.  You can also Ctrl + Left Click on one of the Force input resolution buttons to enter your own resolution select.  You can also save and load your pixel sizing settings to a file.  VCS also allows you to resize your window without preserving the aspect ratio, so those 640x200 modes need not look ridiculously squashed.

Capturing IBM PC Compatible Video

IBM CGA 320x200 40-Column Text Mode (doubled to 640x400)
Unlike most video game consoles, PCs tend to change resolutions frequently.  However with the Datapath getting high quality DOS captures is quite possible.  The early color adapters, CGA, PCjr., Tandy 1000 and EGA in 200-line mode are all 240p friendly devices.  However, the Datapath cannot deal with their digital RGBI signal directly, you must convert it to analog RGB, preferably with a converter like the GGLabs CGA2RGB.  You connect a straight through 9-pin male to female cable to the CGA2RGB, then connect the 15-pin port to a VGA-to-DVI adapter.  Make sure to give the CGA2RGB a +5v power source and set it for separate sync. 

King's Quest I - IBM PCjr.
King's Quest I - IBM PC CGA 320x200













CGA/EGA/TGA/PCjr use 40 and 80-column text modes that have pixel resolutions of 320x200 and 640x200, respectively.  Tandy 1000s use 225-line modes by default, but can be changed to 200-line modes using the DOS MODE 200 command by pressing F2 for the earlier models on startup.  All use 320x200 and 640x200 graphics modes and the Tandy/PCjr also uses a 160x200 graphics mode.


IBM PC CGA 640x200 80-Column Text Mode

I have discovered that the easiest way to deal with the differing graphics mode is to treat them all as 640x200.  You might observe that most games use 320x200, but we can fix the aspect ratio in VCS.  The first thing you should do is to set one of the Force input resolution buttons to 640x200.  Next, on the Output tab, set the Base resolution to 640x400.  Uncheck the Relative scale box.  Ensure your Upscaler and Downscaler menus are set to Nearest and your Color depth is 24-bit (888).  Finally, go back to the Input tab, click on the Adjust video & color button and enter the following settings :

Horizontal size: 912
Horizontal position: 180
Vertical position: 37
Phase: 0
Black level: 8

Leave the Brightness and Contrast settings at their midpoint, 32 and 128, respectively for overall and 128 and 256 for each color.

There is a feature called Alias resolutions which automatically forces a detected input resolution to scale to a set desired output resolution using VCS.  You essentially give VCS a list that says "treat 320x200 as 640x400", "treat 640x200 as 640x400" and so on.  The benefit to you is that you don't have to enter the settings for each mode manually whenever the mode changes and need not show a partially-corrupted graphics screen until you change the mode back to the ideal.  Your 320x200 game images will automatically be scaled, nearest-neighbor to 640x400 and your text modes with an effective resolution of 640x200 will not be squashed.  However, the readme is incorrect in how you setup your alias file.  The readme tells you to do this to tell VCS to "set 320x240 as 720x240" :

320 240 720 240

However, the line needs to look like this :

320,240,720,240

When you have done all these steps, you should see text or graphics in a normal aspect ratio and they should look sharp.  If you see swirly graphic columns, adjust the Phase.  If the screen looks cut off, adjust the Horizontal and Vertical position.  The Vertical position is known to adjust itself without reason at resolution changes, so you may want to Force a 640x240 resolution, which eliminates this issue.

King's Quest I - IBM PC CGA with Borders
If you wish to see border colors, then Force a resolution of 720x240, set the window to 720x480 and set the Horizontal and Vertical position to 138 and 14, respectively.  Border colors in 640x200 modes are not uniform, which is a limitation of the Datapath.  The issue is that the Datapath cannot detect the short horizontal sync pulse properly in the 640x200 text modes.  To fix the issue you have to use a TSR like TVCGAFIX, found here, that can lengthen the horizontal sync of the MC6845 CRTC found on the CGA card.  This should fix your border issues in standard text mode.  For 160x100 games, you have to hack each game.  Compare the unfixed Round 42 with a fixed version :

Round 42 - Unfixed
Round 42 - Fixed
When it comes to VGA, things get a little easier.  Most VGA games use 320x200 graphics, but because VGA is double scanned, you will really be dealing with 640x400.  Some VGA games use 320x240 and 640x480, but there are Force input resolution buttons for those.  One small wrinkle is that VGA Text defaults to a 720x400 pixel resolution, but there is a Force input resolution button for that too.  Here are some settings for common VGA resolutions :

320x200 = 800, 148, 36
640x350 = 800, 146, 59
720x400 = 900, 162, 36, 22
320x200 with borders = 670x414, 800, 130, 29
640x480 = 832, 170, 30

Commander Keen - VGA 320x200 with Borders (double-scanned)
Capturing GameCube Component Video

When I said that you should avoid using Vision and use VCS instead, that advice can't quite remain when it comes to the GameCube and Wii.  The GameCube was the second console after the PlayStation 2 to support Component video by using a cable that converted a digital YCbCr to analog YPbPr from a digital output connector.  The GameCube outputs video in the YUV 4:2:2 format which is then converted to Component or, in PAL consoles, RGB.  The GameCube's native resolution is 720x480 with a 858 horizontal size.  GameCube games often use a resolution of or close to 640x480, but if you want pixel-perfect capture, you should look here for the actual game's resolution.

If your game supports a widescreen mode, the only thing you need to select Maintain Aspect Ratio in Vision and set the Aspect Ratio boxes to 16:9.

Here is where Vision come into play.  Vision has an option to change the color domain used and supports RGB or YUV in Rec. 601, 709 and 2020 limited.  When you first see an image from the Gamecube, it is likely going to have a magenta or green filter over the colors.  You will need to change the color domain to something that will give you normal looking colors.  Usually it is YUV Rec. 601 Full Range.  Vision can also rotate the display, VCS cannot.  I have found it handy in one GameCube game, Ikaruga, which has a "tate mode".

GameCube - Ikaruga Vertical Mode
The GameCube can support 240p signals, even over Component Video, without difficulty.  The only games  that uses 240p mode are releases like Mega Man Anniversary Collection and The Legend of Zelda Collector's Edition, which are collections of games for prior systems running through an emulator.  GameCube games ordinarily default to 480i which will appear as weave artifacts in Vision or VCS.  The GameCube can also support 480p (progressive mode) in certain titles, but the Datapath will handle that automatically.  (All I have said about the GameCube above applies to the Wii).

The Game Boy Player with the Game Boy Interface software also can be set to use 240p and you should use it for capturing Game Boy, Game Boy Color and Game Boy Advance games.

I had previously written about the Game Boy Interface here :
That article is somewhat out of date because instead of GBI, GBI-LL and GBI-ULL, there is now GBI Standard Edition, GBI Speedrunning Edition and GBI High Fidelity Edition, which correspond very roughly to those older Editions in respective order.  The one we want to use for minimal processing and latency and purest video quality output is GBI High Fidelity Edition (GBIHF).  Use Swiss to set the options for GBIHF to use 240p and RGB709.  Then use Vision or VCS (once you have fixed the color with Vision) to the settings on this page here  I had trouble using the .cli file, so I used Swiss to set the parameters manually.

Game Boy Advance - Metroid Zero Mission
If you have the right color domain selected with GBI, you should see a purple border around the active image.  You may need to tweak the color balance settings a little to get properly calibrated colors compared to the author of that page, use the Color Test Bars in the GBA 240p Test Suite.  I set Red to 140, Blue to 130 and Green to 200 (all both brightness and contrast) and overall brightness to 20 and contrast to 120 to get reasonably accurate colors. Vision is much easier to use to crop the image to eliminate the borders but OBS can do that as well.  You only need to capture the GBA's 240x160 or the GB/GBC's 160x144 active displays. If you want to capture the (purple) borders, set the horizontal size to 433, the resolution to 360x240 and use the 240p/60 mode in GBIHF's settings. However, once you have the color domain correct you may need to switch to VCS to get a window with a properly sharp capture, Vision usually shows a somewhat blurry window with 240p sources (but pristine screen captures) making it unsuitable for OBS Window Capture.

Conclusion

The Datapath cards are about as good as you can get these days for analog capture.  The alternatives for good quality video capture, the OSSC and the Frameister, are more expensive external devices and require a separate capture solution.  The prices are very decent on used cards.  With the Datapath you can capture just about any analog RGB or Component signal and you get DVI/HDMI capture as well.  You can use the Datapath if you are otherwise without a suitable monitor for your device.  The display lag is tolerable for some, you can open two instances of VCS and expand one for playing and let the other be recorded at its native resolution to save space.  With lossless/light compression, very high quality video captures can be made and turned into Youtube videos.  I have uploaded a pair so you can see for yourself the justice the Datapath can do to real hardware :



6 comments:

  1. What a piece of article, thanks thanks very much
    Video output in old machines is a very complex thing nowdays

    ReplyDelete
  2. Small correction near the start of the article:

    "Capturing a digital signal is often simply a matter of buying a capture card/box and plugging everything in. But capturing audio signals, at least those signals that do not conform to the "broadcast standards of 525/625i", is not quite so easy. "

    I suspect you meant "But capturing _analog_ signals".

    ReplyDelete
  3. Hello,

    any idea where to get CGA2RGB? (No longer on Ebay and GGLabs don't have it on their shop) Or are there any good alternatives?

    Thanks.

    ReplyDelete
  4. That stinks, I hope gglabs makes more because there really aren't any good alternatives at that price. The best alternative is MCE2VGA, https://youtu.be/9SII7ujB3FY but it isn't just an Digital to Analog converter but also an upscaler to VGA and costs 3x as much for a working solution. https://www.serdashop.com/MCE2VGA

    ReplyDelete
  5. A very useful and well written article!

    Helped me out a few times with my capture setup.

    Will share the article wherever I can.

    Thank you very much!

    ReplyDelete