The number of hours we spend staring at screens is probably best unknown, but how about the technology that makes up the video on the screen? We’ve all seen a reel-to-reel projector on TV or in a movie or maybe you’re old enough to have owned one, surely some of you still have one tucked away real nice. Whether you had the pleasure of operating a projector or just watched it happen in the movies the concept is pretty straight forward. A long piece of film which contains many individual frames pass in front of a high intensity lamp while the shutter hides the film movement from our eyes and our brain draws in the imaginary motion from frame to frame. Staring at a Blu-ray player won’t offer the same intuition, while we won’t get into what must the painful detail of decoding video from a Blu-ray Disc we will look into a few video standards, and how we hack them.
Observations at a glance
I’m sure most of us have noticed that an HDMI connector has quite a few connections, 19 pins make up an HDMI connector. Are all of those pins for carrying video from the set top box (insert your media player) to the display, or is there something else going on here? HDMI is orders of magnitude better in quality than any VGA monitor I’ve ever owned. I know VGA uses a 15 pin D-SUB connector, but you need to know that not all of the pins carry unique signals through a VGA connection, nor are all of them carrying part of the image we see on screen. Looking even further back into the history of video gets us to composite, one glorious yellow-tipped RCA cable. Beauty in simplicity it was, a single signal wire and a shield.
A composite video signal is carried over a single shielded conductor. The signal is broken up into frames and frames are broken up into lines. This is nothing like it sounds, a frame is a static image that covers the entire screen of the display and a line is a smaller unit spanning from the left edge of the screen to the right edge stacked one on top of each other from the top of the screen to the bottom, making up a frame. Is that what it sounded like to you? Well then I suppose it is exactly how it sounds, isn’t it.
The composite signal is broken up into smaller signals representing horizontal lines which are separated by sync signals as seen in Figure 2 (I can’t tell you what happened to Figure 1, it must have snuck out the back when you got here). The picture we see on the screen can be found in the active portion of the horizontal line signal. The amplitude of the signal is responsible for the brightness while the color information is included as a sine wave added to the brightness signal. The colors can be identified as differences in phase between the color burst reference and the signal phase. In a more digestible version: The intensity of individual colors is determined by the amplitude of modulation while the tint is determined by the phase.
That’s not much fun and the amount of non-video information here is minimal, only including the horizontal and vertical sync. The horizontal sync can be seen in the blanking portion of the signal while the vertical sync is added after the last horizontal line of the frame. It’s not necessarily as anticlimactic as it may seem, the low complexity of this video standard lends itself to easy video creation by 8 bit hackers. You can find plenty of implementations of this done in black and white incorporated into 8 bit arcade clones like Hackvision, a controller shaped PCB with audio and video out running on an ATMega328. We’ve also seen [CNLohr]’s color video done with an 8 bit AVR.
Video graphics array (VGA) reigned as the standard for quite some time. With a resolution of 640×480 – 16 color or 320×200 – 256 color (“Mode 13H”) and sometimes flicker-free video it was a dream come true. Although the time taken to draw a single horizontal line with VGA was twice as fast as composite there were still improvements to be made. Those left wanting more could change the graphics controller code to their liking. This became very popular and finding code and/or help with writing your own code was widely available. One of the methods made publicly popular by [Michael Abrash] was referred to as “Mode X” which [Abrash] shared in his column of Dr. Dobbs Journal. Mode X was interesting in that is was not documented by IBM, yet it was arguably the best mode for developers which in turn was better for the users. The advantages of Mode X were quite large in comparison to its other available modes, as [Abrash] states 5 main advantages in his book Graphics Programming Black Book:
- Mode X has a 1:1 aspect ratio, meaning you can code a circle as a circle without having to take into account a stretch which resulted in coding an ellipse to display a circle.
- Mode X utilized memory page flipping rendering smoother animations.
- Allows plane-oriented hardware to be used when processing pixels in parallel, which amounts to a 4x improvement over Mode 13H.
- Mode X and Mode 13H both have the byte-per-pixel mode for controlling each pixel with a byte of memory, which the other modes did not have. To deal with this the other modes had to bit-mask as each byte contained data for more than one pixel.
- Mode X in comparison to Mode 13H had a surplus of available off-screen memory to store image data and write it into the display memory 4 bytes (or 4 pixels) at a time when used with the inherent VGA latches.
Though Mode X was not easy to program for, it was worth the effort and not such a daunting task for an experienced VGA programmer in combination with good documentation such as [Abrash’s] book.
The VGA connection is a 15 pin D-SUB of which there are 3 conductors for the color video signal. These are the same that make up component signals Y-Pr, Y, and Y-Pb. This leaves 12 more pins on the connector, or 11 if you have a keyed connector. Six more of these conductors are dedicated GND signals, we do have a horizontal sync as well as vertical sync leaving 4 conductors unaccounted for. We’ll leave 2 of them alone and only point out that there is an I2C bus within VGA on pins 12 and 15 which are SDA and SCL respectively. These lines can be hijacked and used as standard I2C pins depending on your hardware and operating system. As you might imagine, this is not a new concept as you can see one of the best examples we have found is on the Paint Your Dragon blog and dates back to 2008.
Hack VGA into your 8-bit microcontroller projects is quite a bit more difficult than it was for composite video. Timing becomes an issues with the increased speed of VGA scanning. But it can be done. We’ve seen the Arduino do 640×480 with 8 colors and more recently it was used on a PIC24 by [Arko] for the LayerOne Demoscene Board. After all, that’s what demoscene is all about; more glitz with less bits. ARM generally has more speed so we’ve seen VGA there as well. But what truly shines for VGA is FPGA hardware. It is a straight-forward protocol to implement and the massively parallel nature of FPGA hardware means this is a class assignment for pretty much every college-level FPGA course out there. What’s more awesome that getting course credit for implementing Meat Boy in an FPGA?
High Definition Multimedia Interface (HDMI) the first noticeable difference is that this protocol also includes the transmission of an audio signal. More specifically, HDMI 2.0 delivers 32 channels (4 streams) of audio and dual stream video. The 19-conductor pinout contains 3 sets of shielded, twisted, differential signal pairs (that’s a lot of buzz words that sum up to ‘high speed’).
To be more specific the data lines use a technology called “Transition Minimized Differential Signaling” (TMDS). There are three channels of TMDS used in HDMI and each channel consists of the data-, data+, and shield conductors. It is worth mentioning that the shield for each pair is isolated to an individual pin rather than all tied to the same outer shield of the connector and eventually to the ground plane as we see commonly in USB.
In order to recover the clock signal and sustain signal integrity the bits in the stream of data need to be balanced. TMDS uses a modified version of the standard 8b/10b encoding used in the telecom industry. The idea is to balance the number of 1’s and 0’s in each piece of data which is 8b in length. The 9th and 10th bit describe how the balancing was achieved. The first stage is started by using an algorithm to determine how many bits are set in the original byte. By systematically modifying all bits except the LSB the system gets closer to a balanced number of 1’s and 0’s. If there are more 1’s than 0’s or there are 4 of each then we move into the XNOR system. As stated earlier the LSB (D) is left unchanged and its value is used to determine what will be held in D by XNORing D and D and placing the result into D. The process continues by XNORing D with D and placing the result into D. So on and so forth until D XNOR D and placed into D, the system now needs to leave information stating that an XNOR process was used as opposed to an XOR, which would have been the case if the original byte had more 0’s than 1’s. This is done by setting D=0 for XNOR and D=1 for XOR. The second part of the encoding process is to invert the first 8 bits if necessary to balance the 1’s and 0’s. The 10th bit is set if this inversion took place. Now the data is balanced and ready to be sent to another device.
This is getting beyond the realm of pedestrian embedded hacks. That doesn’t mean it can’t be done, we’re just a long way from composite video now. One of our favorite HDMI hacks is still [Bunnie’s] man-in-the-middle that allows video overlay.
The data lines in the TMDS channels only make up 9 of the conductors, there is also a TMDS twisted pair for the clock signal, putting the conductor count up to 12. We have covered how the picture data is encoded and transmitted at speeds up to 2.25 GB/s for HDMI 2.0 now let’s have a look at some of the other conductors and what they are used for.
HDMI Ethernet Channel (HEC) and Audio Return Channel (ARC) are both assigned to pin 14, and I can say that I have used the ARC function to send an audio signal from the television to the stereo receiver. But I have never, to my knowledge used the HEC function nor have I come across a real world application for this feature. I’m sure they exist, if you have used the HEC in an HDMI connection tell me about it in the comments below.
Pins 15 and 16 are dedicated SCL and SDA respectively and this is where the High-Definition Digital Content Protection (HDCP) takes place. We covered an example of using a splitter to decrypt the HDCP back in March.
Its clear the advancement of technology cause video standards to have more conductors, but is it necessary? After all isn’t most broadband delivered via coax cable? The bits that come into my house plug into the router via single shielded copper conductor with the burden of carrying the entire inter-webs and hundreds of overpriced cable channels that we don’t want or watch. This is then distributed throughout the house to devices and displays which use more conductors by several orders of magnitude to deliver the last stages of that signal.
My estimation is that conductor counts will continue to rise as they have done, but what communication protocols will be used and what new features will we see? Is I2C the set-in-stone communication protocol for video? Will Intel continue to use I2C for HDCP or is it time for a change? What do you think the next standard will be?