If you want to use a display or camera with an FPGA, you will often end up with a MIPI-based solution. As of the Xilinx Vivado 2020.1 release, the MIPI DSI (display serial interface) and CSI (camera serial interface) IP blocks are now bundled with the IDE to be used freely with Xilinx FPGAs.
The Xilinx MIPI CSI2 receiver block implements the CSI-2 v1.1 specification, which although a bit older is essentially the same CSI implementation as on the Raspberry Pi boards. This means that it would allow one to use this IP block on an FPGA with many common CSI camera modules out there. The IP block offers a standard AXI4 interface for connecting up to the rest of a design.
Similarly, the Xilinx MIPI DSI transmitter block implements DSI v1.3 specification. This offers a maximum data rate of 1.5 Gbps, with an AXI4-lite interface to communicate with the rest of the design. Both IP blocks are subject to the Core license agreement, which doesn’t appear to preclude it from being used in a specific fashion, whether commercial or personal.
This is not the only way to use MIPI devices with an FPGA, of course. Take for example [Daveshah]’s CSRIx project on Github.
Writing image data to serially connected (SPI/I2C) displays from a microcontroller is easy enough these days, courtesy of standards defined by the MIPI Alliance, yet there are some gotchas in it which may catch someone using it unaware. [Larry Bank] wrote up a good summary of how one can get maximum performance out of such a display link.
At the core is the distinction between pixel data and command transmissions. The change from command to pixel data mode requires signaling, which takes precious clock cycles away from transferring pixel data between the MCU and display. The common MIPI DCS instruction set allows for a big reduction in needed data transfers by allowing parts of the display to be addressed instead of requiring a full refresh. Yet by not properly segmenting command and data transfers, one ends up unnecessarily slowing down the process.
The result is that one can run something like a Pac-Man emulator on an AVR MCU with a sluggish 320×480 SPI LCD at 60 FPS, as one can see in the video that is embedded after the break. Check the article for another demo video as well.
[Mike Harrison] produces so much quality content that sometimes excellent material slips through the editorial cracks. This time we noticed that one such lost gem was [Mike]’s reverse engineering of the 6th generation iPod Nano display from 2013, as caught when the also prolific [Greg Davill] used one on a recent board. Despite the march of progress in mobile device displays, small screens which are easy to connect to hobbyist style devices are still typically fairly low quality. It’s easy to find fancier displays as salvage but interfacing with them electrically can be brutal, never mind the reverse engineering required to figure out what signal goes where. Suffice to say you probably won’t find a manufacturer data sheet, and it won’t conveniently speak SPI or I2C.
After a few generations of strange form factor exploration Apple has all but abandoned the stand-alone portable media player market; witness the sole surviving member of that once mighty species, the woefully outdated iPod Touch. Luckily thanks to vibrant sales, replacement parts for the little square sixth generation Nano are still inexpensive and easily available. If only there was a convenient interface this would be a great source of comparatively very high quality displays. Enter [Mike].
This particular display speaks a protocol called DSI over a low voltage differential MIPI interface, which is a common combination which is still used to drive big, rich, modern displays. The specifications are somewhat available…if you’re an employee of a company who is a member of the working group that standardizes them — there are membership discounts for companies with yearly revenue below $250 million, and dues are thousands of dollars a quarter.
Fortunately for us, after some experiments [Mike] figured out enough of the command set and signaling to generate easily reproduced schematics and references for the data packets, checksums, etc. The project page has a smattering of information, but the circuit includes some unusual provisions to adjust signal levels and other goodies so try watching the videos for a great explanation of what’s going on and why. At the time [Mike] was using an FPGA to drive the display and that’s certainly only gotten cheaper and easier, but we suspect that his suggestion about using a fast micro and clever tricks would work well too.
It turns out we made incidental mention of this display when covering [Mike]’s tiny thermal imager but it hasn’t turned up much since them. As always, thanks for the accidental tip [Greg]! We’re waiting to see the final result of your experiments with this.
[Adam Taylor] always has interesting FPGA posts and his latest is no exception. He wanted to use a Zynq for image processing. Makes sense. You can do the high-speed parallel parts in the FPGA fabric and do higher-level processing on the built-in CPU. The problem is, of course, you need to get the video data into the system. [Adam] elected to use the Mobile Industry Processor Interface (MIPI) Camera Serial Interface Issue 2 (CSI-2).
This high-speed serial interface is optimized for data flowing in one direction. The camera, or the master, sends a number of bits (at least one) serially with one clock. To increase speed, data transfers on both rising and falling clock edges. The slave also has a pretty standard I2C master to send commands to the camera which, for the purposes of I2C, is the slave.
Thanks to the worldwide proliferation of smartphones, tiny high-resolution displays are common and cheap. Interfacing these displays with anything besides a phone has been a problem. [twl] has a board that does just that, converting HDMI to something these displays can understand, and providing a framebuffer so these displays can be written to through small microcontrollers.
[twl] is using a rather large FPGA to handle all the conversion from HDMI to the DSI the display understands. He’s using an Xilinx Spartan-6-SLX9, one of the most hobbyist friendly devices that is able to be hand soldered. Also on the board is a little bit of SDRAM for a framebuffer, HDMI input, and a power supply for the LCD and its backlight.
On the things [twl] has in his ‘to-do’ list, porting Doom to run on a cellphone display is obviously right at the top. He also wants to test the drawing commands for the Arduino side of his board, allowing any board with the suffix ~’ino to paint graphics and text on small, cheap, high-resolution displays. That’s a capability that just doesn’t exist with products twice [twl]’s projected BOM, and we can’t wait to see what he comes up with.
You can check out the demo video of [twl]’s board displaying the output of a Raspberry Pi below. If you look very closely, you’ll notice the boot/default screen for the display adapter is the Hackaday Jolly Wrencher.
The current marketplace allows hobbyists to easily find inexpensive, well-documented displays, but what if you wanted to interface with something more complicated, such as the screen on an iPod Nano 6? [Mike] has given us a detailed and insightful video showing his process for reverse engineering a device with little-to-no documentation. Here he covers the initial investigation, where one scours the web in search of any available information. In [Mike’s] example, the display uses an MIPI D-PHY interface, which he has never worked with. He learns that the MIPI Alliance will provide design specs in exchange for a signed NDA (Non-Disclosure Agreement) and a modest $8000 fee. Nice.
[Mike] shows off some serious hardware hackery, tackling some extremely difficult soldering in order to set up a proper test platform. He then demonstrates how to use a rather awesome oscilloscope to better understand the display protocol. We found it fascinating to see the video signals displayed as waveforms, especially when he shows how it is possible to count the individual binary values. The amount of information he uncovers with the oscilloscope is nothing short of amazing, proving these little devices are more complex than they seem.