The news here isn’t so much that [Guarav Singh] built this high-quality industrial digital camera from scratch, but it’s in the way it was accomplished. That plus the amount of information that’s packed into the write-up, of course. And the excellent photography.
Modularity was one of [Guarav]’s design goals, with the intention of being able to swap out the sensor as the technology changes. To that end, [Guarav] came up with a stack of three PCBs. The middle board of the stack contains a Lattice FPGA chip along with two 16-MB RAMs and the FPGA config flash. The sensor board lies on one side of the FBGA board, while the USB 3.0 board is on the other. Each six-layer board is a masterpiece of high-density design, and the engineering that went into interfacing them and getting everything squeezed into a 3D-printed case with an integrated aluminum C-mount ring is pretty impressive.
[Guarav]’s write-up goes into a great deal of detail on processing the sensor data on the FPGA. Also, there’s quite a bit of practical information on implementing MIPI (Mobile Industry Processor Interface) and the CSI (Camera Serial Interface) specification. We’ve delved into this world before, but this project is a great hands-on explanation that might really help move your MIPI project along.
The first attempt was using Toshiba’s TC358870XBG ASIC, capable of driving screens over MIPI DSI 1.1 from an HDMI input. [peng-zhihui] designed a simple test module for the chip based on the company’s evaluation board design, with [ylj2000] providing software to help get that solution off the ground.
However, for now that solution is imperfect, so [peng-zhihui] also experimented with the Longxun LT6911 HDMI to MIPI driver. While cheap, information on the part is scarce, and the company’s own source code for using the hardware is only accessible by signing an NDA. However, [peng-zhihui] made pre-compiled firmware available for those that wish to work with the hardware.
If you want to use a display or camera with an FPGA, you will often end up with a MIPI-based solution. As of the Xilinx Vivado 2020.1 release, the MIPI DSI (display serial interface) and CSI (camera serial interface) IP blocks are now bundled with the IDE to be used freely with Xilinx FPGAs.
The Xilinx MIPI CSI2 receiver block implements the CSI-2 v1.1 specification, which although a bit older is essentially the same CSI implementation as on the Raspberry Pi boards. This means that it would allow one to use this IP block on an FPGA with many common CSI camera modules out there. The IP block offers a standard AXI4 interface for connecting up to the rest of a design.
Similarly, the Xilinx MIPI DSI transmitter block implements DSI v1.3 specification. This offers a maximum data rate of 1.5 Gbps, with an AXI4-lite interface to communicate with the rest of the design. Both IP blocks are subject to the Core license agreement, which doesn’t appear to preclude it from being used in a specific fashion, whether commercial or personal.
This is not the only way to use MIPI devices with an FPGA, of course. Take for example [Daveshah]’s CSRIx project on Github.
Writing image data to serially connected (SPI/I2C) displays from a microcontroller is easy enough these days, courtesy of standards defined by the MIPI Alliance, yet there are some gotchas in it which may catch someone using it unaware. [Larry Bank] wrote up a good summary of how one can get maximum performance out of such a display link.
At the core is the distinction between pixel data and command transmissions. The change from command to pixel data mode requires signaling, which takes precious clock cycles away from transferring pixel data between the MCU and display. The common MIPI DCS instruction set allows for a big reduction in needed data transfers by allowing parts of the display to be addressed instead of requiring a full refresh. Yet by not properly segmenting command and data transfers, one ends up unnecessarily slowing down the process.
The result is that one can run something like a Pac-Man emulator on an AVR MCU with a sluggish 320×480 SPI LCD at 60 FPS, as one can see in the video that is embedded after the break. Check the article for another demo video as well.
[Mike Harrison] produces so much quality content that sometimes excellent material slips through the editorial cracks. This time we noticed that one such lost gem was [Mike]’s reverse engineering of the 6th generation iPod Nano display from 2013, as caught when the also prolific [Greg Davill] used one on a recent board. Despite the march of progress in mobile device displays, small screens which are easy to connect to hobbyist style devices are still typically fairly low quality. It’s easy to find fancier displays as salvage but interfacing with them electrically can be brutal, never mind the reverse engineering required to figure out what signal goes where. Suffice to say you probably won’t find a manufacturer data sheet, and it won’t conveniently speak SPI or I2C.
After a few generations of strange form factor exploration Apple has all but abandoned the stand-alone portable media player market; witness the sole surviving member of that once mighty species, the woefully outdated iPod Touch. Luckily thanks to vibrant sales, replacement parts for the little square sixth generation Nano are still inexpensive and easily available. If only there was a convenient interface this would be a great source of comparatively very high quality displays. Enter [Mike].
Outer edge of FPGA and circuit
This particular display speaks a protocol called DSI over a low voltage differential MIPI interface, which is a common combination which is still used to drive big, rich, modern displays. The specifications are somewhat available…if you’re an employee of a company who is a member of the working group that standardizes them — there are membership discounts for companies with yearly revenue below $250 million, and dues are thousands of dollars a quarter.
Fortunately for us, after some experiments [Mike] figured out enough of the command set and signaling to generate easily reproduced schematics and references for the data packets, checksums, etc. The project page has a smattering of information, but the circuit includes some unusual provisions to adjust signal levels and other goodies so try watching the videos for a great explanation of what’s going on and why. At the time [Mike] was using an FPGA to drive the display and that’s certainly only gotten cheaper and easier, but we suspect that his suggestion about using a fast micro and clever tricks would work well too.
It turns out we made incidental mention of this display when covering [Mike]’s tiny thermal imager but it hasn’t turned up much since them. As always, thanks for the accidental tip [Greg]! We’re waiting to see the final result of your experiments with this.
[Adam Taylor] always has interesting FPGA posts and his latest is no exception. He wanted to use a Zynq for image processing. Makes sense. You can do the high-speed parallel parts in the FPGA fabric and do higher-level processing on the built-in CPU. The problem is, of course, you need to get the video data into the system. [Adam] elected to use the Mobile Industry Processor Interface (MIPI) Camera Serial Interface Issue 2 (CSI-2).
This high-speed serial interface is optimized for data flowing in one direction. The camera, or the master, sends a number of bits (at least one) serially with one clock. To increase speed, data transfers on both rising and falling clock edges. The slave also has a pretty standard I2C master to send commands to the camera which, for the purposes of I2C, is the slave.
Thanks to the worldwide proliferation of smartphones, tiny high-resolution displays are common and cheap. Interfacing these displays with anything besides a phone has been a problem. [twl] has a board that does just that, converting HDMI to something these displays can understand, and providing a framebuffer so these displays can be written to through small microcontrollers.
[twl] is using a rather large FPGA to handle all the conversion from HDMI to the DSI the display understands. He’s using an Xilinx Spartan-6-SLX9, one of the most hobbyist friendly devices that is able to be hand soldered. Also on the board is a little bit of SDRAM for a framebuffer, HDMI input, and a power supply for the LCD and its backlight.
On the things [twl] has in his ‘to-do’ list, porting Doom to run on a cellphone display is obviously right at the top. He also wants to test the drawing commands for the Arduino side of his board, allowing any board with the suffix ~’ino to paint graphics and text on small, cheap, high-resolution displays. That’s a capability that just doesn’t exist with products twice [twl]’s projected BOM, and we can’t wait to see what he comes up with.
You can check out the demo video of [twl]’s board displaying the output of a Raspberry Pi below. If you look very closely, you’ll notice the boot/default screen for the display adapter is the Hackaday Jolly Wrencher.
