As open as the Raspberry Pi Foundation has been about their beloved products, they would be the first to admit there’s always more work to be done: Getting a Pi up and running still requires many closed proprietary components. But the foundation works to chip away at it bit by bit, and one of the latest steps is the release of a camera stack built on libcamera.
Most Linux applications interact with the camera via V4L2 or a similar API. These established interfaces were designed back when camera control was limited and consisted of a few simple hardware settings. Today we have far more sophisticated computational techniques for digital photography and video. Algorithms have outgrown dedicated hardware, transforming into software modules that take advantage of CPU and/or GPU processing. In practice, this trend meant bigger and bigger opaque monolithic pieces of proprietary code. Every one a mix of “secret sauce” algorithms commingling with common overhead code wastefully duplicated for each new blob.
We expect camera makers will continue to devise proprietary specialties as they seek a competitive advantage. Fortunately, some of them see benefit in an open-source framework to help break up those monoliths into more manageable pieces, letting them focus on just their own specialized parts. Leveraging something like libcamera for the remainder can reduce their software development workload, leading to faster time to market, lower support cost, and associated benefits to the bottom line that motivates adoption by corporations.
But like every new interface design borne of a grandiose vision, there’s a chicken-and-egg problem. Application developers won’t consume it if there’s no hardware, and hardware manufacturers won’t implement it if no applications use it. For the consumer side, libcamera has modules to interop with V4L2 and other popular interfaces. For the hardware side, it would be useful to have a company with wide reach who believes it is useful to open what they can and isolate the pieces they can’t. This is where the Raspberry Pi foundation found a fit.
The initial release doesn’t support their new High-Quality Camera Module though that is promised soon. In the short term, there is still a lot of work to be done, but we are excited about the long term possibilities. If libcamera can indeed lower the barrier to entry, it would encourage innovation and expanding the set of cameras beyond the officially supported list. We certainly have no shortage of offbeat camera sensor ideas around here, from a 1-kilopixel camera sensor to a decapped DRAM chip.
[via Hackster.io]
After reading the FAQ, it sure reads like it’s just a way to integrate binary blobs into otherwise open source code. I’m not seeing how this benefits the community.
Every ship leaves from a harbor. This is a catalyst play. It’s the harbinger of more to come. If you want to break in you have to start some where. This give open source photography a foot hold to more
It’s a common API for camera and imaging apps to be developed in a driver independent manner. It also allows for programmatic relatively low level control of the camera and ISP. Should be useful for machine vision applications.
man, you just received the first fully open source camera stack, from sensor drivers up to the open source implementation of the 3A algorithms (the first product-quality ones I know of in the free software world), and that’s your reaction ? Is the community benefiting from the otherwise closed source binaries world we have today ? People just complains for the sake of it sometimes. Or maybe you have more useful criticisms to make which I didn’t get. Feel free to expand if that’s the case :)
Does it support other MIPI-CSI devices such as the TC358743 HDMI to MIPI bridge or the ADV7280A-M CVBS to MIPI bus codec? Otherwise it’s not much more useful than the built in apps, liveMedia 555 library or dare I say it python. Actually it then becomes more to learn so it’s a backwards step imho….
Yes,
https://github.com/raspberrypi/linux/blob/804d8f832ecde209afa6201c2e27aeb106e22396/arch/arm/boot/dts/overlays/README
Hi Gordon, I know there are linux drivers for these chips but they don’t work well as generic v4l2 devices hence my question id does anyone know if this library works with them or not. Currently I’m having to use the old firmware driver with raspuvid piping to a livemedia 555 library based application which isn’t ideal but seems to work. I’d rather have more control and use a known up to date driver / device tree overlay with a single library to capture stills, video and rtsp stream too. So far this seems like something simple but is actually quite complex…
If raspy is so open, can someone point me to :
* schematics.
* PCB design.
* Source where the broadcom chips can be bought in low quantities.
* Datasheets for those chips.
They may be available by now, but I stopped being in any way interested in this false board.
Go get it on the debian list of prefereded open boards and then come back.
https://wiki.debian.org/Arm64Port
I only bothered to duckduckgo the first of your inquiries, but the schematics were pretty easy to find:
https://www.raspberrypi.org/documentation/hardware/raspberrypi/schematics/README.md
Don’t recent Pi cameras actually have some DRM thing going on so that third parties can’t produce their own? Not very open, that.
Eeesh really? That’s a bummer. Is there a source for that, I’d like to read more.
Yes, it’s rather ugly to read about: https://www.raspberrypi.org/forums/viewtopic.php?t=178038
Interesting read that however I agree with the RPi foundation and also the get out clause is use a CM3 module. Totally fine for using your own board with any sensor in the github tree, 3 I think are listed but I reckon more will come. Without the RPi foundation making money to cover their initial costs we’d not even be at third point of wanting more. The guys at RPi foundation have been a great source of help in developing our products which use the RPi 3B+, without which I wouldn’t have a job I love and medical data that wastes time, money, paper and therefore possibly lives!
There’s this too: https://hackaday.io/project/19480-raspberry-pi-camera-v21-reversed and https://gist.github.com/marcan/6dde73a9a0c917cd4fc9784a0a73efe3, though the forum thread is probably a better source.
If I wanted to learn how to build a camera driver and a complete video chain up to /dev/video0 can I use this library? Is there a step by step process of implementing it. Showing where each component goes.