Using RealSense Cameras With OS X and Linux

The original Microsoft Kinect was a revolution in computer vision. For less than one hundred dollars, the Kinect gave everyone a webcam with a depth sensor. If you’re doing anything with robots, 3D scanning, or anything else where a computer needs to know where it is in 3D space, it’s awesome. These depth-mapping cameras have improved over the years, with the latest and most capable hardware being Intel’s RealSense 3D camera.

Despite being a very capable depth camera, support for Linux and OS X doesn’t exist. Researchers, roboticists and IoT developers are slightly miffed about this, and it seems like Intel doesn’t care about people using their hardware on platforms that aren’t Windows.

Now, finally, that’s changed. A few developers have taken it upon themselves to build a cross-platform library for the F200, SR300, and R200 Intel RealSense depth cameras.

The librealsense library features proper RealSense camera support for Linux, OS X, and Windows and provides all the functionality of the official Intel SDK. This functionality includes native depth, color, and infrared streams, synthetic streams for rectified images, calibration information, and the most interesting feature: multi-camera capture.

The hardware required to use the RealSense camera is somewhat lightweight – any recent laptop should be able to capture depth images with a RealSense camera. The camera itself requires USB 3, though, so you won’t be building a 3D scanner with a RealSense camera and a Raspberry Pi quite yet. Still, it’s the latest advancement for giving robots 3D vision and building cheap, portable 3D scanners.

21 thoughts on “Using RealSense Cameras With OS X and Linux

  1. Another option to consider is Orbbec’s Astra/Astra Pro, which has official support for Linux and OS X via their beta SDK and through OpenNI. The company is still rather small but their products show a lot of potential.

        1. ~150 lines of dense Intel SSE assembly in the unpack_yuy2 function within src/image.cpp definitely not a 5 minute patch, the change would require a pen, paper and several hours to write custom SIMD code for an architecture other than x86/x64.

  2. Perhaps there’s this sort of mentality that they build the hardware and add support for the OS that gets them the most most money for the least effort. Then they figure if there’s enough interest for the hardware then 3rd party researchers will add support for the unsupported OSes. Removes their R&D costs, reduces support costs, and gives a free boost to their hardware sales.

    If the 3rd part support elemnt doesn’t kick in, that’s alright too. The bean counters already did their work and Intel walks to the bank anyways.

    I can see the much smaller companies depending on 3rd parties to create the support needed. But it seems like I see it alot more with the much larger companies that unquestionably have the resources.

    1. Might even be that in years to come, every house has a fleet of Intel-brained android servants (not Android), and it’s a huge market for their products. But you have to start somewhere, so here they are. They’re in a position to launch something like this, and actually make a profit doing it.

  3. I’ve been looking for one of these cameras off and on for the last few weeks and cannot find them for sale anywhere. I see that Intel doesn’t have any for sale and I can’t find them anywhere else. If anyone has links I would love to look at them. (Thank you)

      1. R200 went to pre-order status again because they ran out of stock — good news is that there’s a huge shipment being delivered in early February. They’ll be shipped out as soon as they’re repackaged.

        SR300 is the next generation F200. The SR300 is still pre-release and nobody has them outside of NDA. No idea about their general availability, but I think sometime in the next month. There is some remaining F200 stock that should appear on the website in the near future.

      2. Yep, thank you I found that page but was confused by the fact that it isn’t in actual release yet.

        “NOTICE: The Intel® RealSense™ F200 developer kit is in the process of transitioning to the SR300 camera. We expect inventory to be available in late January for limited quantities. Check this site for the SR300 dev kit ordering information.”

  4. >> The camera itself requires USB 3, though, so you won’t be building a 3D scanner with a RealSense camera and a Raspberry Pi quite yet

    No, but you could use an Odroid XU4.

  5. “and it seems like Intel doesn’t care about people using their hardware on platforms that aren’t Windows.
    Now, finally, that’s changed. A few developers have taken it upon themselves to build a cross-platform library for the F200, SR300, and R200 Intel RealSense depth cameras.”

    The authors file lists those few developers as…wait for it…Intel employees. Yes, that’s different than official support but a long way from “doesn’t care.”

    “and provides all the functionality of the official Intel SDK”
    From the readme file: “This project is separate from the production software stack available in the Intel® RealSense™ SDK, namely that librealsense only encompasses camera capture functionality without additional computer vision algorithms.”

    please, don’t let the facts get in the way of the story.

  6. “Despite being a very capable depth camera, [Microsoft Kinect] support for Linux and OS X doesn’t exist.”

    I’ll ignore Brian’s dangling modifiers, but not his invented facts. Libfreenect is a fairly mature project with good support for Linux and OS X, while also potentially having the worst project name in FOSS history as an added bonus. We’ve played around with libfreenect in my research lab, as it supports connecting to multiple Kinect devices simultaneously (the official SDK does not).

    Sure, it’s not an official SDK, but neither is the librealsense project Brian mentions in this article. The two projects are more or less in the same category — cross-platform, open-source libraries that provide basic interfacing to depth sensors. It would have been nice to see Libfreenect mentioned here.

  7. Do these sensors have the same specs as the 2nd generation xBox-One Kinect sensors? Im interested in building a 3D-Scanner around one or two of these, what precision can be expected?
    Ben

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s