VR Telepresence Tank From Raspberry Pi, Google Cardboard, And Xbox Controller

It’s great to see different kinds of hardware and software tossed into a project together, allowing someone to mix things that don’t normally go together into something new. [Freddy Kilo] did just that with a project he calls his VR Robot Tank. It’s a telepresence device that uses a wireless Xbox controller to drive a tracked platform, which is itself headed by a Raspberry Pi.

The Pi has two cameras on a pan-tilt mount, and those cameras are both aimed and viewed via a Google Cardboard-like setup. A healthy dose of free software glues it together, allowing things like video streaming (with U4VL) and steering via the wireless controller (with xboxdrv). A bit of fiddling was required for some parts – viewing the stereoscopic cameras for example is done by opening and positioning two video windows just right so as to see them through the headset lenses. It doesn’t warp the image to account for the lens distortion in the headset, and the wireless range might be limited, but the end result seems to work well enough.

The tank is driven with the wireless controller while a mobile phone mounted in a headset lets the user see through the cameras; motion sensing in the phone moves those cameras whenever you move your head to look around. Remote Control hobbyists will recognize the project as doing essentially the same job as FPV setups for model aircraft (for example, Drone Racing or even Snow Sleds) but this project uses a completely different hardware and software toolchain. It demonstrates the benefits of having access to open tools to use as virtual “duct tape”, letting people stick different things together to test a concept. It proves almost anything can be made to work if you have a willingness to fiddle!

17 thoughts on “VR Telepresence Tank From Raspberry Pi, Google Cardboard, And Xbox Controller

  1. Great job ! Congratulations ! I really like the idea of using readily available hardware and software components and combining them creatively to bring to life an interesting and fun project. I believe this kind of projects are truly inspirational for young people with an interest in computers and electronics. What a great opportunity for high school science teachers, to motivate their students, with a limited budget and great return over investment (measured in money, time, effort).

    For the jittery movement of the servos, in response to the accelerometers signals I think you are right, the problem is in the data the accelerometers generate, may be you can “smooth” it by applying a “filter” (ej. a running average, using circular data queue, to average the last “n” angle readings in each coordinate), it would probably be less responsive but it will reduce the jitter. It will require trial and error of different queue sizes and averaging methods.

    Best regards and keep on with the good work !
    Daniel.

    1. Hey thanks for the advice. I’m actually applying a low-pass filter to the orientation values, which is working a lot better with the accelerometer than it is with the magnetometer. There is a way to make it work, google is doing it with their cardboard apps! Must. Keep. Trying….

      1. It will probably work very well… but actually I would not go so far… I think that a simple circular queue with the last 5 (or may be 10) angle readings and the arithmetic average of those values will do. You will have to do it for both coordinates, may be with different queue sizes for each. But don’t take my word for it, I am just guessing, a proper solution will need at least a brief review of the actual data, the frequency of the readings, the range of valid values, etc,

        (And I must confess that I am biased towards the circular list & arithmetic average solution, it was (in another context) part of the correct solution to my Assembler/360 final exam “a little” time ago… ;-) ).

        Best regards,
        Daniel.

        1. Well that would be less work for the CPU too, but the RPi should have the power for something more complex if required. No harm in trying the simplest option first, and the average of 8 or 16 numbers is faster to compute as you can divide with a shift operation.

          I’d like to visit Montevideo one day, is there much of a hardware Hacker or Maker scene in Uruguay?

  2. I’m digging the use of your smartphone for the FP setup. My school is attempting to build a racing quadcopter team, and this would drastically help cut costs if we could just use our smartphones, but things like latency would become bigger issues. Love the progress though!

  3. I thought tele-presence implies that the device has a screen and/or speakers, so that your “presence” can be simulated? This just seems like an FPV vehicle.

    That being said, I think its AWESOME! Great project!

  4. Very cool project. One thought – why not make a single webpage that has the two video streams in equally-sized side-by-side iframes or divs? The headtracking is a very nice touch. I’d love to have that android app. I’ve done quite a few similar projects with the Pololu Maestro.

    1. Good idea. My goal though is to build an actual cardboard app that will handle the UI and the service that controls the servos. The biggest challenge for me would be finding the time to accomplish this.

      1. Good day. For a long time now, I’ve been looking for dual stream usb webcams on gyroscope-controlled servers with the help of pigpio. Unfortunately, all instructions are obsolete. can you help me with the php function code for today’s android devices. possibly contact at DeDavion @ gmail. Good job

Leave a Reply to Jimmy PopCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.