3d tracking

3 Articles

Tracking Drone Flight Path Via Video, Using Cameras We Can Get

November 13, 2020 by 4 Comments

Calculating three-dimensional position from two-dimensional projections are literal textbook examples in geometry, but those examples are the “assume a spherical cow” type of simplifications. Applicable only in an ideal world where the projections are made with mathematically perfect cameras at precisely known locations with infinite resolution. Making things work in the real world is a lot harder. But not only have [Jingtong Li, Jesse Murray et al.] worked through the math of tracking a drone’s 3D flight from 2D video, they’ve released their MultiViewUnsynch software on GitHub so we can all play with it.

Instead of laboratory grade optical instruments, the cameras used in these experiments are available at our local consumer electronics store. A table in their paper Reconstruction of 3D Flight Trajectories from Ad-Hoc Camera Networks (arXiv:2003.04784) listed several Huawei cell phone cameras, a few Sony digital cameras, and a GoPro 3. Video cameras don’t need to be placed in any particular arrangement, because positions are calculated from their video footage. Correlating overlapping footage from dissimilar cameras is a challenge all in itself, since these cameras record at varying framerates ranging from 25 to 59.94 frames per second. Furthermore, these cameras all have rolling shutters, which adds an extra variable as scanlines in a frame are taken at slightly different times. This is not an easy problem.

There is a lot of interest in tracking drone flights, especially those flying where they are not welcome. And not everyone have the budget for high-end equipment or the permission to emit electromagnetic signals. MultiViewUnsynch is not quite there yet, as it tracks a single target and video files were processed afterwards. The eventual goal is to evolve this capability to track multiple targets on live video, and hopefully help reduce frustrating public embarrassments.

[IROS 2020 Presentation video (duration 14:45) requires free registration, available until at least Nov. 25th 2020.]

VR Technology Helps Bring A Galaxy Far, Far Away To Our TV

August 27, 2020 by 18 Comments

Virtual reality is usually an isolated individual experience very different from the shared group experience of a movie screen or even a living room TV. But those worlds of entertainment are more closely intertwined than most audiences are aware. Video game engines have been taking a growing role in film and television production behind the scenes, and now they’re stepping out in front of the camera in a big way for making The Mandalorian TV series.

Big in this case is a three-quarters cylindrical LED array 75 ft (23 m) in diameter and 20 ft (6 m) high. But the LEDs covering its walls and ceiling aren’t pointing outwards like some installation for Times Square. This setup, called the Volume, points inward to display background images for camera and crew working within. It’s an immersive LED backdrop and stage environment.

Incorporating projected imagery on stage is a technique going at least as far back as 1933’s King Kong, but it is very limited. Lighting and camera motion has to be very constrained in order to avoid breaking the fragile illusion. More recently, productions have favored green screens replaced with computer imagery in post production. It removed most camera motion and lighting constraints, but costs a lot of money and time. It is also more difficult for actors to perform their roles convincingly against big blank slabs of green. The Volume solves all of those problems by putting computer-generated imagery on set, rendered in real time via video game engine Unreal.

Continue reading “VR Technology Helps Bring A Galaxy Far, Far Away To Our TV” →

A Low Cost VR Headset

January 3, 2019 by 3 Comments

Virtual reality systems have been at the forefront of development for several decades. While there are  commercial offerings now, it’s interesting to go back in time to when the systems were much more limited. [Colin Ord] recently completed his own VR system, modeled on available systems from 20-30 years ago, which gives us a look inside what those systems would have been like, as well as being built for a very low cost using today’s technology.

The core of this project is a head tracker, which uses two BBC Microbits as they have both the accelerometer and compass needed to achieve the project goals. It is also capable of tracking an item and its position in the virtual space. For this project, [Colin] built everything himself including the electronics and the programming. It also makes use of Google Cardboard to hold the screen, lenses, and sensors all in the headset. All of this keeps the costs down, unlike similar systems when they were first unveiled years ago.

The ground-up approach that this project takes is indeed commendable. Hopefully we can see the code released, and others can build upon this excellent work. You could even use it to take a virtual reality cycling tour of the UK.

Continue reading “A Low Cost VR Headset” →