The “Marauder’s Map” is a magical artifact from the Harry Potter franchise. That sort of magic isn’t real, but as Arthur C. Clarke famously pointed out, it doesn’t need to be — we have technology, and we can make our own magic now. Or, rather, [Dave] on the YouTube Channel Dave’s Armoury can make it.
[Dave]’s hardware store might be in a rough neighborhood, since it has 50 cameras’ worth of CCTV coverage. In this case, the stockman’s loss is the hacker’s gain, as [Dave] has talked his way into accessing all of those various camera feeds and is using machine vision to track every single human in the store.
Of course, locating individuals in a video feed is easy — to locate them in space from that feed, one first needs an accurate map. To do that, [Dave] first 3D scans the entire store with a rover. The scan is in full 3D, and it’s no small amount of data. On the rover, a Jetson AGX is required to handle it; on the bench, a beefy HP Z8 Fury workstation crunches the point cloud into a map. Luckily it came with 500 GB of RAM, since just opening the mesh file generated from that point cloud needs 126 GB. That is processed into a simple 2D floor plan. While the workflow is impressive, we can’t help but wonder if there was an easier way. (Maybe a tape measure?)
Once an accurate map has been generated, it turns out NVIDIA already has a turnkey solution for mapping video feeds to a 2D spatial map. When processing so much data — remember, there are 50 camera feeds in the store — it’s not ideal to be passing the image data from RAM to GPU and back again, but luckily NVIDIA’s “Deep Stream” pipeline will do object detection and tracking (including between different video streams) all on the GPU. There’s also pose estimation right in there for more accurate tracking of where a person is standing than just “inside this red box”. With 50 cameras, it’s all a bit much for one card, but luckily [Dave]’s workstation has two GPUs.
Once the coordinates are spat out of the neural networks, it’s relatively simple to put footprints on the map in true Harry Potter fashion. It really is magic, in the Clarkian sense, what you can do if you throw enough computing power at it.
Unfortunately for show-accuracy (or fortunately, if you prefer to avoid gross privacy violations), it doesn’t track every individual by name, but it does demonstrate the possibility with [Dave] and his robot. If you want a map of something… else… maybe check out this backyard project.
How dare you, muggle!
That has to be one of the dumbest ways to get a 2d map.
It’s hard to believe anyone would think that’s a good strategy, but all these videos are just about generating ad revenue anyway. Turn this crap off, it’s making us much less reasonable and intelligent.
Indeed. The justification for getting the robot to map the environment was tenuous at best.
It might have further applications.. like take new photos every night and difference them for inventory purposes?
That’s a bit of a reach, to be honest.
If a 2d map is all you want yes. But if it is of any use to you to have that 3d model, which in this case I could see – perhaps to map out what is in the blind spots of the individual cameras so your thief tracking map is able to stay aware of the folks currently out of sight, including when a camera is obscured quite possibly perfectly legitimately by the guy loading/unload a shelf in the way etc.
Certainly doubt it was actually used for anything to make it worth it this time, as the whole thing appears to be more an exercise in a fun daft idea taken just far enough to demo it works than anything more real.
Feels like they could have used a robot vacuum or something with the same algorithms.
The 3D point cloud is overkill but is it really “the dumbest way” if you already had a robot that generated point clouds?
“That has to be one of the dumbest ways to get a 2d map.”
Can we say “Overkill”???
95% of “maker” youtube is now just people desperately trying to think of ANY project no matter how stupid or pointless that hasn’t already been done.
Hopefully most of these channels will bankrupt themselves and the few quality ones will be left standing.
impressive, imagine what you could do with the budget of a country that likes to track it’s people? self driving car data and Google maps. can you imagine?
The making of the map screams overengineering.
But displaying it on site with some ‘ads’ against shoplifting could be a good idea.
As someone living in the EU and having to comply with GDPR, I find it honestly scary that a hardware store would just provide some random person with access to their CCTV system and allow them to post it on the internet.
Given the dystopian state surveillance stuff that it seems most of Europe is jumping towards with great enthusiasm at government levels I think there is more to worry about here than this sort of stuff. Not that I disagree at all, it is scary – though I’m not sure this would actually be impacted by the GDPR stuff anyway – the people are not exactly the focus, really identified, its a ‘public’ space etc. So while its not 100% certainly in the clear legally, and I am not a lawyer, I think it probably doesn’t cross the line.
Reminds of one of those Amazon stores.
It all feels pretty Orwellian. No doubt that sort of technology will be everywhere in a few short years.
“(Maybe a tape measure?)”
Laser tape measure with a hookup to an app. Throw in a ladder for some height gain.
What a ridiculous idea, just give staff an RFID tag and kit the place out with readers at strategic points, like the CCTV camera locations
Provide “free wifi access” and triangulate phones that connect with multiple routers? I have a feeling RFID wouldn’t reach beyond a few inches.
That’s because <$$> is an amateur. Tracking people’s phones by various MAC addresses, and tying that to their names via various databases built up as they buy things, is at least 10 years old as a commercially deployed technology. I’m not sure how widely it’s caught on, since I suspect it doesn’t actually pay for itself, but it’s out there.
The randomization of MAC addresses reduced its usefulness.
(There are still ways to make it work, but the reliability was reduced.)
The question on everyone’s mind, from an old movie: “who is paying for all this sh.t?”
We are through buying overpriced chinese-made stuffs.
Will it show the couple clearly in mid-coitus like the original Harry Potter one did?