Unraveling The Secrets Of Apple’s Mysterious Fisheye Format

April 18, 2024 by Donald Papp 12 Comments

Apple has developed a proprietary — even mysterious — “fisheye” projection format used for their immersive videos, such as those played back by the Apple Vision Pro. What’s the mystery? The fact that they stream their immersive content in this format but have provided no elaboration, no details, and no method for anyone else to produce or play back this format. It’s a completely undocumented format and Apple’s silence is deafening when it comes to requests for, well, anything to do with it whatsoever.

Probably those details are eventually forthcoming, but [Mike Swanson] isn’t satisfied to wait. He’s done his own digging into the format and while he hasn’t figured it out completely, he has learned quite a bit and written it all up on a blog post. Apple’s immersive videos have a lot in common with VR180 type videos, but under the hood there is more going on. Apple’s stream is DRM-protected, but there’s an unencrypted intro clip with logo that is streamed in the clear, and that’s what [Mike] has been focusing on.

Most “fisheye” formats are mapped onto square frames in a way similar to what’s seen here, but this is not what Apple is doing.

[Mike] has been able to determine that the format definitely differs from existing fisheye formats recorded by immersive cameras. First of all, the content is rotated 45 degrees. This spreads the horizon of the video across the diagonal, maximizing the number of pixels available in that direction (a trick that calls to mind the heads in home video recorders being tilted to increase the area of tape it can “see” beyond the physical width of the tape itself.) Doing this also spreads the center-vertical axis of the content across the other diagonal, with the same effect.

There’s more to it than just a 45-degree rotation, however. The rest most closely resembles radial stretching, a form of disc-to-square mapping. It’s close, but [Mike] can’t quite find a complete match for what exactly Apple is doing. Probably we’ll all learn more soon, but for now Apple isn’t saying much.

Videos like VR180 videos and Apple’s immersive format display stereoscopic video that allow a user to look around naturally in a scene. But to really deliver a deeper sense of presence and depth takes light fields.

Hacked Oscilloscope Plays Breakout, Hints At More

April 18, 2024 by Dan Maloney 8 Comments

You know things are getting real when the Dremel is one of the first tools you turn to after unboxing your new oscilloscope. But when your goal is to hack the scope to play Breakout, sometimes plastic needs to be sacrificed.

Granted, the scope in question, a Fnirsi DSO152, only cost [David Given] from Poking Technology a couple of bucks. And while the little instrument really isn’t that bad inside, it’s limited to a single channel and 200 kHz of bandwidth, so it’s not exactly lab quality. The big attractions for [David] were the CH32F103 microcontroller and the prominent debug port inside, not to mention the large color LCD panel.

[David]’s attack began with the debug port and case mods to allow access, but quickly ground to a halt when he accidentally erased the original firmware. But no matter — tracing out the pins is always an option. [David] made that easier by overlaying large photos of both sides of the board, which let him figure out which buttons went to which pins, and mapping for the display’s parallel interface. He didn’t mess with any of the analog stuff except to create a quick “Hello, oscilloscope!” program to output a square wave to the calibration pin. He did, however, create a display driver and port a game of breakout to the scope — video after the hop.

We’ve been seeing a lot of buzz around the CH32xx MCUs lately; seeing it start to show up in retail products is perhaps a leading indicator of where the cheap RISC chips are headed. We’ve seen a few interesting hacks with them, but we’ve also heard tell they can be hard to come by. Maybe getting one of these scopes to tear apart can fix that, though.

Continue reading “Hacked Oscilloscope Plays Breakout, Hints At More” →