Hail To The King, Baby: Reverse Engineering Duke

June 8, 2019 by Tom Nardi 15 Comments

If you’re a fan of DOS games from the 1990s, you’ve almost certainly used DOSBox to replay them on a modern computer. It allows you to run software in a virtual environment that replicates an era-appropriate computer. That’s great for historical accuracy, but doesn’t do you much good if you’re trying to leverage modern computing power to breathe some new life into those classic titles. For that, you need to dig in a little deeper.

For the last two and a half years, [Nikolai Wuttke] has been doing exactly that for 1993’s Duke Nukem II. The end result is RigelEngine, an open source drop-in replacement for the original game binary that not only runs on a modern Windows, Linux, or Mac OS machine, but manages to improve on the original in a number of ways. An accomplishment made even more impressive once you learn that the original source code for the game has been lost to time, and that he had to do everything blind.

In a blog post chronicling his progress so far, [Nikolai] explains the arduous process he used to make sure his re-implementation was as accurate as possible to the original game. He spent untold hours studying the original game’s disassembled code in Ida Pro, handwriting out pages of notes and pseudocode as he tried to understand what was happening behind the scenes. Once a particular enemy or element of the game was implemented in RigelEngine, he’d record the gameplay from his version and compare it to the original frame by frame so he could fine tune the experience.

So what’s the end result of more than two years of work and over 25K lines of code? Thanks to the incredible advancements in computing power since the game’s release nearly 30 years ago, [Nikolai] has managed to remove the need for loading screens. His engine is also capable of displaying an unlimited number of particle effects on the screen at once, and multiple sound effects can now be played simultaneously. In the future he’s looking to implement smooth character movement (in the original game, movement was in 8 pixel increments) and adaptive volume for sound effects based on their distance from Duke. Ultimately, RigelEngine should be able to replace the original graphics with new high resolution textures once some issues with the rendering buffer gets sorted out.

It’s hard to overstate how important some of these classic games are to those who grew up playing them. With John Romero still releasing DLC for the original DOOM and hackers disassembling nearly 40 year old games to fix bugs, it doesn’t seem like they’re in any danger of being forgotten.

Continue reading “Hail To The King, Baby: Reverse Engineering Duke” →

Wolfram Engine Now Free… Sort Of

May 22, 2019 by Al Williams 7 Comments

You’ve probably used Wolfram Alpha and maybe even used the company’s desktop software for high-powered math such as Mathematica. One of the interesting things about all of Wolfram’s mathematics software is that it shares a common core engine — the Wolfram Engine. As of this month, the company is allowing free use of the engine in software projects. The catch? It is only for preproduction use. If you are going into production you need a license, although a free open source project can apply for a free license. Naturally, Wolfram gets to decide what is production, although the actual license is pretty clear that non-commercial projects for personal use and approved open source projects can continue to use the free license. In addition, work you do for a school or large company may already be covered by a site license.

Given how comprehensive the engine is, this is reasonably generous. The engine even has access to the Wolfram Knowledgebase (with a free Basic subscription). If you don’t want to be connected, though, you don’t have to be. You just won’t be able to get live data. If you want to play with the engine, you can use the Wolfram Cloud Sandbox in which you can try some samples.

Continue reading “Wolfram Engine Now Free… Sort Of” →

Examine Source Code To Assembly Mapping With PenguinTrace

May 9, 2019 by Ted Yapo 23 Comments

C-programmers who don’t have a mental model of what’s going on underneath their thin veneer of abstraction above assembly code are destined for trouble. In order to provide a convenient way to understand what C-code gets compiled to and how it runs on the machine, [Alex Beharrell] has created penguinTrace, a program which allows you to see what instructions your code compiles to, and examine how it executes.

While you can get somewhat similar functionality out of standard debuggers, penguinTrace was purpose-built to facilitate exploration of how the whole process works. You can single-step through the instructions your code compiled to, examine variables, and look at the stack — the usual debugger stuff — but structured more for exploration and learning than full-on debugging. Based on our experiences when we learned low-level programming, anything that can help novices build that all-important mental picture of what’s going on underneath is a good thing. But, since it was written with a secondary purpose of learning how debuggers themselves work, it’s a great opportunity for exploring that space, too.

The UI harnesses CodeMirror to provide a browser-based interface, and is configurable to use Clang or GCC for compilation. It supports AMD64/X86-64 and AArch64 architectures, and will run on Windows using WSL: if you’ve got a PC running Linux, a Raspberry Pi, or a Windows box, you’re good to go. The code is AGPL-licensed and available on GitHub. So, if you want to gain a better understanding of what happens when you compile and run “hello, world,” grab a copy and start exploring.

This isn’t the only way to debug, though – we previously featured an application that allows a type of debugging for the Arduino platform.

Fortran Goes Interactive

May 3, 2019 by Al Williams 8 Comments

When you think of Fortran you probably think of punched cards and green bar paper. While it is true that Fortran isn’t the go-to language it used to be — pun unintentional — it still has a vibrant community of people who do serious number crunching. However, many members of that community have been seduced away by interactive tools that are also good at number crunching like MATLAB, Julian, and Python with special libraries. The LFortran project aims to create a Fortran environment with interactivity like Python, but retaining the speed that Fortran is known for.

The resulting tool is impressive. You can use it from Jupyter, can parse code targeting existing Fortran compilers, and supports Linux, Mac, and Windows. There is development to make the code fully interoperable with other languages like C or Python as well as take advantage of GPUs and other specialized hardware. They are also zeroing in on full Fortran 2018 support.

Continue reading “Fortran Goes Interactive” →

Pano Logic FGPA Hacking Just Got Easier

April 19, 2019 by Tom Nardi 8 Comments

When Pano Logic went out of business in 2012, their line of unique FPGA-based thin clients suddenly became a burden that IT departments didn’t want anything to do with. New and used units flooded the second-hand market, and for a while you could pick these interesting gadgets up for not much more than the cost of shipping. Thanks to considerable interest from the hacking community the prices for these boxes have climbed a bit on eBay, but they’re still a great way to get your feet wet with FPGA hacking.

Especially now, as Pano Logic fanatic [Skip Hansen] has figured out how to flash a new firmware on them without having to crack open the case and break out the JTAG or SPI programmer. For the seasoned hardware hacker that might not seem like a big deal, but if you’re new to the game or just more interested in the software side of the equation, this trick makes things considerably more accessible. Having an external programmer is still a good idea if things go south, but if you’re just looking to flash some demos and see what the hardware is capable of this is a huge quality of life improvement.

Even if you aren’t interested in fiddling with the orphaned products of a defunct Bay Area startup, the write-up is a fascinating look at practical software reverse engineering. As it turns out, [Skip] didn’t create this new firmware update tool from scratch. He actually opened up the official Linux update utility from Pano Logic in Ghidra and was able to figure out where the firmware image actually lived inside the program. He then wrote his own tool in C which will patch the update tool with a user-supplied firmware image.

After patching, all you need to do is follow the official update procedure, which Pano Logic helpfully documented in the YouTube video after the break. [Skip] mentions he didn’t find any clear license information in the official software he was fiddling with, and of course with the company out of business it’s not too likely anyone is going to come knocking down his door anyway. Still, he says the downloads for the Pano Logic updater are still floating around on the tubes out there for you to find, so he’s not distributing anyone’s code but his own in this project.

There are a number of hackers out there working to turn the Pano Logic thin clients into useful general purpose FPGA platforms, such as [Tom Verbeure], who’s incredible graphics demos got [Skip] inspired to grab his own unit off eBay. With support for USB and SDRAM added by [Wenting Zhang] while getting his FPGA GBA emulator running on the hardware, it seems there’s never been a better time to get on the Pano Logic train.

Continue reading “Pano Logic FGPA Hacking Just Got Easier” →

Tracking Binary Changes: Learn The DIFF-erent Ways Of The ELF

April 7, 2019 by Kerry Scharfglass 3 Comments

Source control is often the first step when starting a new project (or it should be, we’d hope!). Breaking changes down into smaller chunks and managing the changes between them makes it easier to share work between developers and to catch and revert mistakes after they happen. As project complexity increases it’s often desirable to add other nice to have features on top of it like automatic build, test, and deployment.

These are less common for firmware but automatic builds (“Continuous Integration” or CI) is repetitively easy to setup and instantly gives you an eye on a range of potential problems. Forget to check in that new header? Source won’t build. Tweaked the linker script and broke something? Software won’t build. Renamed a variable but forgot a few references? Software won’t build. But just building the software is only the beginning. [noseglasses] put together a tool called elf_diff to make tracking binary changes easier, and it’s a nifty addition to any build pipeline.

In firmware-land, where flash space can be limited, it’s nice to keep a handle on code size. This can be done a number of ways. Manual inspection of .map files (colloquially “mapfiles”) is the easiest place to start but not conducive to automatic tracking over time. Mapfiles are generated by the linker and track the compiled sizes of object files generated during build, as well as the flash and RAM layouts of the final output files. Here’s an example generated by GCC from a small electronic badge. This is a relatively simple single purpose device, and the file is already about 4000 lines long. Want to figure out how much codespace a function takes up? That’s in there but you’re going to need to dig for it.

elf_diff automates that process by wrapping it up in a handy report which can be generated automatically as part of a CI pipeline. Fundamentally the tool takes as inputs an old and a new ELF file and generates HTML or PDF reports like this one that include readouts like the image shown here. The resulting table highlights a few classes of binary changes. The most prominent is size change for the code and RAM sections, but it also breaks down code size changes in individual symbols (think structures and functions). [noseglasses] has a companion script to make the CI process easier by compiling a pair of firmware files and running elf_diff over them to generate reports. This might be a useful starting point for your own build system integration.

Thanks [obra] for the tip! Have any tips and tricks for applying modern software practices to firmware development? Tell us in the comments!

WebAssembly: What Is It And Why Should You Care?

April 4, 2019 by Ben James 45 Comments

If you keep up with the field of web development, you may have heard of WebAssembly. A relatively new kid on the block, it was announced in 2015, and managed to garner standardised support from all major browsers by 2017 – an impressive feat. However, it’s only more recently that the developer community has started to catch up with adoption and support.

So, what is it? What use case is so compelling that causes such quick browser adoption? This post aims to explain the need for WebAssembly, a conceptual overview of the technical side, as well as a small hands-on example for context.

Continue reading “WebAssembly: What Is It And Why Should You Care?” →