If You Give A Dev A Tricked Out Xbox, They’ll Patch Halo 2

[Ryan Miceli] had spent a few years poring over and reverse-engineering Halo 2 when a friend asked for a favor. His friend created an improved Xbox with significant overclocks, RAM upgrades, BIOS hacks, and a processor swap. The goal was simple: patch the hardcoded maximum resolution from 480p to 720p and maybe even 1080p. With double the CPU clock speed but only a 15% overclock on the GPU, [Ryan] got to work.

Step one was to increase the size of the DirectX framebuffers. Increasing the output resolution introduced severe graphical glitches and rendering bugs. The game reuses the framebuffers multiple times as memory views, and each view encodes a header at the top with helpful information like width, height, and tiling. After patching that, [Ryan] had something more legible, but some models weren’t loading (particularly the water in the title screen). The answer was the texture accumulation layer. The Xbox has a hardware limitation of only sampling four textures per shader pass, which means you need a buffer the size of the render resolution to accumulate the textures if you want to sample more than four textures. Trying to boot the game resulted in an out-of-memory crash. The Xbox [Ryan] was working on had been upgraded with an additional 64MB of RAM, but the memory allocator in Halo 2 wasn’t taking advantage of it. Yet.

To see where the memory was going, [Ryan] wrote a new tool called XboxImageGrabber to show where memory was allocated and by whom. Most games make a few substantial initial allocations from the native allocator, then toss it over to a custom allocator tuned for their game. However, the extra 64MB of RAM was in dev consoles and meant as debug RAM, which meant the GPU couldn’t properly access it. Additionally, between the lower 64MB and upper is the Xbox kernel. Now, it became an exercise of patching the allocator to work with two blobs of memory instead of one contiguous one. It also moved runtime data into the upper 64MB while keeping video allocations in the lower. Ultimately, [Ryan] found it easier to patch the kernel to allow memory allocations the GPU could use in the upper 64MB of memory. Running the game at 720p resulted in only a semi-playable framerate, dropping to 10fps in a few scenes.

After some initial tests, [Ryan] concluded that it wasn’t the GPU or the CPU that was the bottleneck but the swap chain. Halo 2 turns VSync on by default, meaning it has to wait until a blank period before swapping between its two framebuffers. A simple tweak is to add a third frame buffer. The average FPS jumped 10%, and the GPU became the next bottleneck to tweak. With a light GPU overclock, the game was getting very close to 30fps. Luckily for [Ryan], no BIOS tweak was needed as the GPU clock hardware can be mapped and tweaked as an MMIO. After reverse engineering, a debugging feature to visual cache evictions, [Ryan] tuned the texture and geometry cache to minimize pop-ins that the original game was infamous for.

Overall, it’s an incredible hack with months of hard work behind it. The code for the patch is on Github, and there’s a video after the break comparing the patched and unpatched games. If you still need more Halo in your life, why not make yourself a realistic battle rifle from the game?

Continue reading “If You Give A Dev A Tricked Out Xbox, They’ll Patch Halo 2

Linux Fu: Easy Kernel Debugging

It used to be that building the Linux kernel was not easy. Testing and debugging were even worse. Nowadays, it is reasonably easy to build a custom kernel and test or debug it using virtualization. But if you still find it daunting, try [deepseagirl’s] script to download, configure, build, and debug the kernel.

The Python program takes command line arguments so you can select a kernel version and different operations. The script can download the source, patch the configuration, build the kernel, and then package it into a Debian package you can boot under qemu. From there, you can test and even debug with gdb. No risk of hosing your everyday system and no need to understand how to configure everything to run.

Continue reading “Linux Fu: Easy Kernel Debugging”

Linux Kernel From First Principles

Want to learn the internals of the Linux kernel? Version 6.5-rc5 has about 36 million lines of code in it, so good luck! [Seiya] has a different approach. Go back to the beginning and examine the 0.01 version of the kernel. Now you are talking about 10,000 lines and, removing comments and blanks, way less.

Sure, some things have changed, but the core ideas are the same. [Seiya] reports, “Reading V0.01 was really for me. It was like visiting Computer History Museum in Mountainview…”

Continue reading “Linux Kernel From First Principles”

Turning The PS4 Into A Useful Linux Machine

When the PlayStation 3 first launched, one of its most lauded features was its ability to officially run full Linux distributions. This was of course famously and permanently borked by Sony with a software update after a few years, presumably since the console was priced too low to make a profit and Sony didn’t want to indirectly fund server farms made out of relatively inexpensive hardware. Of course a decision like this to keep Linux off a computer system is only going to embolden Linux users to put it on those same systems, and in that same vein this project turns a more modern Playstation 4 into a Kubernetes cluster with the help of the infamous OS.

The Playstation 4’s hardware is a little dated by modern desktop standards but it is still quite capable as a general-purpose computer provided you know the unofficial, unsupported methods of installing Psxitarch Linux on one. This is a distribution based on Arch and built specifically for the PS4, but to get it to run the docker images that [Zhekun Hu] wanted to use some tinkering with the kernel needed to be done. With some help from the Gentoo community a custom kernel was eventually compiled, and after spending some time in what [Zhekun Hu] describes as “Linux Kernel Options Hell” eventually a working configuration was found.

The current cluster is composed of two PS4s running this custom software and runs a number of services including Nginx, Calico, Prometheus, and Grafana. For those with unused PlayStation 4s laying around this might be an option to put them back to work, but it should also be a cautionary tale about the hassles of configuring a Linux kernel from scratch. It can still be done on almost any machine, though, as we saw recently using a 386 and a floppy disk.

Yo Dawg, We Heard You Like Retrocomputers

The idea of having software translation programs around to do things like emulate a Super Nintendo on your $3000 gaming computer or, more practically, run x86 software on a new M1 Mac, seems pretty modern since it is so prevalent in the computer world today. The idea of using software like this is in fact much older and easily traces back into the 80s during the era of Commodore and Atari personal computers. Their hardware was actually not too dissimilar, and with a little bit of patience and know-how it’s possible to compile the Commodore 64 kernel on an Atari, with some limitations.

This project comes to us from [unbibium] and was inspired by a recent video he saw where the original Apple computer was emulated on Commodore 64. He took it in a different direction for this build though. The first step was to reformat the C64 code so it would compile on the Atari, which was largely accomplished with a Python script and some manual tweaking. From there he started working on making sure the ROMs would actually run. The memory setups of these two machines are remarkably similar which made this slightly easier, but he needed a few workarounds for a few speed bumps. Finally the cursor and HMIs were configured, and once a few other things were straightened out he has a working system running C64 software on an 8-bit Atari.

Unsurprisingly, there are a few things that aren’t working. There’s no IO besides the keyboard and mouse, and saving and loading programs is not yet possible. However, [unbibium] has made all of his code available on his GitHub page if anyone wants to expand on his work and may also improve upon this project in future builds. If you’re looking for a much easier point-of-entry for emulating Commodore software in the modern era, though, there is a project available to run a C64 from a Raspberry Pi.

Thanks to [Cprossu] for the tip!

Why Blobs Are Important, And Why You Should Care

We are extraordinarily fortunate to live at a time in which hardware with astounding capabilities can be had for only a few dollars. Systems that would once have taken an expensive pile of chips and discretes along with months of development time to assemble are now integrated onto commodity silicon. Whether it is a Linux-capable system-on-chip or a microcontroller, such peripherals as WiFi, GPUs, Bluetooth, or USB stacks now come as part of the chip, just another software library rather than a ton of extra hardware.

Beware The Blob!

An ESP-01 module
The cheapest of chips still comes with a blob.

If there is a price to be paid for this convenience, it comes in the form of the blob. A piece of pre-compiled binary software that does the hard work of talking to the hardware and which presents a unified API to the software. Whether you’re talking to the ESP32 WiFi through an Arduino library or booting a Raspberry Pi with a Linux distribution, while your code may be available or even maybe open source, the blob it relies upon to work is closed source and proprietary. This presents a challenge not only to Software Libre enthusiasts in search of a truly open source computer, but also to the rest of us because we are left reliant upon the willingness of the hardware manufacturer to update and patch their blobs.

An open-source advocate would say that the solution is easy, the manufacturers should simply make their blobs open-source. And it’s true, were all blobs open-source then the Software Libre crowd would be happy and their open-source nature would ease the generation of those updates and patches. So why don’t manufacturers release their blobs as open-source? In some cases that may well be due to a closed-source mindset of never releasing anything to the world to protect company intellectual property, but to leave it at that is not a full answer. To fully understand why that is the case it’s worth looking at how our multifunctional chips are made.

Continue reading “Why Blobs Are Important, And Why You Should Care”

Hackaday Links Column Banner

Hackaday Links: August 2, 2020

If you somehow manage to mentally separate yourself from the human tragedy of the COVID-19 pandemic, it really has provided a fascinating glimpse into how our planet operates, and how much impact seven billion people have on it. Latest among these revelations is that the shutdowns had a salubrious effect in at least one unexpected area: solar power. Researchers found that after the Indian government instituted mandatory lockdowns in March, output from solar power installations in Delhi increased by more than eight percent. The cause: the much-diminished smog, which let more sunlight reach solar panels. We’ve seen similar shutdown-related Earth-impact stories, from decreased anthropogenic seismicity to actually being able to see Los Angeles, and find them all delightfully revealing.

Remember Google Glass? It’s hard to forget, what with all the hype leading up to launch and the bitter disappointment of realizing that actually wearing the device wouldn’t go over well in, say, a locker room. That said, the idea of smart glasses had promise, and several startups tried to make a go of combining functionality with less out-there styling that wouldn’t instantly be seen as probable cause for being a creep. One such outfit was North, who made the more-or-less regular looking (if a bit hipsterish) Focals smart glasses. But alas, North was bought out by Google back in June, and as with so many things Google acquires, Focals smart glasses are being turned off. Anyone who bought the $600 specs will reportedly get their money back, but the features of the smart glasses will no longer function. Except, you know, you’ll still be able to look through them.

It looks like someone has finally come up with a pretty good use case for the adorably terrifying robot mini-dogs from Boston Dynamics. Ford Motors has put two of the yellow robots to work in their sprawling Van Dyke Transmission Plant in Michigan. Dubbed Fluffy and Spot (aww), the dogs wander around the plant with a suite of cameras and sensors, digitally mapping the space to prepare for possible future modifications and expansions. The robots can cover a lot of ground during the two hours that their batteries last, and are even said to be able to hitch a ride on the backs of other robots when they’re tuckered out. Scanning projects like these can keep highly trained — and expensive — engineers busy for weeks, so the investment in robots makes sense. And we’re sure there’s totally no way that Ford is using the disarmingly cute robo-pets to keep track of its employees.

We all know that the Linux kernel has some interesting cruft in it, but did you know that it can actually alert you to the fact that your printer is aflame? We didn’t either untilĀ  Editor-in-Chief Mike Szczys shared this reddit post that details the kernel function lp_check_status and how it assumes the worst if it detects the printer is online but also in “check mode.” The Wikipedia entry on the “lp0 on fire” error message has some interesting history that details how it’s not as implausible as it might seem for a printer, especially one in the early 1970s, to burst into flames under the right conditions. A toner fuser bar running amok on a modern laser printer is one thing, but imagine a printer with a fusing oven running out of control.

And finally, because 2020 is apparently the gift that can’t stop giving, at least in the weirdness department, the US Department of Defense let it slip that the office charged with investigating unidentified aerial phenomena is not quite as disbanded as they once said it was. Reported to have been defunded in 2017, the Advanced Aerospace Threat Identification Program actually appears to live on, as the Unidentified Aerial Phenomena Task Force, operating out of the Office of Naval Intelligence. Their purpose is ostensibly to study things like the Navy videos of high-speed craft out-maneuvering fighter jets, but there are whispers from former members of the task force that “objects of undetermined origin have crashed on earth with materials retrieved for study.” All this could just be a strategic misdirection, of course, but given everything else that has happened this year, we’re prepared to believe just about anything.