Although the Nintendo 64 could deliver solid 3D graphics with assets loaded at a blistering speed from the game cartridge, audio-wise it was a bit more hampered. The main challenge here was that streaming PCM audio data from a roomy CD as on the PlayStation wasn’t an option, and using compressed audio samples was impaired by Nintendo’s refusal to open up more than the CPU for third-party codecs, leading to most N64 games using so-called ‘sound fonts’ that use sample-based tracks.

Once the N64’s RSP vector processor became available for use by developers it became possible to use MP3s without clobbering the CPU, but this was still an imperfect solution. In a recent article by [Dominic Szablewski] it is thus argued that the Ultra Low Complexity (ULC) codec might be a match made in haven for the N64 hardware.

This still uses the same modified discrete cosine transform (MDCT) as many lossy audio codecs including MP3 do, but the ULC code is designed to be extremely light-weight for decoding. [Dominic] first came across it in the form of an old Game Boy Advance demo that played a 70 kbps file with no issues on the 16 MHz ARM CPU.

Naturally the ULC codec’s creator [Aikku93] didn’t employ any unusual black magic to achieve this feat, mostly just foregoing some of the heavier elements of e.g. MP3 like splitting the signal into bands, and using a light-weight syntax to store transformed coefficients.

Here [Dominic]’s goal was to port the ULC code to work with the N64’s RSP so as to offload as  much of the decoding as possible. This required becoming extremely intimate with MIPS assembly, with [Dominic] eventually giving up and asking an LLM coding chatbot to translate bits of ULC C code to [Max Bebök]’s RSP language (RSPL) syntax instead.

The good news here is thus that the open source N64 SDK libDragon now has ULC support, just in case you want more than simple tracker-based music for your homebrew N64 game and do not feel like clobbering the RSP and CPU at the same time with something like MP3 or Ogg Vorbis.