TomTom already runs Linux. The OpenTom project (Internet Archive Link as of 2022) has documented the hardware and software to allow custom software builds to run. The Wiki covers everything from build tools to hardware connections. So far, a mp3 player has been released using the build tools. Hmm, I might have to pick one up to develop on myself.

Thanks to [kniVes788] for the tip.