There’s a line in a [Weird Al] (no relation) song that says, “I upgrade my system at least twice a day…” I know how that is. I primarily use a rolling distro, OpenSuse Tumbleweed, and if I’m having a problem that I’m too lazy to run down, it is extremely tempting to do an upgrade and see if it just happens to fix the problem.
Of course, the problem is often caused by a previous upgrade. Recently, I’ve been having a lot of trouble with the NVIDIA proprietary drivers, so I updated them yet again. After a huge amount of effort to sort out the video problems, I found that the latest kernel didn’t like my MediaTek Bluetooth adapter, which is built into the motherboard’s WiFi chipset.
This post isn’t about how to fix your Bluetooth problem. You probably don’t have the same setup I do, and even if you do, it will be sorted out in a week or two anyway. But how I temporarily fixed this issue is worth documenting. The details are going to apply to Tumbleweed and this particular adapter, but the general approach should work anywhere with any sort of kernel module problem.
My Own Fault
Part of my problem is my own fault, of course. I have a complex disk setup and do not use the recommended btrfs root file system. That means I can’t do the snapshot thing where I can just undo a bad upgrade. If I did, then sure, I should just roll back and wait for an upstream fix.
I do have “normal” backups, but they are not always totally up to date. Worse, I have found that for things like NVIDIA, the user stuff and the kernel module stuff have to match up. That makes it very hard to roll back a kernel with older modules. The modules themselves live with the kernel, but the user space stuff gets pushed out. Or, if you uninstall things, it uninstalls it for all kernels.
Truthfully, NVIDIA and others like that should keep all the user space stuff in a kernel-specific place, and then symlink it at boot to /usr/bin or wherever. But they don’t. In the end, I didn’t want to go through the trouble of rolling things back and decided to push ahead.
Modular
I did a quick search and found a four-day-old post that had the same error message I was getting and mentioned a patch to the kernel module source — literally just two lines needed changing in the btmtk module.
Of course, the trick is how to do that. If you’ve done kernel module development, you are probably all set up for it. If not, how to proceed will vary by distro. For Tumbleweed, something like:
sudo zypper in -t pattern devel_kernel sudo zypper in kernel-source kernel-default-devel kernel-syms gcc make bc flex bison openssl-devel dwarves
For other distros, you need the current kernel’s source code and the same sort of build tools. For example, for Ubuntu and probably other Debian-based distros:
sudo apt update sudo apt install build-essential linux-headers-$(uname -r) linux-source bc flex bison libssl-dev libelf-dev dwarves rsync
Then you’d still need to unpack the source tarball.
For Tumbleweed, you don’t need to unpack, but I did want to get it somewhere in my user directory:
mkdir -p ~/kernel-local # Note: trailing slashes matter here! rsync -a --delete /usr/src/linux/ ~/kernel-local/linux-btmtk/ cd ~/kernel-local/linux-btmtk
Either way, you need to get the running kernel’s configurations into the linux-btmtk directory:
cp /lib/modules/$(uname -r)/build/.config . cp /lib/modules/$(uname -r)/build/Module.symvers . 2>/dev/null || true
The Patch
The next step is to find the btmtk.c file and patch it. In my case, I needed to find this code:
case BTMTK_WMT_FUNC_CTRL: if (!skb_pull_data(data->evt_skb, sizeof(wmt_evt_funcc->status))) {
err = -EINVAL;
goto err_free_skb;
}
Then replace the error return/goto with:
status = BTMTK_WMT_ON_UNDONE; break;
The Build
Now you just have to build and install the module:
make olddefconfig make modules_prepare make M=drivers/bluetooth modules
If you want to use multiple CPUs, put a -j=X line on make (e.g,. -j=8 to use eight cores). This will take a minute.
You’ll wind up with a drivers/bluetooth/btmtk.ko file. Your first instinct will be to simply copy it over the old one. Resist that urge. Instead, try this:
sudo mkdir -p /lib/modules/$(uname -r)/updates/drivers/bluetooth sudo cp drivers/bluetooth/btmtk.ko /lib/modules/$(uname -r)/updates/drivers/bluetooth/ sudo depmod -a
Run It!
If you want to verify things, try:
modinfo -n btmtk
It should show your module and not the stock one. You could try to avoid rebooting by stopping the Bluetooth service, tearing down btusb and btmtk, and then reloading them along with the service. But, yeah, just reboot.
If your distro is different, you might have to modify these instructions a bit. Of course, you also need to know how to fix the bad module, too. Naturally, if you update the kernel, you might have to repeat it all unless your problem has been fixed. Then again, you could set up the module in DKMS to rebuild every time, but I wouldn’t unless you really thought this was going to be a long-term problem.
Once you have all this set up, you could also build your own kernel. That’s another set of headaches, but it can be worth it if you need special setups. Want to write your own modules? We’ve done that.
You might want to fix the formatting under “The Patch”.
The joys of open source. Throw in some Red Bull while doing so.
I have saved myself countless hours of pain over the years by avoiding NVIDIA with Linux!
FTFY
I stopped using Nvidia with Linux in 2007. Never had problems since. AMDGPU / Intel just work properly.
Nvidia really needs to fix their bullshit in Linux.
Remind me when AMD produces GPUs with RTX support. If I wanted to watch youtube videos I could use Intel GMA for what it’s worth.
AMD do have ray tracing etc support, and its pretty darned good if you care to use it. They are perhaps in absolutely peak performance a generation behind on such tech most of the time, but those technologies are almost always entirely optional*, more a marketing exercise than of real value, and only ‘done better’ by Nvidia because developers outside nvidia tend to put a bit more time on making all the bells and whistles work well on the Nvidia cards historically, the AMD hardware tends to be fine, it just takes a bit longer for the software support to catch up.
Now Nvidia is no longer a gaming company nor focused on anything but the AI bubble that is likely to change… Everyone is a little caught up in the AI madness, but Nvidia does appear to be far the most unhinged.
*almost nothing the RTX badge brings is actually that noticeable compared to all the really good much more computationally easy tricks to fake it with regular raster rendering, and in that AMD has usually been as good or ahead, certainly on price for a long long time.
If i wanted to slow my hardware to a crawl for negligible benefit, i could’ve just ran a local LLM instead.
Not at all. For all the seemingly crazy stuff that I’ve had to deal with on Linux, I’ve avoided sooooo much more insanity on Windows. When you break Windows, it’s over, and you better hope the rollback and fix it. Did your registry get bloated? Reinstall!
Having spent a lot of time on both open-source and closed-source systems, I’ll take open-source any day of the week.
Running as a VM gives options.
i almost always compile my own kernel as one of the first tasks for a new computer, without any reference to the debian kernel. IOW, i use http://kernel.org instead of “apt-get source”. Sometimes it’s necessary. On my first PC, it was necessary to disable PCI because the combo PCI-VLB BIOS had invalid PCI tables that crashed the kernel at boot up. More recently, I had a laptop where i needed to get a wifi driver that was newer than the distro kernel. Another where the audio connection routing wasn’t known (i sent that upstream). And i’m using one now where the Intel wifi driver had to be hacked to be tolerant of a slightly non-conforming ROM (that was rejected upstream). But often it’s unnecessary and i do it anyways.
I still don’t know if that’s stupid on my part…reducing memory usage is meaningless, and reducing exposure surface for attacks is not a huge deal to me, and my bias against initrd might be 100% unjustified (never really used it, can’t say). It’s a real pain when confronted with something like docker that uses a huge surface of unusual kernel features, with no convenient list in the documentation. I remember when hard-to-install or configure software used to always come with a howto that started with a list of kernel options it would need. sigh.
But it’s definitely a huge upside that whenever i do track a problem into the kernel, the source is right there for me to dig into. And if i find i want to change a line (more likely to be in .config than in a C source but), it is a path-already-travelled. And i’ll have a source for ‘make oldconfig’ if i want to upgrade.
Right, i remember i’ve had trouble with sleep on a device because of this.
Btw, did UEFI fix this or does it have the same problem?
Distros need to properly support ZFS. I don’t trust BTRFS with my data.
People ask me why I (mostly) run Windows. I do have one or two Un*ces for various reasons, but that’s against a dozen or more Windows boxen.
Windows sucks sometimes. Un*ices suck sometimes. For me, I’ve been doing it so long, Windows is the devil I know.
For others, like the OP, Linux is the devil they know. Like any other religious (or brand of beer or laundry detergent), you won’t get people to change. It’s what they’re used to. No sense in going apoplectic about it, and helpful to post stuff like this.
I’ve been using UNIX/Linux since 4.3 BSD soon after it came out. The problem is that the ways I did things back is still the ways they’re done today. Want to reconfigure something? Open the terminal, edit the configuration file, and then run the command to restart the daemon. A GUI to change a setting? Maybe for the most common and basic things if you’re lucky.
Speaking of GUI, the experience of writing GUI apps in Linux hasn’t really improved either. Frankly, Windows is much, much better for GUI apps.
Windows isn’t all that great for GUI apps these days. But that might not be entirely Microsoft’s fault.
The nice thing about Windows is there is one OS with version numbers and patches being the only differentiator. When I run into a show-stopper on Linux I have to learn up on all the different versions, some of which have adopted very different ways of doing things behind the scenes, then decide if I want to change distros just to fix my problem.
Maybe I just have ADHD and keep throwing Linux at bizzare hardware (Ironically Linux is the solution for very bizzare hardware, such as a PS5 CPU in a BC250 compute blade), but I run into odd hardware incompatibility way too much with Linux, although it is getting better. I also tend to tinker with weird stuff like 3D Vision and VR, and will be using all different types of GPU, not all of which are Linux supported.
Valve is really pushing to make Vulkan on Proton the best way to enjoy casual gaming, so I remain hopeful. The fastest low profile GPU from AMD remains the Dell OEM RX6500, which is reasonably cromulant in performance, though hampered greatly by a x4 lane interface and 4GB of VRAM. x8 lanes and 6 or 8GB of VRAM would have been awesome. Seems odd because AMD is so fiercely competitive they are leaving a massive win on the table and we are forced to purchase RTX 3050 low profile if we want to spend less than $300 on a GPU.
I don’t think it is that odd – Low profile and real budget GPU are largely dead as a concept as the onboard graphics in the CPU are able to be just so good for that market section, and for the most part you’d get a CPU upgrade in the process too on AMD at least – as their sockets stick around.
Also AMD in recent years has produced APU that are relatively affordable and perform better than the low profile GPU probably can even in best case situations for those sort of cards – cooling low profile is just so much larger a challenge than cooling similar or even greater power consumption in the small form factor CPU space still. And if you have a little bit more money something like the Strix Halo is just bonkers impressive.
sed -i ‘s/Recently/For Years/g’
X should be “unbreakable” in consumer Linux.
Video acceleration or 3D could break but lambda user should never be locked into a 80×25 term with no copy/paste and no access to internet doc
[quote] it is extremely tempting to do an upgrade and see if it just happens to fix the problem [/quote] what’s wrong in trying that in the first place ? It as always been my 1st goto solution and searching or posting for something that as already been fixed in an update really is a waste of time IMO?
[quote]lambda user should never be locked into a 80×25 term with no copy/paste and no access to internet doc [/quote] gpm and links are your friends ! (joking …) That said I totally agree.
And to add my unwanted contribution to the GPU proprietary drivers war, the only real trouble had were with fglrx drivers 20 Y+ ago (with a radreon 9700pro) and it was as well the only time I bought a cutting edge GPU. Since I only used Nvidia GPU but 2 to 3 generations older than cutting edge (now looking to upgrade from my 1650) and the driver support as been really great, never had to tweak anything
IF upgrade works. Sometimes it’s a big IF.
The driver probably failed BECAUSE you upgraded.
Reference to Links was expected.