Debloat The Internet With A Self-Hosted Compressing Proxy

It’s no secret that the size of webpages has been slowly — or not so slowly — creeping up over the years. Does a simple website need to span multiple floppy disks? [PortalRunner’s] doesn’t — or rather, he doesn’t want to burn that much of his limited mobile data. So he built a self-hosted HTTPS proxy to compress images and websites, block ads, and save his bandwidth.

The proxy itself is based on mitmproxy rather than reinventing the wheel. The biggest gains are, of course, from knocking images down to size — since he’s only going to be looking at them on a tiny screen, he can not only apply the better compression of .webp rather than the more common .PNG and .JPEG, but also toss unneeded information by reducing the resolution. Exactly how much quality to toss is left up to the individual user in a handy config.

Likewise, if you’re willing to wait a little longer for pages to load, you can force every webpage to get the highest possible level of compression — otherwise the server is only going to do that with ridiculous Java-heavy pages that absolutely need it. As a bonus, going through your home server means this acts like a private VPN to help protect your data on the go. If you want the details, check out the GitHub link above or the video embedded below.

This project is explicitly about saving data; it won’t help older hardware get online the way the publicly hosted FrogFind search engine or private-hosted MacProxy servers do. If you really want to escape the bloat of the modern internet, a proxy can also help you go way back.

Continue reading “Debloat The Internet With A Self-Hosted Compressing Proxy” →

Purely Random TV In Your Browser

One of the things we used to appreciate about broadcast television is that… you kinda just got what you got. You didn’t have to choose beyond picking a channel, and then you settled in to imbibe whatever media the CRT was spraying at you. A vague recreation of part of this experience is now available, in the form of [Sergei’s] RND.TV.

The concept is simple enough—it’s a webpage that plays random videos from The Internet Archive. Each session draws videos from a random sequence, with no algorithm or fancy social media nonsense to push some videos over others. As you might expect, you’re in for lots of random and weird stuff, from old Army videos to middle-school basketball games. You can swipe to flick to a different “channel” if you don’t like what’s playing, or you can mark videos you like if you fancied what came up in the random feed. You can also use your phone as a remote if you want to sit back and flick channels like it’s the 90s again or something.

We’ve featured other projects in a similar vein before—like little TVs from The Simpsons that play episodes on repeat. Sometimes, it’s fun to just avoid the paralysis of choice, and a random stream of content can provide that relief. Or, alternatively, you could always hook up your TV to an antenna and watch free-to-air… it’s still out there, for those that wish to view it!

A Defrag Simulator In Your Browser

Disk Defragmenter was a wonderful utility that Microsoft included with Windows back in the day. Back then, you’d use it to theoretically speed up disk access. Today, you can run a fun simulation right in your browser thanks to [Dennis Morello].

The theory behind defragmenting hard disks was simple. Your hard drive would store data on spinning magnetic platters. Sometimes, a given file or group of files would end up with their different parts scattered across different parts of a platter, or even multiple platters, as the file system tried to slot everything into the space available over time. On a drive accessed with a literal physical mechanism, this fragmentation of files across multiple areas of a disk could cause functional read speeds to drop. To solve this, you could defragment the drive, wherein a utility would grab disparate bits of different files and put them physically closer together on the drive platters, such that the read heads could access larger contiguous chunks of files more easily.

[Dennis’s] project does none of that. It just simulates the visuals and sound of running Microsoft’s disk defragmenter tool from Windows 98. It’s got the little rows of squares that get rearranged, blue for unoptimized data, dark blue for defragmented data, and white boxes for free space. It’s got the progress bar along the bottom, too, and a nice little simulated sound of a hard disk ca-chunking away as it shuffles little bits of data to and fro. This was the kind of thing you’d do on a rainy Saturday afternoon in the 1990s, just watching your PC make itself a few percent faster while you drank coffee and wondered if your ex-wife Jacinta was ever coming back. She never did, but you did notice that Age of Empires II loaded a fair bit quicker after you ran a full defrag on your main drive. Sometimes, that’s as good as it gets.

Modern file systems are better at managing issues like fragmentation, and the virtually instant seek speeds of solid-state drives essentially eliminated the need for defragmentation for good. Still, it’s fun to visit an ancient tool from yesteryear and remember what it meant to us way back when. Maybe you could give Jacinta a call, too, just for old times sake, and discuss that time a Janet Jackson song started crashing brand-new Windows laptops right out of the box…

Self-Hosting Offline Websites

If there’s one thing that’s guaranteed in the tech world it’s that nothing is guaranteed. From AOL, Netscape, Yahoo, and MySpace, every tech empire seems to eventually fall to ruin. One method to reliably maintain information or online experiences that are lost to whims of computer users and markets is to backup, preserve, and host that information on one’s own computers, and the Kiwix project aims to help make offline backups of favorite websites.

The open-source tool started as a way for users to make their own offline backups of Wikipedia but eventually expanded into having the ability to backup many other sites as well. Wiki-type sites are generally the easiest, but it can also create backups of other sites like StackExchange so that when the Internet goes down or the site vanishes from the world, the information is preserved for use. There are plenty of guides online for setting up Kiwix as well, with it running easily on most Linux systems with or without the help of Docker, as well as versions available for Windows, Android, and Apple platforms.

Although the Kiwix software hosts the website backups, creating the backups needs to be done with another tool called Zimit to create the .zim files Kiwix uses. The Kiwix organization also hosts pre-downloaded .zim files of common websites so each user is not independently crawling Wikipedia on their own, a task that could take months on limited consumer hardware. As for what hardware to ultimately host it on, we might recommend an armageddon-proof machine like this one.

Get A Remote Terminal With One Binary, One URL, And Zero Config

Launch a single static binary executable, send someone a QR code or URL (or failing that, text a numerical code or shout it across a room), and they’ll get an encrypted terminal in their browser. No VPN, no port forwarding, no firewall modifications, and no account setup required. It’s BitBang by [Rich LeGrand], and there is a lot to go through in this one.

The best part? It’s not actually limited to just firing off a terminal. It’s a whole open framework for establishing an encrypted peer-to-peer connection between two systems over WebRTC without needing either a trusted central authority, or any special network configuration.

The signaling server brokers the handshake, then has no further involvement. By design, it couldn’t see application data even if it wanted to. Click to enlarge.

Opening a remote terminal, transferring files, or accessing web apps on a remote machine’s network is done with bitbang-cli, an implementation of BitBang focused on providing simple, zero-config remote access.

Before we go on, we want to mention that BitBang does require a lightweight, trustless signaling server only to broker the initial connection, but more on that in a moment.

On the machine to be shared, one first downloads the binary. Easiest way to do that is to go to bitba.ng and download manually, or copy and paste the one-line installer to auto-detect one’s system, download the correct release, and verify the checksum.

After the binary is downloaded, simply run it in a terminal and receive a QR code to scan, a URL to copy & paste, or a numerical code if those are inconvenient. On the remote side, one accesses the signaling server and the connection is made — one gets a terminal on the target machine open in the browser tab, with added options for file sharing and accessing web applications on the target network.

The signaling server isn’t involved in authentication or encryption, and couldn’t see private data between the two ends even if it wanted to. Prefer not to use someone else’s regardless? Run your own local instance with bitbang-server.

Originally developed as an easy way to securely make telepresence robots reachable over the Internet with nothing more than a QR code, today it’s a whole framework.

It includes not just the remote-access tool mentioned above, but also a BitBang Octoprint plugin for cloud-free remote access to 3D printers, and bitbang-python is a library for turning local Python web applications into a URL that can be opened from anywhere.

We’re sure some of you are getting more than a few ideas from this. If it lets you bring a project over the finish line, let us know on our tips line.

The ESP32 Gets A Web Browser

At the Hackaday Europe conference in Italy earlier in the year, we were shown a rather interesting device. The work of [Alun Morris], it was an ESP32-powered Cheap Black Display board, and it was running a web browser. Definitely an achievement.

Lest you imagine that it was sporting the latest and greatest in browser technology, we must disappoint you. The browser in question is a very basic text mode device, but it did happily retrieve Hackaday, which should be the only test a browser should need to pass.

Under the hood it’s running FreeRTOS, with separate HTML retrieval and tokenizing, and UI processes. It can fetch web pages directly, but there’s also a server-side proxy for difficult sites, and for creating image thumbnails.

An ESP32 is a powerful microcontroller, but it’s fair to say it’s not in the league of running a web browser and as far as we can remember this is the first one we’ve seen. We’re sure it’s a field with further progress to be made though, particularly with the more powerful recent chips in the series. This project however is a good start, and more importantly it can be yours for a few dollars on Ali to buy a dev board. What are you waiting for?

Continue reading “The ESP32 Gets A Web Browser” →

The Need For Speed: Internet Speed Measurement (or DIY?)

Car enthusiasts want to know how quickly they can make a quarter mile. Weightlifters are forever trying to add one more plate to the bar. Internet denizens have their own favorite number to brag about: the result from a speed test.

The ritual is familiar. Close a few browser tabs, click the big “Go” button, and watch the needle climb. Perhaps you pay for gigabit service and see 940 megabits per second, which produces a satisfied nod. Perhaps you see 299 megabits and begin obsessing over network hardware. But before you get too excited either way, try another test. There is a fair chance it will give you a different answer.

That does not necessarily mean one test is lying. “Internet speed” is not a single physical quantity waiting to be measured. A speed test measures the performance of a particular device, over a particular local connection, through a particular ISP route, to a particular server, at a particular time using a particular test method. Change any of those things and the answer can change too. Continue reading “The Need For Speed: Internet Speed Measurement (or DIY?)” →