This week Jonathan Bennett and Doc Searls sit down with Mathias Buus Madsen and Paolo Ardoino of Holepunch, to talk about the Pear Runtime and the Keet serverless peer-to-peer platform. What happens when you take the technology built for BitTorrent, and apply it to a messaging app? What else does that allow you to do? And what’s the secret to keeping the service running even after the servers go down?
Continue reading “FLOSS Weekly Episode 781: Resistant To The Wrath Of God”
The Internet has allowed us to communicate more easily than ever before, and thanks to modern cell-phone networks, we don’t even have to be tied down to a hard line anymore. But what if you want something a little more direct? Maybe you’re in an area with no cell-phone coverage, or you don’t want to use public networks for whatever reason. For those cases, you might be interested in this Secure Communication Network project by [Thomas].
By leveraging the plug-and-play qualities of the Raspberry Pi 4 and the Adafruit LoRa Radio Bonnet, [Thomas] has been able to focus on the software side of this system that really turns these parts into something useful.
Rather than a simple point-to-point radio link, a mesh network is built up of any transceivers in range, extending the maximum distance a message can be sent, and building in resilience in case a node goes down. Each node is connected to a PC via Ethernet, and messages are distributed via a “controlled flooding” algorithm that aims to reduce unnecessary network congestion from the blind re-transmission of messages that have already been received.
Security is handled via RSA encryption with 256-byte public/private keys and additional SHA256 hashes for authentication.
The packet-size available through the LoRa device is limited to 256 bytes, of which 80 bytes are reserved for headers. To make matters worse, the remaining 176 bytes must contain encrypted data, which is almost always more lengthy than the raw message it represents. Because of this, longer messages are fragmented by the software, with the fragments sent out individually and re-assembled at the receiving end.
If you’re in need of a decentralized secure radio communications system, then there’s a lot to like about the project that [Thomas] has documented on his Hackaday.io page. He even includes an STL file for a 3D printed case. If you need to send more than text, then this Voice-over-LoRa Mesh Network project may be more your style.
When it comes to peer-to-peer file sharing protocols, BitTorrent is probably one of the best known. It requires a client implementing the program and a tracker to list files available to transfer and to find peer users to transfer those files. Developed in 2001, BitTorrent has since acquired more than a quarter billion users according to some estimates.
While most users choose to use existing clients, [Jesse Li] wanted to build one from scratch in Go, a programming language commonly used for its built-in concurrency features and simplicity compared to C.
The first step for a client is finding peers to download files from. Trackers, web servers running over HTTP, serve as centralized locations for introducing peers to one another. Due to the centralization, the servers are at risk of being discovered and shut down if they facilitate illegal content exchange. Thus, making peer discovery a distributed process is a necessity for preventing trackers from following in the footsteps of the now-defunct TorrentSpy, Popcorn Time, and KickassTorrents.
The client starts off by reading a .torrent file, which describes the contents of the desired file and how to connect to a tracker. The information in the file includes the URL of the tracker, the creation time, and SHA-1 hashes of each piece, or a chunk of the file. One file can be made up of thousands of pieces – the client will need to download the pieces from peers, check the hashes against the torrent file, and finally assemble the pieces together to finally retrieve the file. For the implementation, [Jesse] chose to keep the structures in the Go program reasonably flat, separating application structs from serialization structs. Pieces are also separated into slices of hashes to more easily access individual hashes.
Next, a GET request to an `announce` URL in the torrent file announces the presence of the client to peers and retrieves a response from the tracker with the list of peers. To start downloading pieces, the client starts a TCP connection with a peer, completes a two-way BitTorrent handshake, and exchanges messages to download pieces.
One interesting data structure exchanged in the messages is a bitfield, which acts as a byte array that checks which pieces a peer has. Bits are flipped when their respective piece’s status changes, acting somewhat like a loyalty card with stamps.
While talking to one peer may be straightforward, managing the concurrency of talking to multiple peers at once requires solving a classically Hard problem. [Jesse] implements this in Go by using channels as thread-safe queues, setting up two channels to assign work and collect downloaded pieces. Requests are later pipelined to increase throughput since network round-trips are expensive and sending blocks individually inefficient.
The full implementation is available on GitHub, and is easy enough to use as an alternative client or as a walkthrough if you’d prefer to build your own.
We’ve become used to seeing LoRa appearing in projects on these pages, doing its job as a low-bandwidth wireless data link with a significant range. Usually these LoRa projects take the form of a client that talks to a central Internet connected node, allowing a remote wireless-connected device to connect through that node to the Internet.
It’s interesting then to see a modest application from [Mark C], a chat application designed to use a set of LoRa nodes as a peer-to-peer network. In effect LoRa becomes the network, instead of simply being a tool to access it. He optimistically describes peer-to-peer LoRa networks as the new FidoNet in his tip email to us, which might be a bold statement, but we can certainly see some parallel. It’s important to note that the application is merely a demonstrable proof-of-concept as it stands, however we’d agree that it has some potential.
The hardware used for the project is the Heltec ESP32-based LoRa board, which comes with a handy OLED screen on which the messages appear. As it stands a PC connection is required to provide text input via serial, however it’s not impossible to imagine other more stand-alone interfaces. If it interests you the code can be downloaded from the GitHub repository, so maybe this can become the seed for wider peer-to-peer LoRa networks.
There have been no shortage of LoRa projects featured here over the years. Recent ones include a handy local LoRa packet sniffer, and news of an extreme distance record from a LoRa node on a balloon.
