UDP Broadcasting And The Joys Of IPv4 Subnetting

In the previous installment on UDP broadcasting and service discovery, the basics of both were explored, including an implementation in the form of NyanSD and its protocol. Contained in the comment section was a very good demonstration of why one of the most exciting aspects of software development is the opportunity to share your latest creations with other people. This being the ability to get solid feedback on all the points – including any potential boneheaded omissions – that you really should address, whether intentional or accidental.

The most pertinent point raised was definitely that of broadcast addresses and IPv4 subnets, with the latter topic especially being something that the sysadmins at the office would talk about all the time, but which us software developers were always happy to ignore as something that didn’t concern us. Turns out the joke was on me and everyone else – like our esteemed readers – who thought that they could escape the fascinating world of subnets, as today we’ll take an in-depth look at what subnets are and how they are relevant to the world of UDP network discovery.

I somewhat alluded in the first article to the topic of ‘which broadcast address to use’ as being somewhat of a rough topic to figure out, which is clearly why I just stuck to a blatantly ‘works for me’ /24 subnet that usually will work on networks, until it does not.

Continue reading “UDP Broadcasting And The Joys Of IPv4 Subnetting”

Between-Device Sharing Still Sucks

Once upon a time, computing was simple. You had files on a floppy disk. If you wanted to take them to a different computer, you ejected the disk from one machine and put it in another. It wasn’t fast, but it was easy and intuitive. Besides, you probably only had one computer of your own, anyway.

Life has since gotten a lot more complex. You’ve got a desktop, a laptop, a work laptop, your personal and business phones, and a smart watch to boot. You live amongst a swirling maelstrom of terabytes of data. Despite all the technical advances that got you here, it’s still a pain to get a file from one device to another, even when they’re sitting on the same desk. Why?!

Continue reading “Between-Device Sharing Still Sucks”

When Mains Networking Fails, Use Phone Wires

A quiet shift over the last couple of decades in many places has been the disappearance of the traditional copper phone line. First the corded landline phone was replaced by cordless, then the phone migrated to a mobile device, and finally DSL connections are being supplanted by fiber. This leaves copper-era infrastructure in houses, which [TheHFTguy] decided to use for Ethernet.

The hack here isn’t that he bought some specialized network boxes from Germany, though knowing they exist is useful. Instead it comes in his suggestion that they use the same technology as mains networking. Mains network plugs are a dime a dozen, but noisy power lines can make them of limited use. Our hacking curiosity is whetted by the question of whether a cheap mains networking plug can have its networking — in reality a set of RF subcarriers — separated from its mains power supply, and persuaded to do the same job at a fraction of the cost. Come on commenters – has anyone ever tried this?

Tolerating Delay With DTN

The Internet has spoiled us. You assume network packets either show up pretty quickly or they are never going to show up. Even if you are using WiFi in a crowded sports stadium or LTE on the side of a deserted highway, you probably either have no connection or a fairly robust, although perhaps intermittent, network. But it hasn’t always been that way. Radio networks, especially, used to be very hit or miss and, in some cases, still are.

Perhaps the least reliable network today is one connecting things in deep space. That’s why NASA has a keen interest in Delay Tolerant Networking (DTN). Note that this is the name of a protocol, not just a wish for a certain quality in your network. DTN has been around a while, seen real use, and is available for you to use, too.

Think about it. On Earth, a long ping time might be 400 ms, and most of that is in equipment, not physical distance. Add a geostationary orbital relay, and you get 600 ms to 800 ms. The moon? The delay is 1.3 sec. Mars? Somewhere between 3 min and 22 min, depending on how far away it is at the moment. Voyager 1? Nearly a two-day round trip. That’s latency!

Continue reading “Tolerating Delay With DTN”

Escaping The Linux Networking Stack At Cloudflare

Courtesy of the complex routing and network configurations that Cloudflare uses, their engineers like to push the Linux network stack to its limits and ideally beyond. In a blog article [Chris Branch] details how they ran into limitations while expanding their use of soft-unicast functionality that fits with their extensive use of anycast to push as much redundancy onto the external network as possible.

The particular issue that they ran into had to do with the Netfilter connection tracking (conntrack) module and the Linux socket subsystem when you use packet rewriting. For soft-unicast it is important that multiple processes are aware of the same connection, yet due to how Linux works this made it impossible to use packet rewriting. Instead they had to use a local proxy initially, but this creates overhead.

To work around this the solution appeared to be to abuse the TCP_REPAIR socket option in Linux, which normally exists to e.g. migrate VM network connections. This enables one to describe the entire socket connection state, thus ‘repairing’ it. Combined with TCP Fast Open to skip the whole handshake bit with a TFO ‘cookie’. This still left a few more issues to fix, with an early demux providing a potential solution.

Ironically, ultimately it was decided to not break the Linux networking stack that much and stick with the much less complicated local proxy to terminate TCP connections and redirect traffic to a local socket. Unfortunately escaping the Linux networking stack isn’t that straightforward.

A graph of download speeds is shown, with two triangular spikes and declines. Above the graph, the label “8 MB/s” is shown.

A Quick Introduction To TCP Congestion Control

It’s hard to imagine now, but in the mid-1980s, the Internet came close to collapsing due to the number of users congesting its networks. Computers would request packets as quickly as they could, and when a router failed to process a packet in time, the transmitting computer would immediately request it again. This tended to result in an unintentional denial-of-service, and was degrading performance significantly. [Navek]’s recent video goes over TCP congestion control, the solution to this problem which allows our much larger modern internet to work.

In a 1987 paper, Van Jacobson described a method to restrain congestion: in a TCP connection, each side of the exchange estimates how much data it can have in transit (sent, but not yet acknowledged) at any given time. The sender and receiver exchange their estimates, and use the smaller estimate as the congestion window. Every time a packet is successfully delivered across the connection, the size of the window doubles.

Once packets start dropping, the sender and receiver divide the size of the window, then slowly and linearly ramp up the size of the window until it again starts dropping packets. This is called additive increase/multiplicative decrease, and the overall result is that the size of the window hovers somewhere around the limit. Any time congestion starts to occur, the computers back off. One way to visualize this is to look at a graph of download speed: the process of periodically hitting and cutting back from the congestion limit tends to create a sawtooth wave.

[Navek] notes that this algorithm has rather harsh behavior, and that there are new algorithms that both recover faster from hitting the congestion limit and take longer to reach it. The overall concept, though, remains in widespread use.

If you’re interested in reading more, we’ve previously covered network congestion control in more detail. We’ve also covered [Navek]’s previous video on IPV5. Continue reading “A Quick Introduction To TCP Congestion Control”