In this short series, we’re going to get you set up with a completely DIY home automation system using MQTT. Why? Because it’s just about the easiest thing under the sun, and it’s something that many of you out there will be able to do with material on-hand: a Raspberry Pi as a server and an ESP8266 node as a sensor client. Expanding out to something more complicated is left as an exercise to the motivated reader, or can be simply left to mission creep.
We’ll do this in four baby steps. Each one should take you only fifteen minutes and is completely self-contained. There’s a bunch more that you can learn and explore, but we’re going to get you a taste of the power with the absolute minimal hassle.
In this installment, we’re going to build a broker on a Raspberry Pi, which is the hub of your MQTT network. Next time, we’ll get an ESP8266 up and running and start logging some data. After that, we’ll do some back-end scripting in Python to make the data speak, and in the last installment, we’ll explore some of the useful frills and fancy bits. Let’s get started!
The venerable ESP8266 has rocked the Internet of Things world. Originally little more than a curious $3 WiFi-to-serial bridge, bit by bit, the true power of the ESP has become known, fully programmable, with a treasure trove of peripherals it seemed that the list of things the ESP couldn’t do was short. On that list, at least until today was Ethernet.
No, despite the misleading title, the ESP does not have a MAC and/or PHY, but what it does have is an incredible 80 MHz DMA-able shift register which can be used to communicate 10BASE-T Ethernet using a new project, espthernet. Join me after the break for video proof, and a deep dive into how this is possible.
You want to put your credit card number into a web site. You know to look for a secure web site. But what does that really prove? And now that so many electronic projects have Web servers (ok, I’ll say it… the Internet of Things), do you need to secure your web server?
There was a time when getting a secure certificate (at least one that was meaningful) cost a pretty penny. However, a new initiative backed by some major players (like Cisco, Google, Mozilla, and many others) wants to give you a free SSL certificate. One reason they can afford to do this is they have automated the verification process so the cost to provide a certificate is very low.
Researchers at University College London successfully transferred data over an optical transmission system at a rate of 1.125 Tb/s. That’s over ten times as fast as typical commercial optical systems, and thousands of times faster than the standard broadband connection. The study appeared in Scientific Reports and takes advantage of encoding techniques usually seen in wireless systems.
The prototype system uses fifteen channels on different wavelengths. Each channel used 256QAM encoding (the same as you see on cable modems, among other things). A single receiver recovers all of the channels together. The technology isn’t commercially available yet. It is worth noting that the experiment used a transmitter and receiver very close to each other. Future tests will examine how the system performs when there are hundreds or thousands of feet of optical fiber between them.
Transferring data without error when there is a lot of background noise is a challenge. Dr. Claude E. Shannon in 1948 provided guidance with his theory addressing the capacity of a communications channel in the presence of noise. His work quickly spread beyond communications into other fields. Even other aspects of computer use were impacted. An example is the transfer of data from a storage medium, like a hard drive or CD-ROM. These media and their sensors are not 100% reliable so errors occur. Just as Shannon’s work defines communication channel capacity it defines the transfer rate from a media surface to the read head.
Shannon told us how much could be passed through a channel but didn’t say how. Since 1948 a lot of research effort went into accurately detecting errors and correcting them. Error correction codes (ECC) add extra bits to messages, but their cost pays off in their ability to work around errors. For instance, without ECC the two Voyager spacecraft, now leaving our solar system, would be unable to phone home with the information they’ve gathered because noise would overwhelm their signals. Typically in hardware, like memory, error correction is referred to as ECC. In communications, the term forward error correction (FEC) is used.
Robust communication, or data transfer, is a combination of fancy software and tricky signal processing. I’m going to focus on the software side in this article. You may find some of these techniques useful in communicating data among your devices. While I’ll be using the term communications keep in mind this is applicable to transferring data in general.
My DSL line downloads at 6 megabits per second. I just ran the test. This is over a pair of copper twisted wires, the same Plain Old Telephone Service (POTS) twisted pair that connected your Grandmother’s phone to the rest of the world. In fact, if you had that phone you could connect and use it today.
I can remember the old 110 bps acoustic coupler modems. Maybe some of you can also. Do you remember upgrading to 300 bps? Wow! Triple the speed. Gradually the speed increased through 1200 to 2400, and then finally, 56.6k. All over the same of wires. Now we feel short changed if were not getting multiple megabits from DSL over that same POTS line. How can we get such speeds over a system that still allows your grandmother’s phone to be connected and dialed? How did the engineers know these increased speeds were possible?
The answer lies back in 1948 with Dr. Claude Shannon who wrote a seminal paper, “A Mathematical Theory of Communication”. In that paper he laid the groundwork for Information Theory. Shannon also is recognized for applying Boolean algebra, developed by George Boole, to electrical circuits. Shannon recognized that switches, at that time, and today’s logic circuits followed the rules of Boolean Algebra. This was his Master’s Thesis written in 1937.
Shannon’s Theory of Communications explains how much information you can send through a communications channel at a specified error rate. In summary, the theory says:
There is a maximum channel capacity, C,
If the rate of transmission, R, is less than C, information can be transferred at a selected small error probability using smart coding techniques,
The coding techniques require intelligent encoding techniques with longer blocks of signal data.
What the theory doesn’t provide is information on the smart coding techniques. The theory says you can do it, but not how.
In this article I’m going to describe this work without getting into the mathematics of the derivations. In another article I’ll discuss some of the smart coding techniques used to approach channel capacity. If you can understand the mathematics, here is the first part of the paper as published in the Bell System Technical Journal in July 1948 and the remainder published later that year. To walk though the system used to fit so much information on a twisted copper pair, keep reading.
Though we’ve never used their cables, [Blue Jeans Cable] out of Seattle, WA sure does seem to take the black art of cable manufacture seriously. When they read the Cat 6 specification, they knew they couldn’t just keep building the cables the way they used to. So they did some research and purchased a Fluke certification tester for a measly 12,000 US dollars. While they were purchasing the device, they ran across an interesting tidbit in the fluke knowledge base. Fluke said that 80% of the consumer Cat 6 cables they tested didn’t begin to meet the Cat 6 specification.
This is the part where [Blue Jeans Cable] earns our respect; like good scientists, they set out to replicate Fluke’s results. Sure enough, 80% of the Cat 6 cables they tested from big box stores etc. failed the specification. More surprising, many of them didn’t even pass the Cat 5e specification. [Blue Jeans Cable] asserts that this is possible because the Ethernet cable specification is policed via the honor system, allowing manufacturers to be fairly brazen about what they label as Cat 6.