Back at the 2017 Superconference, Hackaday Managing Editor Elliot Williams started his talk about the so-called “Internet of Things” by explaining the only part he doesn’t like about the idea is the Internet… and the things. It’s a statement that most of us would still agree with today. If anything, the situation has gotten worse in the intervening years. Commercial smart gadgets are now cheaper and more plentiful than they’ve ever been, but it seems like precious little has been done to improve their inherent privacy and security issues.
But his talk doesn’t serve to bash the companies producing these devices or even the services that ultimately folded and left their customers with neigh useless gadgets. That’s not his style. The central theme of “Nexus Technologies: Or How I Learned to Love WiFi” is that a smart home can be wonderful thing, assuming it works the way you want it to. Elliot argues that between low-cost modular hardware and open source software, the average hacker has everything they need to build their own self-contained home automation ecosystem. One that’s not only cheaper than what they’re selling at the Big Box electronics store, but also doesn’t invite any of the corporate giants to the party.
Of course, it wasn’t always so. A decade ago it would have been all but impossible, and five years ago it would have been too expensive to be practical. As Elliot details his journey towards a truly personal smart home, he explains the advances in hardware and software that have made it not just possible on the DIY level, but approachable. The real takeaway is that once more people realize how cheap and easy it is to roll your own smart home gadgets, they may end up more than willing to kick Big Brother to the curb and do IoT on their own terms.
This previously unpublished recording somehow slipped between the cracks of the editing room floor but upon recent discovery, it’s still just as relevant today. Take a look at Elliot’s view on Nexus Technologies, then join us after the break for a deeper dive. Make sure to subscribe to Hackaday’s YouTube channel to get in on the 2019 Hackaday Superconference live stream starting Saturday, November 16th.
Espressif to the Rescue
Anyone who’s been reading Hackaday for the last couple of years probably won’t be surprised to find that Elliot credits the ESP8266 from Espressif as one of the key elements that’s brought IoT within the hobbyist’s reach. Back in 2012, we were excited about the Electric Imp because the WiFi-enabled microcontroller and associated development board meant we could get our homebrew projects on the Internet for only $35 a pop. Today, the going rate for an ESP8266 and its development board is literally 1/10th of that.
But what newer players might not realize is that the ESP8266 was never designed to be the last word in Internet-enabled microcontrollers. In fact, it was quite the opposite. Originally, Espressif intended for the ESP line to essentially be simple WiFi “modems” for other devices, complete with an AT command set. It took the work of several dedicated hackers in the community to turn them into the all-in-one Arduino and MicroPython compatible IoT wonder-chip that we have today.
According to Elliot, the ESP’s new found abilities are something of a mixed blessing. While it’s difficult to overstate the impact these low-cost MCUs have had on DIY hardware, he believes that keeping things simple has its advantages. Rather than tasking the diminutive ESP8266 with running a web server and handling a dozen different tasks, he’d rather give each one a simple assignment and push the logistics of tying it all together to a higher level. With as cheap as they are, multitasking just doesn’t make a whole lot of sense.
The Rise of MQTT
As it so happens, a bunch of TCP/IP capable “dumb” devices are the perfect match for the other major component of Elliot’s home automation system: MQTT. This lightweight, publish-subscribe messaging protocol is one of those things most of us have heard about at one time or another, but perhaps haven’t taken the time to really investigate closely. An impromptu poll during the talk showed that less than a third of those in attendance were actively using the protocol, and while we’d bet that number would be a bit higher today, it’s still fairly niche.
For anyone who’s been on the fence about diving into the wonderful world of MQTT, Elliot’s talk is the perfect thing to push you over the edge. The bulk of his presentation discusses how he deployed his system, ranging from how he uses the freeform, hierarchical nature of MQTT topics to logically group together areas of his home to step-by-step instructions for installing the Mosquitto broker on a Raspberry Pi. You may remember his excellent four-part guide on using MQTT.
Elliot goes on to explain how the synergistic relationship between his ESP8266 laden home appliances and the ever-expanding list of topics on his MQTT broker. With hardware so cheap to be negligible and the ability to easily add new parameters to the broker whenever he wishes, plugging new devices into the system becomes second nature.
He goes on to show off a number of devices he’s built with MQTT support, from an understated wooden annunciator panel that displays whatever variables have his attention at the moment, to a digital microscope that was augmented with network control because pushing a physical button induced too much motion. All of which were built with an ESP8266, a handful of components, and a few lines of code. Once you know the steps, the dance is easy.
The Golden Age of Hacking
Looking back on it after a few years, Elliot’s talk serves as a signpost roughly marking a minor revolution in hardware hacking. At least in some cases, we’ve reached the point where building your own stuff has become so cheap and easy that it isn’t just something you do because you’re looking for an intellectual challenge, but because it simply makes sense. If the Arduino was the onramp to a new era of hardware hacking, then the ESP8266, the Raspberry Pi, and the wealth of open source software available today have put us on the Autobahn; and there’s no sign things are slowing down anytime soon.
In truth, there are times when we miss the “old” days of hardware hacking, when a disused Linksys router ponderously mounted atop a gutted remote controlled truck made for an enviable Linux robotics platform. There’s probably many of you in the audience that feel the same. But we’d be fools to turn our collective noses up at a project because it didn’t use the digital equivalents of stone knives and bearskins. Instead we should celebrate the fact that today, regardless of whatever tools they decide to use, more people than ever have the ability to build the things they’ve always dreamed of.
I’m all for the IoT, as long as it works offline and doesn’t become trash when the cloud service dissapears.
The stove can send data on exactly what I’m cooking directly to China, Russia, and the NSA for all I care.
So long as it’s still legal and semi-practical for the people who DO need privacy, if it prevents even a 0.01% chance of a house fire, it’s probably worth it for most.
I’m a lot more likely to forget my keys at home than I am to get a somewhat decent smart lock hacked.
This is a large problem as a lot of the consumer IOT stuff i am looking at and some I already own is next to useless if the vendor ever decides to kill whatever AWS instance is running that ‘cloud’.
Case in point i have a really nice Logi camera setup I bought the girlfriend to use for her cats when we are on vacation.
If the logi service ever goes down, they are literally useless as all of the traffic goes up and out, and has to come back down through their web interface or apps.
I have been reading into a lot of the localised smart home projects like openhab and home assistant because I really like the modular smart home idea, but I really only have a few niche cases where i actually want it in the cloud.
Statements like these sound okay at the first glance and like a sacrifice worth living with, but i don’t think we should allow that. The problem is that you will never be able to control what happens with your data. The things you are in control of are your habits. You can quite easily change your habit to make sure you got your keys with you.
It’s not practical for anyone to send your data around the whole globe, except for the company who is receiving it. So please stop doing this and stop buying things that do this. We should never support an action like this, just because it solves some imaginary problem which we already had solved without technology.
Ehh. That’s like saying we shouldn’t have Netflix or other streaming services because they can tell what we are watching. We should just all watch broadcast TV, because trying to watch something you enjoy more than whatever happens to be on broadcast is an imaginary problem.
It has been possible to tell what you’re watching since before TV became common. Or what radio station you are listening to, because it’s the same principle. Not on a mass scale, but from hundreds of yards away using certain electronic eavesdropping techniques. The principles can be demonstrated with two AM radios, tune one to a station, tune the other until it squeals, you’ll find that’s exactly 455khz below what the other is tuned to, the IF frequency. IIRC it’s 1.6MHZ for TVs. Wouldn’t take a white van these days, small drone with SDR loitering far enough up/away to be hard to notice.
I’ve left the house without my keys. More than once. All kinds of accidents of varying levels of severity happen to all kinds of people.
Data breaches don’t cause nearly as much trouble for most people as mistakes do.
If someone is using data to influence us for evil, that should be handled directly.
Maybe at the “output” side, with open source social media that doesn’t serve ads or mess with what posts you see or don’t see.
We should obviously be pushing open source tech, because closed source devices are e waste waiting to happen, but I’m not going to get rid of my Tile trackers on my wallet and keys.
And I’m going to keep using Etcher to flash images instead of dd. A 40MB electron app to move some bits is perfectly reasonable if it prevents one “dd accident” which Reddit proves happen fairly often.
Lack of standards is one of the biggest issue to a smart home.
Lack of a legitimate use for smart gadgets is one of the biggest issues to a smart home.
I mean, sure, windows that roll up automatically are nice, but… most of the things in a house are pretty much fire and forget, and the IoT device just introduces another battery to change, another power vampire with a wall-wart, and another thing to replace 10 years hence when the capacitors die and the standards and infrastructure has moved on to something more interesting.
The smartest thing you really in a house is a thermostat you can set at 22 C. If it needs anything more, then the house was designed wrong.
Would never give up my automatic electric shutter closing when the sun goes down. As you said, it’s set up, and I never have to think about it. I still have a manual shutter on some other windows and it’s a pain to open and close everyday, fix it when it’s windy and so on. Yes, it does consume 4W for the beaglebone “brain” to drive the relay, but it’s far more useful than any other 4W lost on TV’s sleep power or whatever useless toy.
Similarly, I’ve spent a hundred buck for a capable alarm system that tells me via SMS when main power is cut and this saved my freezer’s content numerous time in the past. Another example of 5W well spent. I’m still looking for a capable leak detector system but I also think, once found, that it’ll give a true value for my dollars.
I don’t have any fancy RGB wifi light or Alexa stuff, everything works without an internet connection (although it’s WIFI underneath or GSM/3G for the alarm system). IoT does not always mean “cloud”.
>”Another example of 5W well spent.”
Let’s see. If electricity costs on average 15 cents a kWh then you’re paying $6.57 a year in electricity. Then assume the cheapest SIM card costs $20 a month… that’s about $250 a year…
…nope, I would just stack the freezer with some ice blocks and if the power stays off for longer than a day, then it’s probably a bigger fault and the food’s going to melt anyways.
I have a freezer in the garage. There is some issue with the GFCI that makes it trip for reasons I have never been able to figure. My house was built on the cheap by the bank, and the single GFCI for the garage, the outside outlets on the front and back of the house AND both bathroom outlets are all on the SAME circuit. Pretty sure they spent more on the extra wire to reach all those on one circuit than extra GFCIs would cost. I would rewire it, but the breaker panel is in the basement, with no easy path to run a wire, AND if I had a pro do it, they should move it, since the builder put it right under the kitchen sink drain. It’s pretty far down the list of priorities.
I have a night light plugged into the GFCI outlet as a “pilot light”, but it relies on the unobservant who live with me to realize the light is out and reset the GFCI. This where some form of smart tech would really help me. Yes, a freezer is plug it in and it works, but if it waits for me to notice the circuit is off, and I happen to be traveling….
In France, a SIM card is 2€ per month (2G + 50MB/month) and if you get discount during sales, it’s down to 0.5€ per month (6€ per year, with no data). Electricity is around your price, yet. Anyway, the SIM card is also used for the alarm system so it’s not a dedicated for this task nor is the beaglebone for the home automation. In the end, I assume the cost for this “feature” is close to 30€ per year, that’s one meal at the local restaurant… Acceptable.
Do X when it gets dark is a single transistor problem.
That describes 99% of the microcontroller problems on the internet.
Lack of standards is preventing grandpa from installing commodity smart home. But if he could code a few lines of the scripting language of his choice on a Pi, he’d have a hell of a good time.
Two standards are extraordinarily important to me/my setup: 802.11 and MQTT. The rest are somewhere between negotiable and irrelevant.
“Two standards are extraordinarily important to me/my setup: 802.11 and MQTT.”
I would only add low power radio to that: Bluetooth, Zigbee and 433MHz.
For my setup, Zigbee is especially interesting because wiht zigbee2mqtt* I can integrate consumer IoT devices into home network in a secure way that is completely independent from manufacturer’s cloud service.
* I only wish there was a C/C++/Python implementation without npm dependency hell
There is no reason for anyone to use the cloud for home automation except to make the cloud companies a lot of money while acting as ” big brother” able to watch your every movement. I like to use the term ” cloud free IOT” to describe a personal network of things completely controlled by the user. I use a system very similar to the one described in the video. Some comments: My system uses two MQTT brokers. One provides unsecured MQTT for local communications between ESPs and a local hub. Wifi security is adequate for that. The second supports secure MQTT with full client side certificates for external communications (eg to my phone app). Also, TCP is not good for battery powered sensors and actuators that spend a lot of time sleeping. Establishing a session takes a lot of time an power. I use CoAP over UDP for that instead of MQTT. For hard wired sensors etc I use RS485 serial. I would like to be able to share RS485 serial lines using a protocol like UDP but have yet to find suitable software and I don’t have the time to write it. In the video the issue of message semantics was mentioned but not discussed. This is a big problem as, without standardized semantics, new types, or even just variations, of devices are tedious to integrate. The use of JSON at least helps with message syntax which is a start. As an aside, it is possible to make money out of Cloud Free IOT but it requires a permission based advertising approach rather than a surveillance based advertising approach. The big cloud companies seem to lack the imagination, or maybe the will, to do that.
“According to Elliot, the ESP’s new found abilities are something of a mixed blessing. While it’s difficult to overstate the impact these low-cost MCUs have had on DIY hardware, he believes that keeping things simple has its advantages. Rather than tasking the diminutive ESP8266 with running a web server and handling a dozen different tasks, he’d rather give each one a simple assignment and push the logistics of tying it all together to a higher level. With as cheap as they are, multitasking just doesn’t make a whole lot of sense.”
So true. I jumped on the ESP8266 bandwagon in 2014, had a ringside seat as some truly talented OS developers pried the lid off and gave us access to programming these $5 wonders, and marveled as people applied the things.
Also, much love to MQTT and the people who developed and shared implementations of it on different platforms.
And what I forgot to mention was the steady flow of idiots who wanted to implement Apache, WordPress and other inane uses on the ESP8266.
>Originally, Espressif intended for the ESP line to essentially be simple WiFi “modems” for other devices, complete with an AT command set.
Originally, in 2013, Espressif manufactured dumb SDIO wifi controllers for throwaway Rockchip powered $50 tablets. They called the chip ESP8089. Guess what is etched on ESP8266 die? ;-)
https://zeptobars.com/en/read/Espressif-ESP8266-wifi-serial-rs232-ESP8089-IoT
Ha covered ESP8266 running in ESP8089 mode in 2015.
Of course nowadays the ESP32 is the way to go as it costs little more than the ESP8266 and has greater capabilities, including lower power configurations. I love the architectural approach of using lots of cheap single purpose hardware rather than loading up everything onto one Pi or something similar. So, I use several cheap routers to partition my network and provide several levels of firewall, many ESPs to perform individual tasks and several Pis as MQTT brokers, web server, Node-Red platform etc. They are all bound together mainly with MQTT. I find that one function per piece of hardware is more flexible and resilient.
Wow, I did not know that; I was going by some early article that suggested that the ESP32 was not as frugal a “sleeper” as the ‘8266… but you’re right – the ’32 has more sleep modes AND a lower possible sleep current. Thanks.
Short version: Nobody wants in Internet of Things. Everyone wants an Intranet of Things.
Really pretty well information here we get, quality content has been provided
Every time I read a home automation article, I get fired up to do it. But then I sit down and think, what really needs to be automated? The only things I’d really like to automate are the lights and the thermostat, and both of those already have commercial implementations that aren’t too expensive. (Top rated smart thermostat out there isn’t the pricey Nest, it’s a $30 Honeywell).