In this installment of Minimal MQTT, I’m going to cover two loose ends: one on the sensor node side, and one on the MQTT server side. Specifically, I’ll tackle the NodeMCU’s sleep mode to reduce power and step you through bridging MQTT servers to get your data securely out of your home server and into “the cloud”, which is really just other people’s servers.
If you’re just stepping into this series now, you should really check out the other three posts, where I set up a server, then build up some sensor nodes, and then flesh-out a few ways to control everything from your phone or the web. That’s the coolest material, anyway. This last installment just refines what we’ve built on. Let’s go!
Using Less Power
The ESP8266 units are awesome. They pack a decent amount of computing power in a tiny package with WiFi. That makes everything easy. Everything except battery-operated use, that is. Without spending some effort on taming it, the ESP8266 is a power hog.
Let’s quantify that really quickly. I’ve connected the ESP8266 to my power supply through a roughly one ohm resistor, and am measuring the voltage across it; every millivolt on the scope is a milliamp of current passed through the ESP. The first scope shot shows the high current draw during startup, the second zooms in on a standby pattern, and the third shows startup again with the voltage scale zoomed out so that you can see the nearly 350 mA peaks.
When using WiFi, the ESP8266 on my desk draws a roughly constant 70 mA from the 5 V supply. (As many people have noticed, though, it occasionally peaks up to 350 mA for milliseconds at a time, which is what causes brownouts with poor power supplies.) While it’s connecting to the base station, it uses quite a bit of power. Once it has connected, it settles down to a cruising altitude of roughly 15-20 mA, but still exhibits the bumps and spikes. We’ll call it 20 mA on average.
There’s not all that much we can do about the WiFi standby and transmission current requirements — physics demands power to make electromagnetic waves. For an application like a long-running temperature and humidity monitor, however, we can reduce the duty cycle — taking measurements every ten minutes or so instead of every ten seconds and powering down into sleep mode for most of the time. The less time the ESP spends awake, the longer our batteries will last.
To put all of this in perspective, I’m going to frame everything in terms of a 1000 mAh battery because it makes the math easy and is about right for a small Li-Ion pack. If it’s just idling, that’ll be 50 hours — we’ll call it two days. Sending data will only shorten that lifetime. That’s no good.
But if we can take infrequent reports and turn it off in-between, we’ll save on most of that constant 20 mA. Imagine that a connection and temperature report takes about ten seconds, and measurements are made every ten minutes. Because this includes the high-current connection phase, let’s assume something like 50 ma average across the ten seconds. That would only be 20 hours of run-time if it were on constantly, but since it’s on only 1/60th of the time, it could last as long as 1200 hours, or 50 days. That’s getting somewhere.
But we can’t quite turn the ESP8266 off: what would turn it back on again? Instead, it can go into a deep sleep mode which shuts everything down except an alarm clock that will wake it back up. When my ESP8266 is in deep sleep, it only uses around 145 microamps. (I’ve seen reports as low as 78 microamps, and I assume the difference is standby consumption of the built-in voltage regulators on mine. Or mis-measurement.)
Putting an ESP8266 with the NodeMCU firmware into deep sleep mode is very easy — just use the
node.dsleep() function and tell it how many microseconds you’d like it to sleep. Easy enough.
There’s one catch, with two consequences. To wake up from deep sleep, the ESP8266 needs to reset itself, and to do this pin
D0 / GPIO16 has to be connected to the reset line. (It’s the one labelled
wake in the pinout diagram.) This is necessary because the only thing left running in deep sleep mode is the real-time clock (RTC). Just before entering sleep mode, the RTC is configured to toggle the
D0 / GPIO16 after the elapsed time.
Connect your node to your computer and test it out right now. Don’t forget to hook up a wire from
D0 to the reset line. Type in
node.dsleep(10*1000000) and watch the chip become entirely unresponsive for ten seconds and then spring back to life.
If you followed along, you just experienced the second consequence: the chip completely resets itself after deep sleep. This means that each time it wakes up, it’s forgotten everything, and it is starting over again with
init.lua and a clean slate. For something like a simple sensor node that does the same thing each time it wakes up, this is no problem. But sometimes our processes will need to store some state.
Life for the ESP8266 is like the movie “Memento”. And if you saw that movie, you’ll remember that the solution is to leave yourself good notes. Short of adding non-volatile memory to your setup, you can abuse MQTT’s persistence feature to stash data for reboots. Designate a topic (or two, or more!) to store the data that should persist after deep sleep. Before going to sleep, write the data to the topic using the “retain” flag and wait a few hundred milliseconds for the message to be delivered. On waking back up and connecting to the MQTT network, subscribe to
home/powertest/memento. The first value read from that topic is going to be the data left over from the last session.
As an aside, there is a function,
node.bootreason(), that you could in principle test to see if the chip came out of a power-up or a deep-sleep reset, but there is a hardware bug and workaround that prevents this from functioning correctly. The upshot is that you always get the same return code, 2, and that’s essentially useless. Bummer.
A workaround for the
bootreason() bug can be found in the
powertest example in my MQTT GitHub repository. That code also demonstrates setting up
command channels to talk to the node. To play around and see how it works, open up a window to listen to the MQTT traffic:
mosquitto_sub -t home/powertest/# -v -h 192.168.1.49 and then send a command to the node:
mosquitto_pub -h 192.168.1.49 -t home/powertest/command -m "sleep".
To conclude: the trick with using deep sleep on the ESP8266 when running NodeMCU is to write your code so that it does the right thing on reset, and stores any long-run data somewhere. And don’t forget to connect the
wake pin to the
reset pin — the chip will stay asleep until you pulse the
For the temperature sensor nodes, it’s worth mentioning that they’re going to be battery-drainers in the standard configuration. My powered-up DHT-11 module draws a continuous 486 microamps — let’s call it half a milliamp. That’s a big dent in our energy budget now that we’ve cut the duty cycle down by sleeping. We got our average energy consumption down from around twenty milliamps to just under one milliamp. It’d be a shame to spoil it by leaving an unnecessary peripheral device burning half of that away.
The same often goes for other peripherals, sensors, and LEDs. Get rid of them if you don’t need them, and reduce the on-time to the bare minimum for those that are necessary. A useful method is to power the peripherals through a PNP transistor or P-MOSFET and turn that transistor on using a GPIO pin on the node.
In the end, you should be able to get a few months from batteries if you’re careful and making infrequent measurements. If you need the node to be always on, or need longer life, a 2.4 GHz radio module and a MQTT gateway might be just the ticket.
Thinking About Security
Finally, I got savaged in the comments at the beginning of this series, because some readers mistakenly thought that, just because something can be put on the Internet, it must. And this would mean that our sensor nodes need encryption. But that’s the wrong approach, in my opinion.
The system we’ve built up so far — a Raspberry Pi talking over a home WiFi router to a bunch of ESP-based nodes — is about as secure as you could ask for. All communications are encrypted and authenticated by your WiFi router, and constrained to the local area. Indeed, half of the reason to run your own MQTT broker at home is that your data never leaves the confines of your own four walls unless you need it to. Not connecting devices to the Internet that don’t need to be connected to the Internet is security rule number one. But don’t take my word for it, see what Shodan — troll and entertaining presenter Dan Tentler has to say.
As it stands, everything is (hopefully) behind your router’s firewall and encrypted in transit by WPA2 with a good strong random password, right? And you turned off UPnP? Great. That’s good enough for any but the most paranoid. Or you can pound your head against encryption on the ESP8266 if you want to, but since Espressif has announced that they’re not coming out with TLS 1.2 support until 2017, you’re probably wasting your time.
Configure MQTT to Bridge
Still, you might want to get your data out into the big wide world, or you might want to control your home system when you’re away. There are tons of neat services out there that speak MQTT. And if the data is even the least bit sensitive, you’re going to want to encrypt it in transit.
MQTT broker bridging does exactly what we need in this case: it duplicates topics between brokers. By bridging, we can take our home network which is already secured through our WiFi’s WPA2 encryption and access control, and securely forward data on to another MQTT broker “in the cloud”. In this case, we’ll use the test.mosquitto.org server that we used in the first installment.
To bridge a topic, we need to add a new configuration file on our home Pi-based MQTT server. In this example, I’m calling the file
connection TestBridge address test.mosquitto.org:8883 topic home/outdoors/temperature both 0 bridge_cafile /etc/mosquitto/ca_certificates/mosquitto.org.crt
The “connection” names the bridge and provides a client id for the remote broker, so pick something unique and descriptive. The “address” is the remote broker’s address, and in this case port 8883 is the port that
test.mosquitto.org uses for SSL/TLS connections.
For every topic that you’d like forwarded between the two brokers, you’ll need a “topic” line. In this case I’m forwarding my temperature topic in “both” directions (“in” and “out” are the other choices) and using QoS 0. See the manual for
mosquitto.conf for all the bridging options. Since the bridge is really functioning as a client on the remote broker, you’ll find that most everything works intuitively.
Finally, since we’re using TLS to secure the connection, we need the remote broker’s certificate authority file. You’ll have to get this from whatever remote service you use. In this case, I downloaded it here and saved it in the certificate authority directory that came with the Mosquitto installation.
sudo /etc/init.d/mosquitto restart everything should be good to go. Try publishing to the temperature topic on either server and you’ll find it appearing in both places. Voilà, your local network has been securely forwarded on to the cloud, without exposing the individual devices that lie behind your Pi-firewall. That’s the easy way to do IoT security. And if you bought one of those new Raspberry Pi 3s with WiFi built in, you could even run the IoT network on the built-in WiFi.
I really like MQTT, if you haven’t guessed. It’s a data-agnostic transport mechanism with a little bit of memory that allows you to do a lot with a little. You can very quickly hack together a system that lets one device control another, integrate data from multiple sensors, and wrap it all up nicely with a web interface (for instance). It’s an open standard, with an implementation for any programming language or platform you can think of.
There are a lot more ways to expand this system outwards. We haven’t yet touched on storing, analyzing, or summarizing the data — that’s the job of applications on your server that can listen in as clients to specific topics. An MQTT-to-REST bridge is a few lines of Python code, and again you can encrypt it outbound however you like. Once you start writing server-side client scripts, you’ll find that creating triggers that depend on other inputs are pretty easy and soon you’ve got your own, very flexible version of IFTTT.