Last time on Minimal MQTT, we used a Raspberry Pi to set up an MQTT broker — the central hub of a home data network. Now it’s time to add some sensor and display nodes and get this thing running. So pull out your ESP-8266 module of choice, and let’s get going.
For hardware, we’re using a WeMos D1 Mini because they’re really cute, and absolutely dirt cheap, but basically any ESP module will do. For instance, you can do the same on the simplest ESP-01 module if you’ve got your own USB-serial adapter and are willing to jumper some pins to get it into bootloader mode. If you insist on a deluxe development board that bears the Jolly Wrencher, we know some people.
NodeMCU: Getting the Firmware
We’re using the NodeMCU firmware because it’s quick and easy to get running. But you’re not stuck with NodeMCU if you want to go it alone: MQTT has broad support. [TuanPM] ported over an MQTT library to the native ESP8266 SDK and of course there’s espduino, a port for an Arduino-plus-ESP combo. He also ported the MQTT module to NodeMCU that we’ll be using today. Thanks, [TuanPM]!
Anyway, back to our story. If you haven’t tried out NodeMCU on at least one ESP8266 module, you should give it a try. Here’s the painless way. Head over to NodeMCU custom builds and build yourself a custom firmware with only the modules compiled in that you need. Here, I’ve included dht, file, gpio, mqtt, node, tmr, uart, wifi, and ws2812. (I’ve also included this image as a download in the GitHub repo for this series.)
To upload the NodeMCU firmware to the ESP8266, I use esptool.py, which should work on any platform that Python runs on. Follow the instructions on the
esptool GitHub, or just
esptool.py --port /dev/ttyUSB0 --baud 57600 write_flash 0x00000 firmware.bin if you’ve done this sort of thing before. (Windows, use
COMx for your serial port. Mac people use
/dev/cu.usbserial-whatever.) If you have an ESP module without USB-flashing support hardware, you might have to pull
GPIO0 to ground on restart — you might have a “flash” button that does this. Again, the instructions at the
esptool GitHub should get you squared away.
NodeMCU: Getting Around
If you’ve never used NodeMCU before, don’t fret. It’s a little bit confusing at first because many functions run asynchronously. We’ll handle that when we’re looking at the code. Other than that, Lua is not a difficult language to learn, and you could do worse than to have Programming in Lua open in another browser tab.
The cool feature of NodeMCU is that it’s run through an interactive serial terminal, so plug up a USB-serial converter if your ESP module doesn’t already have one onboard, connect at 9600 baud, and you’ll be greeted by a friendly
> prompt. To get the module registered on your local WiFi network, try:
Wait around ten seconds and you should be set. Type
print(wifi.sta.getip()) to confirm that all’s well. We’ll upload a script to automate this before we’re done, but it’s good to see how simple and interactive things can be right from the start.
MQTT in NodeMCU
So let’s get a little experience with MQTT on the NodeMCU platform. The online documentation is pretty good, so open that up to follow along. The MQTT library centers around the
mqtt.Client() object and its methods,
close(). When you create the client, you can also register some callback functions that you’d like to be triggered by certain events, namely
message for when the client connects, goes offline, or receives a message.
If you’ve got your broker set up from last time, let’s send it some messages and monitor them on the broker. Open up a window on the Pi and type
mosquitto_sub -h localhost -v -t home/#.
The node’s MQTT client needs an ID, keepalive time in seconds, username and password, and a clean-session flag. Connecting to the server can be as simple as specifying its IP address, and publishing takes the usual topic, message, QoS, and retain flags. Here’s a minimal demo:
m = mqtt.Client("myNodeName", 120, "", "") -- blank user and password m:connect("192.168.1.49") -- my local broker's IP m:publish("home/test", "hi from node", 0, 0) -- no QoS, not retained
Bam! Your first MQTT publish from an ESP8266 in three lines of code. With the addition of some callback functions, we can make it user-friendly and start receiving MQTT data as well.
m = mqtt.Client("myNode", 120, "", "") -- blank user and password m:on("connect", function() print("connected") end ) m:on("message", function(client, topic, data) print(topic .. ": " .. data) end ) m:connect("192.168.1.49") -- my local broker's IP m:subscribe("home/#", 0) -- no QoS
The client code continually listens for messages or state changes, and sends the appropriate strings to the
Now if you send a message to the broker,
mosquitto_pub -h localhost -m "heya" -t home/test, you’ll see it printed out at the node’s serial terminal. There’s a bit more to say, but that’s the gist. You’ve got publish and subscribe functionality on the ESP8266. And if you’d like to subscribe to or publish data to the bigger Internet, there’s nothing stopping you. (Although note that everything you send to
test.mosquitto.org is public by default.)
m:close() m:connect("test.mosquitto.org") m:subscribe("hackaday/#", 0) m:publish("hackaday/mychannel", "your message here!", 0, 0)
Let’s work this base into a realistic two-node, home-automation example.
The WeMos module I used has DHT-11 and DHT-22 shields available. The sensor is hooked up to NodeMCU pin
D4, which is
GPIO2 in the ESP8266 datasheet. The NodeMCU DHT library docs for using the DHT units are short and sweet, with one function that you’ll care about,
Anyway, if you’ve got a DHT-11 or similar hooked up to the NodeMCU pin
GPIO2), it should just work.
print(dht.read(4)) returns five values:
status, temp, humidity, temp_decimal, humidity_decimal. Verify that, and you’re set. You can wrap it in a function like this:
temp = 20 function updateTemp() status, temp, humid, temp_dec, humid_dec = dht.read(4) end
If you don’t have a temperature sensor hooked up, but you’d like to play along, you can code up a random-walk temperature simulator in short order.
temp = 20 function updateTemp() temp = temp + math.random(3) - 2 end
print(updateTemp()) a few times to verify that it’s working.
So, whether real or fake, we’ve got temperature data stored in a variable. We already know how to set up an MQTT client and publish data to a given topic. What’s left? Making sure it happens every few seconds, and wrapping everything up nicely into a file to upload to the node.
As mentioned above, NodeMCU runs many commands asynchronously. That means that we can’t just write code that says “do this, wait for a return value, and then do that” like you often do in C or C++, because the first function doesn’t block. Everything happens almost at once. The upside of this is that we call
m:subscribe() and it just runs in the background. The downside is that when we don’t want everything happening at once, we need a timer.
Say we want to publish the temperature of our sensor every ten seconds. We set a repeating timer, for ten seconds, that calls our publishing function, and we’re done. The timer command runs as soon as it is called, but the function that the timer contains is only called periodically. Let’s test that out.
-- assumes MQTT client setup and connection function post_temp() updateTemp() m:publish("home/outdoors/temperature", temp, 0, 1) end tmr.alarm(0, 10*1000, tmr.ALARM_AUTO, post_temp)
You should now see the current temperature (real or imaginary) showing up on the topic on your broker every ten seconds. I also set the “retain” bit for the temperature report, because you probably will only want to report a room temperature once every few minutes, but when you connect with a client you’ll want to see the last value without waiting. (And note that the timer is set in milliseconds, hence multiplying by one thousand.)
In the next article, we’ll talk about data logging and graphing and all sorts of fancy server-side stuff. For now, let’s make a quick device that lets us gauge the outside temperature on a human scale, encoded in color: blue is cold, green is brisk, yellow is warm, and red is hot. To do so, we’ll connect a WS2812B LED to another ESP8266 node, read the temperature from the broker, and display it.
There’s actually not much to add to the display node. We already know how to subscribe to a topic and receive data as it comes in. All that’s left is a display function, and hooking it into the received-message action. A fleshed-out version is available in
mqtt_display_node.lua, but here’s the important bits:
function display_temp(data) data = tonumber(data) r = math.min(2*math.max(data-20, 0), 40) b = math.max(40-2*data, 0) g = 20 - math.min(math.abs(20 - data), 20) ws2812.writergb(ws2812b_pin, string.char(r, g, b)) end m:on("message", function(client, topic, data) display_temp(data) end )
Writing out to a WS2812 pixel (or chain) is as simple as calling
ws2812.writergb() with a string that corresponds to the RGB data that you’d like to set for each pixel in the chain. Here, we’re just setting a single one. Otherwise, the argument would look something like
string.char(r1, g1, b1, r2, g2, b2, ... ).
That’s it for the display node.
Typing code directly into the nodes clearly won’t do. We need to store our code on the device. NodeMCU has an emulated filesystem that we can use to upload our code to. You can call these files manually with the
dofile("filename") command, but you can also use the fact that the system automatically runs a file called
init.lua on reboot to kick everything off. My initialization files usually set up my WiFi configuration, wait until it’s connected, and then start running the code that makes the device do what it does. (Examples in the GitHub.)
Getting your Lua source files onto the ESP8266 is best done by using uploader utilities that essentially just type them out across the serial line. I use nodemcu-uploader.py myself, because it works nicely with any editor and a command-line build environment. You’ll end up going back and forth between trying something out on the node in the serial console, and writing down what works in a file. Occasionally, you’ll upload this file to the device, maybe restart it, debug, and repeat. This can be a little clumsy, so we can see the appeal of something more integrated like esplorer. Try ’em out and see what works for you.
That’s it for now, but it’s a lot. You’ve got a central home server running, with a temperature (and humidity) logging node that connects to it, and another that pulls out the temperature and displays it on an RGB LED. There are a million ways to elaborate out on this simple system. Motion sensors, light sensors, and relay- or SSR-drivers could flush out a complete home automation solution. If you don’t already have a new-Hackaday-article notifier on your desktop, you probably need one.
Next time, we’ll have a look into server-side extensions that run on the Raspberry Pi broker that we set up last time: this gets us data logging, web pages, and weather forecasts. Following that, as a final installment, we’ll take care of some (cool) loose ends, like connecting your cellphone and other human controls into this whole system, and thinking a little bit about security.