If you read Hackaday, it is a good bet you’ve heard of MQTT — Message Queueing Telemetry Transport. If you’ve not used MQTT before, you should check out Ably’s [Kayla Matthews’] post entitled MQTT: A Conceptual Deep Dive paper. She does mention their MQTT protocol connector at the end, and has a few notes about Ably’s products, but most of the post is a normal white paper and has a lot of good info.
MQTT’s claim to fame, of course, is that it is very tiny and is made to minimize power consumption compared to heavier-weight protocols. When you are trying to provide or consume data from a device that has to last a year on a coin cell, MQTT is your friend.
What we liked about the white paper is that it covered the kind of architectural decisions you have to make when designing a system. There’s a section titled “When might you use MQTT?” and another titled “When shouldn’t you use MQTT?” Towards the end, the post even covers using Mosquitto, MQTT.js, and MQTTnet.
[Elliot Williams] did our coverage of MQTT in his Minimal MQTT series. We’ve also seen a lot of tutorial projects like this doorbell.
Just about to do a review for a client on her country 432MHz network and options such as data collisions, what fun, good timing, thanks for post :-)
“When you are trying to provide or consume data from a device that has to last a year on a coin cell, MQTT is your friend.”
No, when you want that, you ditch MQTT because it still has a lot of overhead and use some proprietary thing.
There is a smaller version called MQTT For Sensor Networks (MQTT-SN). Specs here (PDF) http://www.mqtt.org/MQTTs_Specification_V1.0.pdf
But overall, MQTT has a lot of things it needs to do. This is why, at least on hobby level, people use some proprietary format to send the data to/from the battery powered node to a gateway that then bridges to MQTT. I did it like that too.
^— This. Not everything is TCP/IP or needs to be TCP/IP, especially if you want to make it truly low power. MQTT-SN isn’t really better in that regard.
Spoken like a true pair of paper code jockeys who clearly haven’t read the SN spec and with zero experience in the real world 🙄. Hilarious how you think dictionary subscriptions and UDP transport outweigh the loss of features which require more power to replace at the application layer.
I did read the -SN spec when I was deciding what to use for my sensor network and I am using regular MQTT for everything that is not battery powered. But for the link between battery powered nodes i stick to something a lot simpler which requires less power and less memory.
That does not mean the -SN does not have its place, it does. But it is trading up simplicity and energy efficiency for higher flexibility and extra resource requirements.
I’m with you on this one. MQTT is best (read: easiest) when paired with WiFi, and that means very short duty cycles if you want to run on battery at all. A plugged-in whatever-to-WiFi gateway can be made for a few bucks, though, and translate things over to MQTT. Run low power on whatever radio.
Still, I have a big red button on my desk that, when pressed, wakes up an ESP8266 and broadcasts a couple MQTT messages to the server before going back to sleep. It’s on a LiPo of medium size, and I only recharge it once per year or so. But I only use it a couple times per day. And that’s not a coin cell.
MQTT-SN basically trades topic names for numbers, and makes a couple more simplifications. If you’re still using WiFi, the trivially shorter messages don’t really matter much compared to the pairing-with-the router overhead, IME. (Admittedly, I haven’t really tried to optimize that step.) The basic idea — make a shorter protocol that’s easily transferrable over to MQTT — is a good one, though.
From the paper:
“Notably, MQTT uses plain text to transmit security credentials and does not otherwise provide support for authentication or security functionality. That’s where the SSL framework comes in, helping protect transmitted information from being intercepted or otherwise tampered with”
Maybe they can use a RUSTy tool to speed things up?
https://www.zdnet.com/article/a-rust-based-tls-library-outperformed-openssl-in-almost-every-category/
MQTT is easy to setup and use.
I have had it running for 2 years controlling lights (15) and blinds (4) in a room.
I have a video detailing how I did it for those interested.
https://hackaday.io/project/46428-room-control
Sorry for my droning voice . . . . . .
I made a practical MQTT demonstration on FRDM-K64F and FRDM-K66F boards:
https://www.youtube.com/watch?v=-_CV2aXHyjw
I wrote this document: http://www.utasker.com/docs/uTasker/uTasker_MQTT.pdf
and have used it together with mbedTLS for TLS encryption.
MQTT client is integrated in the open source project at https://github.com/uTasker/uTasker-Kinetis
A few months ago, i was in the need of a solution to monitor the on/off status of a fault led, probing it every 15s, during several weeks. The goal was to be able to store the collected data, and extract later any “on” status (and drawing a curve in case of repeated events).
I never used MQTT, but thought it would probably be the way to go. But i haven’t be able to find any free MQTT service provider to store collected data, and it seemed to require to write some code/application on the server side to store data in a SQL database (maybe i’m wrong or just haven’t found the right provider?).
Finally, i’ve discovered Adafruit IO (http://io.adafruit.com) and was more or less able to achieve my goal, using Adafruit IO Client library for Arduino. The only problem was that the curve widget was not very good at displaying lot of data and lack the ability to zoom in a portion of data. But well, it did the trick anyway.
Does anybody know if it would have been easily achieve with MQTT?
If you need to transmit snipits of information between multiple devices MQTT shines.
When I say devices this can also be a python, java, lua, etc script.
In your case something connected to your light, as soon as the status changes it publishes the state.
Then a script subscribed to that MQTT topic, when a message comes in act accordingly ie add SQL entry.
You would also be able to have a web page that will tell you the state of the light. With “last will and testament” feature that last message is sent to a new subscribing device.
You can easily have your own MQTT server on any Linux machine. I have one on a Raspberry PI Zero W ($10). One command: “apt install mosquito” and your away.
Thanks for your reply.
I was expecting to find some MQTT services in the cloud as easy to use than Adafruit IO: in a few clics, i can create a feed, and associate it to a provided dashboard/widget. Done.
Having to set up a server with some scripts, a database, and creating web pages to display data is quite more involving.
I’ve used MQTT for my own and customer projects and it has several issues:
* Using MQTT (and TLS, as you would) it’s far from lightweight (for something like CAT-NB where every byte matters it’s a no-go)
* Its QoS levels are poorly defined: i.e. re-transmitting unacknowledged packets on a healthy TCP connection is discouraged (MQTT 3.11) or forbidden (MQTT 5)
* Most major providers (AWS, Azure, GCloud) don’t have support for QoS2 and only support MQTT as a wrapper for their native REST-based API.
The Google comparison article mentioned within this comparing HTTP vs MQTT is good and matches my real world experiences (https://cloud.google.com/blog/products/iot-devices/http-vs-mqtt-a-tale-of-two-iot-protocols)
I have developed real work enterprise iot products with both MQTT and HTTP and I still think HTTP should not be discarded so quickly for applications at the outset. When trying to install devices at customer locations through their IT team, it’s often a much easier conversation to have 443 HTTPS urls/ports opened at a firewall than MQTT. Also in the case of OTA upgrades, a HTTPS file storage location is normally needed anyway!
Development at a device level for MQTT or HTTP/S appears to be much of a muchness now with good support for both. But I’d caution looking at MQTT at the cloud level. Often a decision will be made to use a cloud provider as the broker (Azure IOT Hub, AWS IOT, cloud mqtt, etc.) but the problem I’ve seen (had) with this is that you become quickly locked into a pretty critical piece of infrastructure. Sure you can migrate away, but the plan to do this is challenging, especially if you implement secure connections with provisioned token exchanges. Using HTTPS connections allows server selection or larger architecture changes with a DNS change at a URL with limited to no change at a device.
The thing is, you can use virtually anything for your IoT-project if you are not restricted by energy and/or bandwith.
But this is the point: if your ARE restricted, then you’ll go the route of a custom protocol anyway up to the point where you have plenty of energy and bandwith (in words: a gateway) and from there you are free to use anything.
So yeah – your “real world experience” is obviously based on devices connected to the grid – not battery powered devices. My real world experience is that MQTT is useless if you need very small data packets (like max 16 – 48 byte per packet) for your battery powered device to work properly for 10+ years in the field.
Actually my “real world experience” is 100k’s of well known, publicly available, battery powered devices which either used Zigbee or Wifi (2x product variants).
For battery device the protocol for the application layer is actually less relevant than the underlying transmission protocol. WIFI is a hog of battery life and MQTT layered on top is less relevant battery compared to the WIFI protocol itself. Zigbee is by far the superior protocol for battery life (when compared to WIFI in the same 2.4 GHZ freq). Mostly due to retries, or holding open connections with extended periods for check in to the router.