Voice-Controlled Smart Home From The Foundation Up

Smart homes are becoming an increasingly popular way to automate one’s home, whether it’s turning on lights, closing blinds, or even feeding pets. But the commercial offerings often rely on an internet connection to reach servers in order to work, which invites a lot of privacy concerns for a large percentage of us as well as being inconvenient when the internet is down. Essentially the only way to have a privacy-respecting, self-sufficient smart home is to build one on your own from the ground up, which is exactly what [Xasin] has done with this project.

This build is based on ESP32 modules with a Raspberry Pi as a hub, but it’s not as simple as a MQTT implementation. Not only does the self-contained home automation setup not rely on any outside services, but a failure of the central Pi server will not impact the nodes either as they are configured to continue operating independently even without central control. This allows for a robust home automation implementation without a single point of failure, and also includes some other features that are helpful as well including voice control, all while retaining a core design philosophy that makes it relatively easy to build.

Not only is the build technologically impressive for its standalone capabilities and its elimination of privacy concerns, but [Xasin] also did an excellent job with the physical design as well, adding plenty of RGB and a hexagonal enclosure that gives it a unique look wherever its is placed. If you’re renting right now or otherwise unable to interface any automation with your current home, be sure to take a look at some projects that do home automation without making any permanent changes.

20 thoughts on “Voice-Controlled Smart Home From The Foundation Up

    1. Hah, why thank you!
      I would like to say it’s easy enough for someone else to replicate, but since I wrote it all… No clue how hard the code is for others to sift through ^^’

      Either way, all the necessary files are on GitHub, and I’d love to see what kinds of derivatives we’ll see from this project.
      I’m still working on a bit of user-friendliness, such as a BLE-based WiFi configuration instead of … Hard-coding things… <.<
      As well as being able to reconfigure the OTA HTTPS address to your own server.

      But as long as you have some HTTPS and MQTT server (can be as small as a Raspi with Let's Encrypt CA), you're good to go :D

      If you have any questions, feel free to send a message over on Hackaday.io!

    2. Hah, thank you!
      I’d love to see what kind of derivatives people make with our code and hardware as inspiration ^^

      Everything is free on GitHub for people to look into and make their own set.
      I am still working on some user-friendliness upgrades though – stuff like not needing to hard-code the WiFi data and instead use a BLE connection to set things up, as well as being able to reconfigure the HTTPS OTA used to download new firmware.

      But as long as you have a HTTPS and MQTT server (signed by the Let’s Encrypt R3 CA, but that can be changed), you should be able to get it going quite easily.

      If you have any questions, feel free to send over a message!

    1. Hi, Xasin, the author here!
      I did quite actively consider using something like a CAN bus network to interface things together.
      While it would be more sturdy, it doesn’t quite work out.

      The CAN bus transceivers still need some form of MCU, and the combination of CAN+MCU is actually about as expensive as some of the cheaper ESP32 modules – and you’d still tape a small computer to the ceiling, just one without WiFi.
      You also have to factor in the added cost of routing the CAN network through your home, which takes time and copper cables.

      Still, I personally also prefer wired networks over wireless – but in this case, WiFi just seemed more useful.

      1. Well, impressive work :)
        I want to create something similar, but with cables.

        Why is CAN more expensive? Because of the transceiver chips? CAN interface itself is even in blue pill boards.

        How do those ESP WLAN modules hold up during a lightning strike nearby? My house is somewhat exposed (on a high bank near a river, land lines still wired up to the roof), so a new DSL gateway every 12-14 months on average is kind of “normal”. I use the cheapest DSL modems in bridge mode right on the attic and go down in the cellar to the network cabbinet by POF.

        What a pity that there is no optical equivalent to a CAN bus.

        1. Adjusting this system to run on cables using similar ideas should be easy enough!

          There are a couple of reasons I didn’t go for CAN, actually.
          It is more expensive because most CAN tranceiver ICs are pricey. About 1.5€ in single quantity.
          That doesn’t sound like much, but considering that the ESP32 modules are just 3€ in total, and for CAN you will also need extra connectors and extra cables (cables are expensive over longer distances, and a CAN network would have to have continuous wiring to all the nodes that you want to connect, whereas you can draw power from any nearby outlet)…

          It’s a fair bit of extra work.

          Also, the CAN protocol is nice, but a bit unwieldy for the way I personally transmit data, like cleartext JSON.
          MQTT is just a lot more comfortable.

          Regarding lightning strikes I can’t say anything about the RF part.
          The 5V supply can, however, be protected with a bigger TVS diode and other suppressive elements.

          And I’m sure there is – laying down a glass fiber system with junction boxes doesn’t sound that hard, and if you were to swap the CAN transceivers with optical transmitters/receivers, nothing speaks against it!
          The protocol itself doesn’t rely on the electrical levels, only on 1/0 in a pull-down fashion (dominant 0), which can be done optically for sure.

  1. Yeah, I’m not going to make my house listen to me or anyone else. And the problems with the commercial offerings aren’t limited to just privacy. Longevity of service is a big one, too. More than a few services have bit the dust and left behind a bunch of bricked hardware.

    Anyway, I use a mix of Insteon (legacy) and Z-wave (new) with a local controller from Universal Devices that does the automation. It’s been working well for more than a decade. (Though I’ve had to change the controller once to support Z-wave.)

    With this home automation stuff it’s easy to miss that the power supplies in all the various little devices are easily blown. That’s illustrated to the unprepared when during a thunderstorm a bunch of things stop working. It’s worth it to invest in at least a basic type 1 and 2 Surge Protective Device at the main fuse box. And additional type 2’s at any other fuse boxes if there are any. That has kept my 100+ devices humming without mass brickings.

      1. You must mean the cold stand-by that I have collecting dust? That’s the other thing about Universal Devices. Their stuff is (or at least was) reasonably priced.

        In general I think the future is Wifi, as it resolves all the issues that these current devices have. But it’s just not there yet. While there’s an entire ecosystem of cool ESP32 stuff and the like, I haven’t seen any Wifi-based UL-listed wall switches yet. (They could be out there; I haven’t looked in a little while.)

        As to UL listing, that becomes important if the house ever burns down. If there is even a remote chance that an unlisted device started the fire, some insurance companies might take that as a cue to reduce the claim or outright disallow it.

        1. Great work, @Xasin! I think you’re on the right track here. Thank you for exposing all of your code and hardware designs. I can’t wait to see where this goes.

          And thank you, @aki009, for your comment on UL certification; that’s often overlooked and is one of my major hurdles in the HA arena. I think everyone’s PSU should be UL-listed for insurability, but the low-voltage side is fair game for hacking. If you’re hacking on mains powered lines and it starts a fire, your insurance carrier has every reason to deny any claim.

          The ability to leave my HA efforts in place and usable when the home sells is another pillar of mine. I’m okay offering a tutorial and leaving good documentation behind, but I don’t want to be on-call or on the hook when the new owner can’t turn on the garage light… One must also consider how intuitive the controls are; I call it the Father-in-law test. If he can’t figure it out without my help, it’s not good enough.

          I just returned a smart light because it couldn’t be dimmed locally; you had to use a phone app or integrate with an assistant service like Alexa. What the industry needs are smart wall switches with enough processing power to manage their smart lights locally, but with the ability to still be managed themselves. A hub could tell the switch to turn its lights on, the switch could tell the hub someone manually turned them off, and everything would still work fine without the hub.

          Insteon goes a long way in that direction, but has other limits, including longevity issues. Plus their protocol is proprietary and poorly documented. That said, their switches can directly control their connected lights and still respond to or initiate actions on other devices. They have a fair selection of devices, but I don’t like their single-gang light-fan combo solution as well as Lutron’s.

          I’d also like to see voice control integrated into a home intercom setup. The intercom providers are still two decades behind state of the art and are ripe for disruption, especially if someone can offer voice assistant throughout the home using pre-existing Cat-3 or Cat-5 wiring for retrofits. Even more so if it plays well with others and can do it all without needing Google, Amazon, or the internet in general.

          1. Hey, no problem!
            One of the reasons I do these projects is because I want to express some of my inspirations, and maybe allow others to learn and gain something from it as well <3

            There are a lot of interlinked projects coming up that will all sort of feed into improving the smart home, directly or indirectly, so it'll slowly grow alongside things.
            I'm talking giving it a proper website that will, hopefully, be a generalized interface to watch over any number of my ESP32 projects (health measurement, ping and heap watching, etc.), as well as to configure and control systems (i.e. turn lights on and off!)

            There's a BLE setup procedure underway, and some extensions, for example to connect the Smart Home to Octoprint and get 3D printer notes~

            Also, a database connection for better persistence and user management…
            Yeah, as one can see I am getting a bit carried away here ^^'

            There's a reason I did not bother with the AC/DC adapter in here.
            I want to drive bigger LEDs anyways, in the order of 30W or more, so building my own adapter made no sense. So instead I designed the board with a larger input voltage range in mind and let people find the adapter they need :D

            My system is… Definitely not "Father-In-Law" compatible, at least not yet, but that's because this is the most comfortable for me.
            No reason it can't be tweaked for other stuff though :D

            Regarding your management problem, that is a core aspect of what annoyed me with premade options – the reliance on a central service just to be able to turn them on and off!!
            In fact, I wrote a library for myself that does exactly what you described – it turns the ESP32 into a small server that keeps track of different parameters (Lights on/off, light color, etc.)

            You can configure these parameters through different connections (HTTP requests to the ESP itself, the MQTT server, BLE), and the updated state is propagated to other connections.

            Essentially, you can have a central server that can control all your smart home applications through this interface – but if that central server goes offline, each individual node is still fully operational through a bunch of different connections.

            It's also all JSON based, so really easy to understand. A BLE packet for example would look like "{"set":{"lights_on":{"value":true}, "light_color":{"value":"#FF0000"}}"

            Super easy to implement in any language that has JSON support :D

            Regarding intercom, the ESP here should actually baareeely be able to transmit and receive the audio simultaneously, in real time.
            Using a different compression scheme like LC3 (LowComplexity3), or by using both cores of the ESP just for audio, you might get full-duplex 48kHz audio.

            Establishing a basic intercom would be super easy by adjusting some of my code to pipe audio from one ESP into another, and in fact I basically did this with my DSKorder project already and streamed computer audio to the ESP :D

    1. That’s fine – the microphone is optional, after all.
      I even added an LED that is hard-wired to the power supply of the mic chip, and you can tweak the code to disable continuous voice activation and replace that with a button-press and “mic on demand”, sort of.

      Longevity also kind of is a problem here?
      If I stop maintaining it it’ll break down. But as long as I use it I’ll gladly tinker with it and keep it running :D

      Regarding the power supply, that’s variable here too. The board itself just needs anything from 5V to 24V, so you can buy a cheapo USB power supply, or a proper sturdy 100W 24V brick for a big light installation.

      I do have a TVS diode and a fuse on-board on the PCB though, maybe that helps a bit :>

      1. That’s great. The best thing about self-made is the tweaking part. I love it. (And I should have added earlier that I love the build.)

        From what I gather the power supply issues mostly hit the integrated devices that have the mains-to-5v (or something like that) implemented using minimal parts, and when those parts are sourced from places where they might not always meet spec, especially at the top edge of the envelope. An external power supply with TVS protection should get you there.

        Then there’s the thing, that as I have gotten older the amount of power conditioning and protection I design into everything has gone way up. Perhaps one day my house will be Carrington event proof, not that it’s the goal, but that’s where it’s headed.

  2. I’ve also been working on a similar project with the goal of privacy first voice controls! It’s been live in my home for over a year and integrates with Mozilla Webthings. I’m definitely going to take some notes from this project, especially the shell’s design.

    Here is a web-socket server wrapper around PocketSphinx that can provide sound files, it will perform TTS from a list of words and transmit back: https://github.com/malceore/whisper

    The Client, uses Snowboy hotword: https://github.com/malceore/voice-assistant-client

    1. Awesome, thank you so much for the links!!

      You’ll actually love hearing this, but the wakeword detection here is happening fully on the ESP32
      That means there’s no continuous audio transmission going on unless it detected the wakeword.

      Espressif actually has a Wakeword detection code using a simple neural network. Per default it has “Hi Lexin” as word activated – though it’s not that good and has plenty of false alarms…

      I might look at your links because they seem really useful, and maybe switching to a UDP audio stream and computer-based wakeword detection could be a bit more sturdy…

  3. To tag onto my earlier comments about UL listed Wifi switches, it seems that Leviton has added Wifi capability to some of its decora smart switches, e.g. the D215S. The price is similar to (just a bit higher) than competing Z-Wave Plus offerings. And it’s UL listed.

    There is also an ultra-low cost Wifi connected switch from Kasa, HS200, that claims to be UL listed. Less than half the price of most other offerings. It seems to be tp-link (of Ethernet networking products fame) related.

    Yay. Only question remains how well they will play without using any of the outside services. And how large the product enclosures are. The first generation Z-Wave Plus switches were very deep, creating problems for busy wiring boxes.

    1. Heh, I actually went down a bit of a rabbithole regarding these switches, or rather how to make one myself.
      There seem to be cheap 240V AC to 5V DC modules from Recom or similar which take care of the most critical part, generating the lower control voltage – for only 6€ a piece, completely potted in resin and PCB-mountable

      That, combined with a good relay or two running at 5V, and a ESP32-C3 on a LDO, should make for a sturdy system.
      With a properly designed PCB it’d be fairly cheap (relay and AD/DC Module cost 10€ together, but you can probably drive multiple relays on a single AC/DC PSU)

      Now I kind of want to make myself one – preferably as a secondary safety power switch for my 3D printer or something :P

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.