A Lightweight Smart Home Server

Working towards automating a few things in a home often seems simple on the surface, but it’s easy for these projects to snowball into dozens of sensors and various servos, switches, and cameras strewn about one’s living space. The same sort of feature creep sneaks into some of the more popular self-hosted home server platforms as well, with things like openHAB requiring so much computing power that they barely function on something like a Raspberry Pi. [Paulo] thought there should be a more lightweight way of tackling a project like this, and set about building his own smart home server with help from some interesting software.

The project is based around the Dirigera hub from Ikea, partially because [Paulo] is planning to use other smart home devices from Ikea as he can easily find them where he is, and also because these devices tend to use Zigbee, a non-proprietary communications standard. This means that if he ever wants to swap out the hub for another one in the future, it won’t be difficult to do. From here the major hurdle is that using the default software from these devices is fairly limiting, so [Paulo] reached for a Raspbee 2 Zigbee gateway for use with a Raspberry Pi and an extremely lightweight and customizable web server called Mako to make this happen. Using Lua as the high-level language to tie everything together he was able to easily deploy the server to control the Ikea hub and devices and automate them in any way he sees fit.

While it is true that software like openHAB and others already exists to do virtually any home automation task that could be imagined, if you’re looking to do something with a bare minimum of computing power something like [Paulo]’s solution is likely going to be the fastest and most reliable method of getting a few things automated around the home. If you’re looking for something completely open source and built from the ground up, though, we have seen a few alternative smart home solutions like this one which don’t rely on any proprietary hardware or software, but do take a little bit more effort on the user’s part.

Reinterpreting The Lua Interpreter

The idea behind Lua is a beautiful one. A simple and concise syntax offers almost all of the niceties of a first-class language. Moreover, a naive implementation of an interpreter with a giant switch case can be implemented in an afternoon. But assembly is your go-to to get decent performance in a JIT-style interpreter. So [Haoran Xu] started to ask himself if he could achieve better performance without hand-rolled assembly, and after a few months of work, he published a work-in-progress called LuaJIT Remake (LJR).

Currently, it supports Lua 5.1, and on a smattering of 34 benchmarks, LJR beats the leading fastest Lua, LuaJIT, by around 28% and the official Lua engine by 3x. [Haoran] offers a great explanation of interpreters that provides excellent background and context for the problem.

But the long and short of it is that switch cases are expensive and hard to optimize for compilers, so using tail calling is a reasonable solution that comes with some significant drawbacks. With tail calls, each case statement becomes a “function” that is jumped to and then jumped out of without mucking with the stack or the registers too much.

However, the calling convention requires any callee-saved registers to be preserved, which means you lose some registers as there is no way to tell the compiler that this function is allowed to break the calling convention. Clang is currently the only compiler that offers a guaranteed tail-call annotation ([[clang::musttail]]). There are other limitations too, for instance requiring the caller and callee to have identical function prototypes to prevent unbounded stack growth.

So [Haoran] went back to the drawing board and wrote two new tools: C++ bytecode semantical description and a special compiler called Deegen. The C++ bytecode looks like this:

void Add(TValue lhs, TValue rhs) {
  if (!lhs.Is<tDouble>() || !rhs.Is<tDouble>()) {
    ThrowError("Can't add!");
  } else {
    double res = lhs.As<tDouble>() + rhs.As<tDouble>();
    Return(TValue::Create<tDouble>(res));
  }
}
DEEGEN_DEFINE_BYTECODE(Add) {
  Operands(
    BytecodeSlotOrConstant("lhs"),
    BytecodeSlotOrConstant("rhs")
  );
  Result(BytecodeValue);
  Implementation(Add);
  Variant(
    Op("lhs").IsBytecodeSlot(),
    Op("rhs").IsBytecodeSlot()
  );
  Variant(
    Op("lhs").IsConstant(),
    Op("rhs").IsBytecodeSlot()
  );
  Variant(
    Op("lhs").IsBytecodeSlot(),
    Op("rhs").IsConstant()
  );
}

Note that this is not the C keyword return. Instead, there is a definition of the bytecode and then an implementation. This bytecode is converted into LLVM IR and then fed into Deegen, which can transform the functions to do tail calls correctly, use the GHC calling conventions, and a few other optimizations like inline caching through a clever C++ lambda mechanism. The blog post is exceptionally well-written and offers a fantastic glimpse into the wild world of interpreters.

The code is on Github. But if you’re interested in a more whimsical interpreter, here’s a Brainf**k interpreter written in Befunge.

Open Source LXI Tools Free Us From Vendor Bloat

LXI, or LAN eXtensions for Instrumentation is a modern control standard for connecting electronics instrumentation which supports ethernet. It replaces the older GPIB standard, giving much better performance and lower cost of implementation. This is a good thing.  [Martin Lund] has created the open source lxi-tools project which enables us to detach ourselves from the often bloated vendor tools usually required for talking LXI to your bench equipment. This is a partial rewrite of an earlier version of the tool, and now sports some rather nice features such as mDNS for instrument discovery, support for screen grabbing, and a LUA-based scripting backend. (API Link)

SCPI or Standard Commands for Programmable Instruments is the text-based language spoken by many instruments, allowing control and querying of an instrument. Just to be clear, SCPI is not at all a new thing, and older instruments that have GPIB or RS232 connectors, still could talk SCPI. lxi-tools is not for those. Some instruments can also be very picky about the formatting of commands, especially if they’re buggy, so the ability to interactively debug commands is very desirable. It is quite possible to make poor use of SCPI commands in your test script and end up with tests that just take far longer to execute that they need to. lxi-tools has a benchmarking tool too, which helps you to dig in and find out where all the time is going and make suitable adjustments.

We’ve not seen much about LXI on Hackaday, but we did cover using PyVISA for dealing with SCPI-over-GPIB in python.  If you have an older instrument  with GPIB and you don’t want to sell a internal organ to pay for a USB adaptor, here’s one you can make yourself.

Dynamicland Makes The Whole Building The Computer

Every once is a while a research project comes along that has the potential to totally shake up computing and what it even means to interact with a system. The project Dynamicland.org, is a result of [Bret Victor]’s research journey over the years, looking into various aspects of human computer interaction and what it even means to think like a human.

One of the overhead projectors tied to a realbox
In Realtalk, paper is your programming medium

Dynamicland is an instantiation of a Realtalk ecosystem, deployed into a whole building. Tables are used as computing surfaces, with physical objects such as pieces of paper, notebooks, anything which can be read by one of the overhead cameras, becoming the program listing, as well as the user interface. The camera is associated with a projector, with the actual hardware hooked into so-called ‘Realboxes’ which are Linux machines running the Realtalk software. Separate Realboxes (and other hardware such as a Raspberry Pi, running Realtalk) are all federated together using the Realtalk protocol, which allows communication from hardware in the ceiling, to any on the desk, and also to other desks and computing surfaces.

Realtalk itself is described as an environment for authoring and using computation media. The Realtalk system provides a language extension to Lua. Together these form a domain-specific language. Realtalk is also a kind of reactive database, which means that the emphasis is on the flow of data and connections between data producing things, and data consuming things. For a bit more explanation of how reactive programming can be used with modern relational databases, check out this article on the subject.

For a good overview of how this works in practice, from a programming perspective, checkout [Omar Rizwan]’s article about his ‘Geokit’ project. Another interesting read is the work by [Andrés Cuervo.]

Continue reading “Dynamicland Makes The Whole Building The Computer”

Slaying Dragons In Notepad

We all have our favorite text editor, and are willing to defend its superiority above all other editors by any means necessary. And then there’s Notepad. But what Notepad may lack in text manipulation features, it compensates with its inconspicuous qualities as a gaming platform. Yes, you read that correctly, and [Sheepolution] delivers the proof with a text-based adventure game running within Notepad.

What started out with [Sheepolution] jokingly wondering what such a game may look like, ended up as an actual implementation as answer to it. Behind the scenes, a script written in Lua using the LÖVE framework — for which he also created an extensive tutorial — monitors the state of several text files that make up the game world. Each location is a separate text file to open in Notepad, showing the current state of the game, telling the story with text and ASCII art, and offering choices to the player. The game is played by modifying and saving those text files, which the script then processes to push the gameplay forward by simply updating the content of those files with the new state. Check out the game’s trailer after the break to get a feel of what that looks like.

Unfortunately, Notepad itself doesn’t automatically reload the file when its content changes, so to provide a smoother gaming experience, [Sheepolution] modified the open source implementation Notepad2 to work around this, and bundled it as part of the game’s executable. Initially, he even added animations to the ASCII graphics, but in the end decided against most of them to avoid constant disk writes and race conditions caused by them.

Sure, this is no Game Boy emulator in a text editor, and it may not be as groundbreaking as Notepad’s latest feature, but it’s always amusing to see alternative uses for well-established tools.

Continue reading “Slaying Dragons In Notepad”

LuaRadio Gives Insight Into SDR

In theory, you shouldn’t need any help to develop a software-defined radio (SDR) application. But in real life you really don’t want to roll your own code every time to read the IQ samples, perform various transformations on them, and then drive audio output. At worst, you’ll use some libraries (perhaps GNU Radio) but usually, you’ll use some higher-level construct such as GNU Radio Companion (GRC). GRC is a bit heavyweight, though, so if you’ve found it daunting before, you might check out some of the material on the LuaRadio website.

We’ve looked at LuaRadio several years ago, but it has undergone a lot of changes since then and has some excellent documentation. Like Lua itself, LuaRadio emphasizes fast scripting. It supports quite a few pieces of common hardware and nearly anything that feeds data through a soundcard.

Continue reading “LuaRadio Gives Insight Into SDR”

Compiling NodeMCU For The ESP32 With Support For Public-Private Key Encryption

When I began programming microcontrollers in 2003, I had picked up the Atmel STK-500 and learned assembler for their ATtiny and ATmega lines. At the time I thought it was great – the emulator and development boards were good, and I could add a microcontroller permanently to a project for a dollar. Then the ESP8266 came out.

I was pretty blown away by its features, switched platforms, except for timing-sensitive applications, and it’s been my chip of choice for a few years. A short while ago, a friend gave me an ESP32, the much faster, dual core version of the ESP8266. As I rarely used much of the computing power on the ESP8266, none of the features looked like game changers, and it remained a ‘desk ornament’ for a while.

About seven weeks ago, support for the libSodium Elliptic Curve Cryptography library was added. Cryptography is not the strongest feature of IoT devices, and some of the methods I’ve used on the ESP8266 were less than ideal. Being able to more easily perform public-private key encryption would be enough for me to consider switching hardware for some projects.

However, my preferred automated build tool for NodeMCU wasn’t available on the ESP32 yet. Compiling the firmware was required – this turned out to be a surprisingly user-friendly experience, so I thought I’d share it with you. If I had known it would be so quick, this chip wouldn’t have sat on my desk unused quite so long!  Continue reading “Compiling NodeMCU For The ESP32 With Support For Public-Private Key Encryption”