Creating Black Holes: Division By Zero In Practice

November 21, 2018 by Sven Gregori 31 Comments

Dividing by zero — the fundamental no-can-do of arithmetic. It is somewhat surrounded by mystery, and is a constant source for internet humor, whether it involves exploding microcontrollers, the collapse of the universe, or crashing your own world by having Siri tell you that you have no friends.

It’s also one of the few things gcc will warn you about by default, which caused a rather vivid discussion with interesting insights when I recently wrote about compiler warnings. And if you’re running a modern operating system, it might even send you a signal that something’s gone wrong and let you handle it in your code. Dividing by zero is more than theoretical, and serves as a great introduction to signals, so let’s have a closer look at it.

Chances are, the first time you heard about division itself back in elementary school, it was taught that dividing by zero is strictly forbidden — and obviously you didn’t want your teacher call the cops on you, so you obeyed and refrained from it. But as with many other things in life, the older you get, the less restrictive they become, and dividing by zero eventually turned from forbidden into simply being impossible and yielding an undefined result.

And indeed, if a = b/0, it would mean in reverse that a×0 = b. If b itself was zero, the equation would be true for every single number there is, making it impossible to define a concrete value for a. And if b was any other value, no single value multiplied by zero could result in anything non-zero. Once we move into the realms of calculus, we will learn that infinity appears to be the answer, but that’s in the end just replacing one abstract, mind-boggling concept with another one. And it won’t answer one question: how does all this play out in a processor? Continue reading “Creating Black Holes: Division By Zero In Practice” →

Jittery Back Off To Speed Up

November 21, 2018 by Kerry Scharfglass 6 Comments

In systems where there are multiple participants who need to interact with a shared resource some sort of concurrency protection is usually appropriate. The obvious technique is to use locking (and fun words like “mutex”) but this adds a constant performance hit as every participant needs to spend time interacting with the lock regardless of the number of other participants. It turns out this is actually a Big Problem that garners original research, but there are techniques that can yield great effect without a PhD. Years ago [Marc] posted a great walkthrough of one such method, exponential backoff with jitter, to Amazon’s AWS blog which is a great introduction to one such solution.

The blog post was written specifically to deal systems using a specific technique called optimistic concurrency control (OCC) but that doesn’t mean the advice isn’t generally applicable. OCC refers to a method where each writer checks for a write collision only after performing the write (but before committing it), which works well in scenarios where writes are relatively uncommon. [Marc] observed that even in systems where this is a safe assumption things bog down significantly when there are too many writers clamoring for attention all at once.

The traditional solution to such a problem is for each writer to pause for an exponentially increasing amount of time before trying again, but as writers come back in big groups the same problem can recur. He provides a discussion of simple modifications to that strategy which result in significantly reduced wait times for writers.

Problems like this are not especially relevant for single Arduino sensor networks, but even small groups of systems can have concurrency trouble and it’s nice to see such an accessible write up with solutions which are viable even for simple systems. Bonus points to [Marc] for posting source to his test tool online. It doesn’t require anything outside of your computer to run (no AWS required) so if you have any brainwaves about speeding up multi-writer environments it might make a nice test environment! Maybe don’t mention the blog post in your PhD applications though.

Warnings Are Your Friend – A Code Quality Primer

November 6, 2018 by Sven Gregori 63 Comments

If there’s one thing C is known and (in)famous for, it’s the ease of shooting yourself in the foot with it. And there’s indeed no denying that the freedom C offers comes with the price of making it our own responsibility to tame and keep the language under control. On the bright side, since the language’s flaws are so well known, we have a wide selection of tools available that help us to eliminate the most common problems and blunders that could come back to bite us further down the road. The catch is, we have to really want it ourselves, and actively listen to what the tools have to say.

We often look at this from a security point of view and focus on exploitable vulnerabilities, which you may not see as valid threat or something you need to worry about in your project. And you are probably right with that, not every flaw in your code will lead to attackers taking over your network or burning down your house, the far more likely consequences are a lot more mundane and boring. But that doesn’t mean you shouldn’t care about them.

Buggy, unreliable software is the number one cause for violence against computers, and whether you like it or not, people will judge you by your code quality. Just because Linus Torvalds wants to get off Santa’s naughty list, doesn’t mean the technical field will suddenly become less critical or loses its hostility, and in a time where it’s never been easier to share your work with the world, reliable, high quality code will prevail and make you stand out from the masses.

Continue reading “Warnings Are Your Friend – A Code Quality Primer” →

Code Review Lamp Subtly Reminds You To Help Your Fellow Developer

November 1, 2018 by Donald Papp 10 Comments

[Dimitris Platis] works in an environment with a peer review process for accepting code changes. Code reviews generally are a good thing. One downside though, is that a lack of responsiveness from other developers can result in a big hit to team’s development speed. It isn’t that other developers are unwilling to do the reviews, it’s more that individuals are often absorbed in their own work and notification emails are easily missed. There is also a bit of a “tragedy of the commons” vibe to the situation, where it’s easy to feel that someone else will surely attend to the situation, but often no one does. To combat this, [Dimitris] built this Code Review Lamp, a subtle notification that aims to prod reviewers into action.

The lamp is based on a ring of RGB LEDs and a Wemos D1 Mini board. The Wemos utilizes the popular ESP8266, so it’s easy to develop for. The LED ring and Wemos are tied together with a slick custom PCB. Mounting the LED ring on the top of the PCB and the Wemos on the bottom allows for easy powering via a USB cable while directing light upward. The assembly is placed in a translucent 3D printed enclosure creating a pleasant diffuse light source.

Every developer gets a Code Review Lamp. The lamps automatically log in to the change management system to check whether anything is awaiting review. If a review is ready, the Lamp glows in a color specific to the individual developer. All this serves as a gentle but persistent reminder that someone’s work is being held up until a review is completed.

We love the way that the device has a clear purpose: it does its job without any unnecessary features or parts. It’s similar to this ESP8266 IoT Motion Sensor in that it has a single job to do, and focuses on it well.

Continue reading “Code Review Lamp Subtly Reminds You To Help Your Fellow Developer” →

When Good Software Goes Bad: Malware In Open Source

October 31, 2018 by Jonathan Bennett 28 Comments

Open Source software is always trustworthy, right? [Bertus] broke a story about a malicious Python package called “Colourama”. When used, it secretly installs a VBscript that watches the system clipboard for a Bitcoin address, and replaces that address with a hardcoded one. Essentially this plugin attempts to redirects Bitcoin payments to whoever wrote the “colourama” library.

Why would anyone install this thing? There is a legitimate package named “Colorama” that takes ANSI color commands, and translates them to the Windows terminal. It’s a fairly popular library, but more importantly, the name contains a word with multiple spellings. If you ask a friend to recommend a color library and she says “coulourama” with a British accent, you might just spell it that way. So the attack is simple: copy the original project’s code into a new misspelled project, and add a nasty surprise.

Sneaking malicious software into existing codebases isn’t new, and this particular cheap and easy attack vector has a name: “typo-squatting”. But how did this package get hosted on PyPi, the main source of community contributed goodness for Python? How many of you have downloaded packages from PyPi without looking through all of the source? pip install colorama? We’d guess that it’s nearly all of us who use Python.

It’s not just Python, either. A similar issue was found on the NPM javascript repository in 2017. A user submitted a handful of new packages, all typo-squatting on existing, popular packages. Each package contained malicious code that grabbed environment variables and uploaded them to the author. How many web devs installed these packages in a hurry?

Of course, this problem isn’t unique to open source. “Abstractism” was a game hosted on Steam, until it was discovered to be mining Monero while gamers were playing. There are plenty of other examples of malicious software masquerading as something else– a sizable chunk of my day job is cleaning up computers after someone tried to download Flash Player from a shady website.

Buyer Beware

In the open source world, we’ve become accustomed to simply downloading libraries that purport to do exactly the cool thing we’re looking for, and none of us have the time to pore through the code line by line. How can you trust them?

Repositories like PyPi do a good job of faithfully packaging the libraries and programs that are submitted to them. As the size of these repositories grow, it becomes less and less practical for every package to be manually reviewed. PyPi lists 156,750 projeccts. Automated scanning like [Bertus] was doing is a great step towards keeping malicious code out of our repositories. Indeed, [Bertus] has found eleven other malicious packages while testing the PyPi repository. But cleverer hackers will probably find their way around automated testing.

That the libraries are open source does add an extra layer of reliability, because the code can in principal be audited by anyone, anytime. As libraries are used, bugs are found, and features are added, more and more people are intentionally and unintentionally reviewing the code. In the “colourama” example, a long Base64 string was decoded and executed. It doesn’t take a professional researcher to realize something fishy is going on. At some point, enough people have reviewed a codebase that it can be reasonably trusted. “Colorama” has well over a thousand stars on Github, and 28 contributors. But did you check that before downloading it?

Typo-squatting abuses trust, taking advantage of a similar name and whoever isn’t paying quite close enough attention. It’s not practical for every user to check every package in their operating system. How, then, do we have any trust in any install? Cryptography solves some of these problems, but it cannot overcome the human element. A typo in a url, trusting a brand new project, or even obfuscated C code can fool the best of us from time to time.

What’s the solution? How do we have any confidence in any of our software? When downloading from the web, there are some good habits that go a long way to protect against attacks. Cross check that the project’s website and source code actually point to each other. Check for typos in URLs. Don’t trust a download just because it’s located on a popular repository.

But most importantly, check the project’s reputation, the number of contributors to the project, and maybe even their reputation. You wouldn’t order something on eBay without checking the seller’s feedback, would you? Do the same for software libraries.

A further layer of security can be found in using libraries supported by popular distributions. In quality distributions, each package has a maintainer that is familiar with the project being maintained. While they aren’t checking each line of code of every project, they are ensuring that “colorama” gets packaged instead of “colourama”. In contrast to PyPi’s 156,750 Python modules, Fedora packages only around 4,000. This selection is a good thing.

Repositories like PyPi and NPM are simply not the carefully curated sources of trustworthy software that we sometimes think them to be– and we should act accordingly. Look carefully into the project’s reputation. If the library is packaged by your distribution of choice, you can probably pass this job off to the distribution’s maintainers.

At the end of the day, short of going through the code line by line, some trust anchor is necessary. If you’re blindly installing random libraries, even from a “trustworthy” repository, you’re letting your guard down.

Jump Into AI With A Neural Network Of Your Own

October 22, 2018 by Al Williams 6 Comments

One of the difficulties in learning about neural networks is finding a problem that is complex enough to be instructive but not so complex as to impede learning. [ThomasNield] had an idea: Create a neural network to learn if you should put a light or dark font on a particular colored background. He has a great video explaining it all (see below) and code in Kotlin.

[Thomas] is very interested in optimization, so his approach is very much based on mathematics and algorithms of optimization. One thing that’s handy is that there is already an algorithm for making this determination. He found it on Stack Exchange, but we’re sure it’s in a textbook or paper somewhere. The existing algorithm makes the neural network really impractical, but it makes training easy since you can algorithmically develop a training set of data.

Once trained, the neural network works well. He wrote a small GUI and you can even select among various models.

Don’t let the Kotlin put you off. It is a derivative of Java and uses the same JVM. The code is very similar, other than it infers types and also adds functional program tools. However, the libraries and the principles employed will work with Java and, in many cases, the concepts will apply no matter what you are doing.

If you want to hardware accelerate your neural networks, there’s a stick for that. If you prefer C and you want something lean and mean, try TINN.

Continue reading “Jump Into AI With A Neural Network Of Your Own” →

Easy Access Point Configuration On ESP8266

October 4, 2018 by Tom Nardi 19 Comments

One of the biggest advantages of using the ESP8266 in your projects is how easy it is to get WiFi up and running. Just plug in the WiFi library, put the SSID and encryption key in your source code, and away you go. It authenticates with your network in seconds and you can get on with building your project. But things get a little trickier if you want to take your project someplace else, or distribute your source code to others. Quickly we learn the downside of using static variables for authentication.

While there are already a few solutions to this problem out there, [Martin Raynsford] wasn’t too thrilled with them. Usually they put the ESP8266 in Access Point mode, allow the user to connect, and then ask which network they should authenticate with. But he didn’t want his projects to require an existing network, and figured he could do just as well making a field-configurable AP.

Using it is simple. Once the ESP8266 starts up it will create a new network in the form of “APConfig XXXXXX”, which should be easy enough to find from your client side device. Once connected, you can go to a simple administration page which allows you to configure a new AP name and encryption key. You even have the option to create an open AP by leaving the “Password” field blank. Once rebooted, the ESP8266 will create a new network with the defined parameters.

[Martin] has also included a “backdoor” to let anyone with physical access to the ESP8266 board create a new open AP that can be used to reconfigure the network settings. During boot up there is a brief period, indicated with specific blinks of the LED, wherein you can hit the reset button and trigger the open AP. This keeps you from getting locked out of your own project if you forget what key you gave it.

If you’re not one to go the austere route, take a look at some of the more robust solutions we’ve seen for easier end-user setup of the ESP8266.