software development

48 Articles

A Cheat Sheet For Publishing Python Packages

December 13, 2018 by 5 Comments

[Brendan Herger] was warned that the process of publishing a Python package would be challenging. He relishes a challenge, however, and so he went at it with gusto. The exhausting process led him to share a cheat sheet for publishing Python packages with the goal of making the next time smoother, while also letting other people benefit from his experience and get a running start.

[Brendan] describes publishing a Python package as “tying together many different solutions with brittle interchanges.” His cheat sheet takes the form of an ordered workflow for getting everything in place, with some important decisions and suggestions about things like formatting and continuous integration (CI) made up-front.

The guide is brief, but [Brendan] has made errors and hit dead ends in the hopes that others won’t have to. The whole thing came about from his work in deep learning, and his desire to create a package that allows rapid building and iterating on deep learning models.

Deep learning is a type of machine learning that involves finding representations in large amounts of data. [Brendan] used it in a project to automatically decide whether a Reddit post contains Star Wars plot spoilers, and we recently saw it featured in a method of capturing video footage only if a hummingbird is present.

Warnings On Steroids – Static Code Analysis Tools

December 12, 2018 by 11 Comments

A little while back, we were talking about utilizing compiler warnings as first step to make our C code less error-prone and increase its general stability and quality. We know now that the C compiler itself can help us here, but we also saw that there’s a limit to it. While it warns us about the most obvious mistakes and suspicious code constructs, it will leave us hanging when things get a bit more complex.

But once again, that doesn’t mean compiler warnings are useless, we simply need to see them for what they are: a first step. So today we are going to take the next step, and have a look at some other common static code analysis tools that can give us more insight about our code.

You may think that voluntarily choosing C as primary language in this day and age might seem nostalgic or anachronistic, but preach and oxidate all you want: C won’t be going anywhere. So let’s make use of the tools we have available that help us write better code, and to defy the pitfalls C is infamous for. And the general concept of static code analysis is universal. After all, many times a bug or other issue isn’t necessarily caused by the language, but rather some general flaw in the code’s logic.

Continue reading “Warnings On Steroids – Static Code Analysis Tools” →

Warnings Are Your Friend – A Code Quality Primer

November 6, 2018 by 63 Comments

If there’s one thing C is known and (in)famous for, it’s the ease of shooting yourself in the foot with it. And there’s indeed no denying that the freedom C offers comes with the price of making it our own responsibility to tame and keep the language under control. On the bright side, since the language’s flaws are so well known, we have a wide selection of tools available that help us to eliminate the most common problems and blunders that could come back to bite us further down the road. The catch is, we have to really want it ourselves, and actively listen to what the tools have to say.

We often look at this from a security point of view and focus on exploitable vulnerabilities, which you may not see as valid threat or something you need to worry about in your project. And you are probably right with that, not every flaw in your code will lead to attackers taking over your network or burning down your house, the far more likely consequences are a lot more mundane and boring. But that doesn’t mean you shouldn’t care about them.

Buggy, unreliable software is the number one cause for violence against computers, and whether you like it or not, people will judge you by your code quality. Just because Linus Torvalds wants to get off Santa’s naughty list, doesn’t mean the technical field will suddenly become less critical or loses its hostility, and in a time where it’s never been easier to share your work with the world, reliable, high quality code will prevail and make you stand out from the masses.

Continue reading “Warnings Are Your Friend – A Code Quality Primer” →

Code Review Lamp Subtly Reminds You To Help Your Fellow Developer

November 1, 2018 by 10 Comments

[Dimitris Platis] works in an environment with a peer review process for accepting code changes. Code reviews generally are a good thing. One downside though, is that a lack of responsiveness from other developers can result in a big hit to team’s development speed. It isn’t that other developers are unwilling to do the reviews, it’s more that individuals are often absorbed in their own work and notification emails are easily missed. There is also a bit of a “tragedy of the commons” vibe to the situation, where it’s easy to feel that someone else will surely attend to the situation, but often no one does. To combat this, [Dimitris] built this Code Review Lamp, a subtle notification that aims to prod reviewers into action.

The lamp is based on a ring of RGB LEDs and a Wemos D1 Mini board. The Wemos utilizes the popular ESP8266, so it’s easy to develop for. The LED ring and Wemos are tied together with a slick custom PCB. Mounting the LED ring on the top of the PCB and the Wemos on the bottom allows for easy powering via a USB cable while directing light upward.  The assembly is placed in a translucent 3D printed enclosure creating a pleasant diffuse light source.

Every developer gets a Code Review Lamp. The lamps automatically log in to the change management system to check whether anything is awaiting review. If a review is ready, the Lamp glows in a color specific to the individual developer. All this serves as a gentle but persistent reminder that someone’s work is being held up until a review is completed.

We love the way that the device has a clear purpose: it does its job without any unnecessary features or parts. It’s similar to this ESP8266 IoT Motion Sensor in that it has a single job to do, and focuses on it well.

Continue reading “Code Review Lamp Subtly Reminds You To Help Your Fellow Developer” →

Branch Out Your SQLite Database With LiteTree

September 2, 2018 by 5 Comments

Whether you want some quick and dirty data storage, or simply don’t have that heavy requirements for your local database system, SQLite is always a good choice. With its portable single-file approach, bindings to all major languages, and availability on systems of all sizes, it is relatively easy to integrate a SQLite database in your undertakings. And if you tend to develop directly in your production environment, you may be interested to hear that the folks at [aergo] made this a lot more flexible (and interesting) by adding Git-style branching to the SQLite engine.

Similar to Git, each database operation is now stored as a commit with a unique id as reference point, and new branches will keep track how they diverge from their parent reference point. This essentially lets you modify your data set or database schema on the fly, while keeping your original data not only untouched, but fully isolated and functional. Unfortunately, merging branches is not yet supported, but it is planned for the near future.

In case you don’t see much use for git-alike functionality in a database, how about the other way around then: using Git as a database, among other tricks?

Directly Executing Chunks Of Memory: Function Pointers In C

May 2, 2018 by 75 Comments

In the first part of this series, we covered the basics of pointers in C, and went on to more complex arrangements and pointer arithmetic in the second part. Both times, we focused solely on pointers representing data in memory.

But data isn’t the only thing residing in memory. All the program code is accessible through either the RAM or some other executable type of memory, giving each function a specific address inside that memory as entry point. Once again, pointers are simply memory addresses, and to fully utilize this similarity, C provides the concept of function pointers. Function pointers provide us with ways to make conditional code execution faster, implement callbacks to make code more modular, and even provide a foothold into the running machine code itself for reverse engineering or exploitation. So read on!

Function Pointers

In general, function pointers aren’t any more mysterious than data pointers: the main difference is that one references variables and the other references functions. If you recall from last time how arrays decay into pointers to their first element, a function equally decays into a pointer to the address of its entry point, with the () operator executing whatever is at that address. As a result, we can declare a function pointer variable fptr and assign a function func() to it: fptr = func;. Calling fptr(); will then resolve to the entry point of function func() and execute it.

Admittedly, the idea of turning a function into a variable may seem strange at first and might require some getting used to, but it gets easier with time and it can be a very useful idiom. The same is true for the function pointer syntax, which can be intimidating and confusing in the beginning. But let’s have a look at that ourselves.

Continue reading “Directly Executing Chunks Of Memory: Function Pointers In C” →

When 4 + 1 Equals 8: An Advanced Take On Pointers In C

April 19, 2018 by 104 Comments

In our first part on pointers, we covered the basics and common pitfalls of pointers in C. If we had to break it down into one sentence, the main principle of pointers is that they are simply data types storing a memory address, and as long as we make sure that we have enough memory allocated at that address, everything is going to be fine.

In this second part, we are going to continue with some more advanced pointer topics, including pointer arithmetic, pointers with another pointer as underlying data type, and the relationship between arrays and pointers. But first, there is one particular pointer we haven’t talked about yet.

The one proverbial exception to the rule that pointers are just memory addresses is the most (in)famous pointer of all: the NULL pointer. Commonly defined as preprocessor macro (void *) 0, we can assign NULL like any other pointer.

Continue reading “When 4 + 1 Equals 8: An Advanced Take On Pointers In C” →