Using Valgrind To Analyze Code For Bottlenecks Makes Faster, Less Power-Hungry Programs

May 20, 2020 by Maya Posch 27 Comments

What is the right time to optimize code? This is a very good question, which usually comes down to two answers. The first answer is to have a good design for the code to begin with, because ‘optimization’ does not mean ‘fixing bad design decisions’. The second answer is that it should happen after the application has been sufficiently debugged and its developers are at risk of getting bored.

There should also be a goal for the optimization, based on what makes sense for the application. Does it need to process data faster? Should it send less data over the network or to disk? Shouldn’t one really have a look at that memory usage? And just what is going on inside those CPU caches that makes performance sometimes drop off a cliff on a single core?

All of this and more can be analyzed using tools from the Valgrind suite, including Cachegrind, Callgrind, DHAT and Massif.

Keeping Those Cores Cool

Modern day processors are designed with low power usage in mind, regardless of whether they are aimed at servers, desktop systems or embedded applications. This essentially means that they are in a low power state when not doing any work (idle loop), with some CPUs and microcontrollers turning off power to parts of the chip which are not being used. Consequently, the more the processor has to do, the more power it will use and the hotter it will get.

Continue reading “Using Valgrind To Analyze Code For Bottlenecks Makes Faster, Less Power-Hungry Programs” →

Poking Around Inside Of A Linux Security Camera

May 20, 2020 by Tom Nardi 21 Comments

This deep dive into the Linux-powered Reolink B800 IP camera started because of a broken promise from its manufacturer. When [George Hilliard] purchased a kit that included six of the cameras and a video recorder, the website said they were capable of outputting standard RTSP video. But once he took delivery of the goods, and naturally after his return window had closed, the site was updated to say that the cameras can only function with the included recorder.

Taking that as something of challenge, [George] got to work. His first big break came when he desoldered the camera’s SPI flash chip and replaced it with a socket. That allowed him to easily take the chip out of the device for reading and flashing as he tinkered with modifying the firmware. After adding cross-compiled versions of busybox, gdb, and strace to the extracted firmware, he bundled it back up and flashed it back to the hardware.

If you think that’s the end of the story, it isn’t. In fact, it’s just the beginning. While getting root-level access to the camera’s OS would have potentially allowed for [George] to dump all the proprietary software it was running and replace it with open alternatives, he decided to take a different approach.

Instead of replacing the camera’s original software, he used his newly granted root powers to analyze it and figure out how it worked. This allowed for to sniff out some very suspect “encryption” routines built into the software, and eventually write his own server side in Rust that finally allowed him to use the cameras with his own server…albeit with a bit more work than he bargained for.

Projects like these are a fantastic look at real world reverse engineering, and a reminder that sometimes achieving your ultimate goal means taking the long way around. Even if you’re not in the market for a hacked security camera, there’s no doubt that reading the thorough write-up [George] has prepared will teach you a few things. But of course, we’d expect no less from a guy who runs Linux on his business card.

Writing Android Apps In C, No Java Required

May 13, 2020 by Tom Nardi 32 Comments

Older Android devices can be had for a song, and in many cases are still packing considerable computational power. With built in networking, a battery, and a big touch screen, they could easily take the place of a Raspberry Pi and external display in many applications. As it so happens, Google has made it very easy to develop your own Android software. There’s only one problem: you’ve got to do it in Java.

Looking to get away from all that bloat and overhead, [CNLohr] set out to see what it would take to get 100% C code running on an Android device. After collecting information and resources from the deepest and darkest corners of the Internet, he found out that the process actually wasn’t that bad. He’s crafted a makefile which can be used to get your own C program up and running in seconds.

We mean that literally. As demonstrated in the video after the break, [CNLohr] is able to compile, upload, and run a C Android program in less than two seconds with a single command. This rapid development cycle allows you to spend more time on actually getting work done, as you can iterate through versions of your code almost as quickly as if you were running them on your local machine.

[CNLohr] says you’ll still need to have Google’s Android Studio installed, so it’s not as if this is some clean room implementation. But once it’s installed, you can just call everything from his makefile and never have to interact with it directly. Even if you don’t have any problem with the official Android development tools, there’s certainly something to be said for being able to write a “Hello World” that doesn’t clock in at multiple-megabytes.

Continue reading “Writing Android Apps In C, No Java Required” →

Using Valgrind To Track Down Known And Unknown Bugs In Your Code

April 29, 2020 by Maya Posch 17 Comments

We all know what bugs in code are. We don’t like them when they are in programs we use, and they’re even worse when they are in code which we have written. Clearly, the best code is bug-free, but how do we get there?

This isn’t a new question, of course, just one that has become ever more important as the total number of lines of code (LoC) that run modern day society keeps increasing and which is affecting even hobbyists more and more often now that everything has a microcontroller inside.

Although many of us know the smug satisfaction of watching a full row of green result markers light up across the board after running the unit tests for a project, the painful reality is that you don’t know whether the code really is functionally correct until it runs in an environment that is akin to the production environment. Yet how can one test an application in this situation?

This is where tools like those contained in the Valgrind suite come into play, allowing us to profile, analyze and otherwise nitpick every single opcode and memory read or write. Let’s take a look, shall we?

Continue reading “Using Valgrind To Track Down Known And Unknown Bugs In Your Code” →

Core Devs Say A Fond But Firm Farewell To Python 2

April 24, 2020 by Tom Nardi 34 Comments

Saying that it was finally time for the community to bid a “fond but firm farewell to Python 2”, core developer Benjamin Peterson marked the release of Python 2.7.18 on April 20th; officially ending support for the 2.x branch of the popular programming language. It was hardly a snap decision. Python 3.0 was released all the way back in December 2008, and it was never a secret that the newer branch was not only incompatible with the earlier version, but that it would eventually superseded it to become the standard.

But migrating the incredible amount of Python code in the wild over to the latest and greatest was easier said than done. Millions upon millions of lines of code used in everything from Linux distributions to virtually every major web service needed to be reviewed and migrated over to Python 3. In many cases the changes were relatively minor, but when code is being used in mission critical applications, even the smallest of changes are often avoided unless it’s absolutely necessary. The voluntary migration took far longer than expected, and the end-of-life (EOL) for Python 2 was pushed back by years to accommodate developers who hadn’t made the necessary changes yet.

Given the somewhat fluid nature of the Python 2 EOL date, it seems fitting that this last final release would come several months after the “official” January 2020 deadline. The intention was for it to coincide with PyCon 2020, but just like so many of the events planned for the first half of the year, the in-person conference had to be canceled in favor of a virtual one due to the COVID-19 epidemic. That might have stymied the celebration somewhat, but the release of Python 2.7.18 will still be looked on as a special moment for everyone involved.

Continue reading “Core Devs Say A Fond But Firm Farewell To Python 2” →

COBOL Isn’t The Issue: A Misinterpreted Crisis

April 20, 2020 by Sven Gregori 144 Comments

Is history doomed to repeat itself? Or rather, is there really any doubt that it isn’t, considering recent events that made the news? I am of course talking about New Jersey’s call for COBOL programmers to fix their ancient unemployment system, collapsing under the application spikes caused by the COVID-19 lockdown. Soon after, other states joined in, and it becomes painfully apparent that we have learned absolutely nothing from Y2K: we still rely on the same old antiques running our infrastructure, and we still think people want to voluntarily write COBOL.

Or maybe they do? Following the calls for aid, things went strangely intense. IBM announced to offer free COBOL trainings and launched a forum where programmers can plug their skills and availability. The Open Mainframe Project’s COBOL programming course suddenly tops the list of trending GitHub projects, and Google Trends shows a massive peak for COBOL as well. COBOL is seemingly on its way to be one of the hottest languages of 2020, and it feels like it’s only a matter of time until we see some MicroCOBOL running on a Teensy.

However, the unemployment systems in question are unfortunately only a tiny selection of systems relying on decades old software, written in a language that went out of fashion a long time ago, which makes it difficult to find programmers in today’s times. Why is that?

Continue reading “COBOL Isn’t The Issue: A Misinterpreted Crisis” →

What Does GitHub’s Npm Acquisition Mean For Developers?

March 26, 2020 by Ben James 33 Comments

Microsoft’s open-source shopping spree has claimed another victim: npm. [Nat Friedman], CEO of GitHub (owned by Microsoft), announced the move recently on the GitHub blog.

So what motivated the acquisition, and what changes are we likely to see as a result of it? There are some obvious upsides and integrations, but these will be accompanied by the usual dose of skepticism from the open-source community. The company history and working culture of npm has also had its moments in the news, which may well have contributed to the current situation. This post aims to explore some of the rationale behind the acquisition, and what it’s likely to mean for developers in the future.

Continue reading “What Does GitHub’s Npm Acquisition Mean For Developers?” →