Symbiflow Open Source FPGA Toolchain

November 1, 2019 by Gerrit Coetzee 12 Comments

Anyone who’s ever had the pleasure of programming FPGAs knows that it’s a land of proprietary tools that almost require marriage level commitment to a specific platform to be effective. Symbiflow hopes to solve this by becoming the GCC of FPGAs.

Rather than a tool built around a specific chip or architecture, Symbiflow will provide a more universal interface. Users can program in Verilog; architecture definitions define how the code will be compiled for the right chip. They are currently targeting the popular Xilinx 7-series, the very affordable iCE40 series from lattice, and the ECP5 FPGAs also from Lattice.

If you’re headed to Hackaday Supercon this year, [Timothy Ansell] will be giving a talk on how Symbiflow is making this process much more approachable and much less proprietary. Overall we’re very excited about a common interface, especially as the price of FPGAs keep dropping into micro controller territory while also increasing in capability.

(Speaking of Supercon, and maybe this is a spoiler, the badge would not have been possible without Symbiflow, Project Trellis, Yosys, and NextPNR.)

Unix Tell All Book From Kernighan Hits The Shelves

October 29, 2019 by Al Williams 56 Comments

When you think of the Unix and C revolution that grew out of Bell Labs, there are a few famous names. Dennis Ritchie, Ken Thompson, and Brian Kernighan come to mind. After all, the K in both K&R C and in AWK stand for Kernighan. While Kernighan is no stranger to book authorship — he’s written several classics including “the white book” for C and Unix — he has a new book out that is part historical record and part memoir about the birth of Unix.

Usually, when a famous person writes a retrospective like this, it is full of salacious details, but we don’t expect much of that here. The book talks about Bell Labs and Multics, of course. There’s serious coverage of the first, sixth, and seventh editions with biographies of people integral to those releases.

Continue reading “Unix Tell All Book From Kernighan Hits The Shelves” →

An Algorithm For De-Biasing AI Systems

October 21, 2019 by Sharon Lin 52 Comments

A fundamental truth about AI systems is that training the system with biased data creates biased results. This can be especially dangerous when the systems are being used to predict crime or select sentences for criminals, since they can hinge on unrelated traits such as race or gender to make determinations.

A group of researchers from the Massachusetts Institute of Technology (MIT) CSAIL is working on a solution to “de-bias” data by resampling it to be more balanced. The paper published by PhD students [Alexander Amini] and [Ava Soleimany] describes an algorithm that can learn a specific task – such as facial recognition – as well as the structure of the training data, which allows it to identify and minimize any hidden biases.

Testing showed that the algorithm minimized “categorical bias” by over 60% compared against other widely cited facial detection models, all while maintaining the same precision of detection. This figure was maintained when the team evaluated a facial-image dataset from the Algorithmic Justice League, a spin-off group from the MIT Media Lab.

The team says that their algorithm would be particularly relevant for large datasets that can’t easily be vetted by a human, and can potentially rectify algorithms used in security, law enforcement, and other domains beyond facial detection.

File Compression By Steganography

October 19, 2019 by Sharon Lin 29 Comments

In a world with finite storage and an infinite need for more storage space, data compression becomes a very necessary problem. Several algorithms for data compression may be more familiar – Huffman coding, LZW compression – and some a bit more arcane.

[Labunsky] decided to put to use his knowledge of steganography to create a wholly unique form of file compression, perhaps one that may gain greater notoriety among other information theorists.

Steganography refers to the method of concealing messages or files within another file, coming from the Greek words steganos for “covered or concealed” and graphe for “writing”. The practice has been around for ages, from writing in invisible ink to storing messages in moon cakes. The methods used range from hiding messages in images to evade censorship to hiding viruses in files to cause mayhem.

The developer explains that since every file is just a bit sequence, observing files leads to the realization that a majority of bits will be equal on the same places. Rather than storing all of the bits of a file, making modifications to the hard drive at certain locations can save storage space. What is important to avoid, however, is lossy file compression that can wreak havoc on quality during the compression stage.

The compression technique they ended up implementing is based on the F5 algorithm that embeds binary data into JPEG files to reduce total space in the memory. The compression uses libjpeg for JPEG decoding and encoding, pcre for POSIX regular expressions support, and tinydir for platform-independent filesystem traversal. One of the major modifications was to save computation resources by disabling a password-based permutative straddling that uniformly spreads data among multiple files.

One caveat – changing even one bit of the compressed file could lead to total corruption of all of the data stored, so use with caution!

Reverse Engineering Liberates Dash Cam Video

October 15, 2019 by Tom Nardi 12 Comments

If you’ve purchased a piece of consumer electronics in the last few years, there’s an excellent chance that you were forced to use some proprietary application (likely on a mobile device) to unlock its full functionality. It’s a depressing reality of modern technology, and unless you’re willing to roll your own hardware, it can be difficult to avoid. But [krishnan793] decided to take another route, and reverse engineered his DDPAI dash camera so he could get a live video stream from it without using the companion smartphone application.

Like many modern gadgets, the DDPAI camera creates its own WiFi access point that you need to connect to for configuration. By putting his computer’s wireless card into Monitor mode and running Wireshark, [krishnan793] was able to see that the smartphone was communicating with the camera using some type of REST API. After watching the clear-text exchanges for awhile, he not only discovered a few default usernames and passwords, but the commands necessary to configure the camera and start the video stream.

After hitting it with the proper REST messages, an nmap scan confirmed that several new services had started up on the device. Unfortunately, he didn’t get any video when he pointed VLC to the likely port numbers. At this point [krishnan793] checked the datasheet for the camera’s Hi3516E SoC and saw that it supported H.264 encoding. By manually specifying that as the video codec when invoking VLC, it was able to play a video stream from port 6200. A little later, he discovered that port 6100 was serving up the live audio.

Technically that’s all he wanted to do in the first place, as he was looking to feed the video into OpenCV for other projects. But while he was in the area, [krishnan793] also decided to find the download URL for the camera’s firmware, and ran it through binwalk to see what he could find out. Not surprisingly the security turned out to be fairly lax through the entire device, so he was able to glean some information that could be useful for future projects.

Of course, if you’d rather go with the first option and build your own custom dash camera so you don’t have to jump through so many hoops just to get a usable video stream, we’ve got some good news for you.

Linux Fu: Python GUIs For Command Line Programs (Almost) Instantly

October 14, 2019 by Al Williams 7 Comments

Not every programmer likes creating GUI code. Most hacker types don’t mind a command line interface, but very few ordinary users appreciate them. However, if you write command line programs in Python, Gooey can help. By leveraging some Python features and a common Python idiom, you can convert a command line program into a GUI with very little effort.

The idea is pretty simple. Nearly all command line Python programs use argparse to simplify picking options and arguments off the command line as well as providing some help. The Gooey decorator picks up all your options and arguments and creates a GUI for it. You can make it more complicated if you want to change specific things, but if you are happy with the defaults, there’s not much else to it.

At first, this article might seem like a Python Fu and not a Linux Fu, since — at first — we are going to focus on Python. But just stand by and you’ll see how this can do a lot of things on many operating systems, including Linux.

Continue reading “Linux Fu: Python GUIs For Command Line Programs (Almost) Instantly” →

This Sentence‌‌‌‌‍‌ Isn’‌‌‌‌‍‌‬t Just ‌‌‌‌‌‬‌‌a‌‌‌‌‍‬‬‍ Sentence‌‌‌‌‍‌‌‌‌‌‌‬‌‌‌‌‌‌‍‬‬‍‌‌‌‌‍‍‌‌‌‌‌‌

October 10, 2019 by Sharon Lin 22 Comments

Some sentences have more than meets the eye, and we’re not talking about interpretive nonsense. Rather, some sentences may contain up to four paragraphs’ worth of hidden text, invisible to readers.

Thanks to Zero Width Obfuscation, it is possible to use Zero Width Characters – Unicode characters that are invisible even when you try to highlight them. They’re typically used for abstract foreign languages that require separators that don’t take up an entire space. In this case, they’re used to obfuscate and de-obfuscate hidden messages sent through text.

[inzerosight] published a browser extension that identifies, de-obfuscates, and obfuscates these messages for you on the web. It does this by querying each page for the Unicode of the Zero Width Characters (U+FEFF, U+200C, U+200D, U+200E, U+2060, U+180E) and highlighting where they’ve been spotted. The encoding replaces each Unicode character with a permutation of two of the Zero Width Characters, essentially doing a find and replace across the text message.

I’m just waiting to see how long it takes for Zero Width Obfuscation to become the next Konami Code Easter Egg.