Fail Of The Week: GitLab Goes Down

February 2, 2017 by Jenny List 57 Comments

Has work been a little stressful this week, are things getting you down? Spare a thought for an unnamed sysadmin at the GitHub-alike startup GitLab, who early yesterday performed a deletion task on a PostgreSQL database in response to some problems they were having in the wake of an attack by spammers. Unfortunately due to a command line error he ran the deletion on one of the databases behind the company’s main service, forcing it to be taken down. By the time the deletion was stopped, only 4.5 Gb of the 300 Gb trove of data remained.

Reading their log of the incident the scale of the disaster unfolds, and we can’t help wincing at the phrase “out of 5 backup/replication techniques deployed none are working reliably or set up in the first place“. In the end they were able to restore most of the data from a staging server, but at the cost of a lost six hours of issues and merge requests. Fortunately for them their git repositories were not affected.

For 707 GitLab users then there has been a small amount of lost data, the entire web service was down for a while, and the incident has gained them more publicity in a day than their marketing department could have achieved in a year. The post-mortem document makes for a fascinating read, and will probably leave more than one reader nervously thinking about the integrity of whichever services they are responsible for. We have to hand it to them for being so open about it all and for admitting a failure of their whole company for its backup failures rather than heaping blame on one employee. In many companies it would all have been swept under the carpet. We suspect that GitLab’s data will be shepherded with much more care henceforth.

We trust an increasing amount of our assets to online providers these days, and this tale highlights some of the hazards inherent in placing absolute trust in them. GitLab had moved from a cloud provider to their own data centre, though whether or not this incident would have been any less harmful wherever it was hosted is up for debate. Perhaps it’s a timely reminder to us all: keep your own backups, and most importantly: test them to ensure they work.

Thanks [Jack Laidlaw] for the tip.

Rack server image: Trique303 [CC BY-SA 4.0], via Wikimedia Commons.

33C3: Hunz Deconstructs The Amazon Dash Button

February 2, 2017 by Elliot Williams 22 Comments

The Amazon Dash button is now in its second hardware revision, and in a talk at the 33rd Chaos Communications Congress, [Hunz] not only tears it apart and illuminates the differences with the first version, but he also manages to reverse engineer it enough to get his own code running. This opens up a whole raft of possibilities that go beyond the simple “intercept the IP traffic” style hacks that we’ve seen.

Just getting into the Dash is a bit of work, so buy two: one to cut apart and locate the parts that you have to avoid next time. Once you get in, everything is tiny! There are a lot of 0201 SMD parts. Hidden underneath a plastic blob (acetone!) is an Atmel ATSAMG55, a 120 MHz ARM Cortex-M4 with FPU, and a beefy CPU all around. There is also a 2.4 GHz radio with a built-in IP stack that handles all the WiFi, with built-in TLS support. Other parts include a boost voltage converter, a BTLE chipset, an LED, a microphone, and some SPI flash.

The strangest part of the device is the sleep mode. The voltage regulator is turned on by user button press and held on using a GPIO pin on the CPU. Once the microcontroller lets go of the power supply, all power is off until the button is pressed again. It’s hard to use any less power when sleeping. Even so, the microcontroller monitors the battery voltage and presumably phones home when it gets low.
Continue reading “33C3: Hunz Deconstructs The Amazon Dash Button” →

Turning Television Into A Simple Tapestry

February 1, 2017 by James Hobson 6 Comments

Teleknitting, the brainchild of Moscow artist [vtol], is an interesting project. On one hand, it doesn’t knit anything that is useful in a traditional sense, but on the other, it attempts the complex task of deconstructing broadcasted media into a simpler form of information transmission.

Teleknitting’s three main components are the processing and display block — made up of the antenna, Android tablet, and speaker — the dyeing machine with its ink, sponges, actuators, and Arduino Uno, and the rotating platform for the sacrificial object. A program running on the tablet analyzes the received signal and — as displayed on its screen — gradually halves the number of pixels in the image until there is only one left with a basic representation of the picture’s colour. From there, thread passes over five sponges which dye it the appropriate colour, with an armature that responds to the broadcast’s volume directing where the thread will bind the object.

Continue reading “Turning Television Into A Simple Tapestry” →

A Very MIDI Christmas Lightshow

February 1, 2017 by James Hobson 22 Comments

Christmas light displays winking and flashing in sync to music are a surefire way to rack up views on YouTube and annoy your neighbours. Inspired by one such video, [Akshay James] set up his own display and catalogued the process in this handy tutorial to get you started on your own for the next holiday season.

[James], using the digital audio workstation Studio One, took the MIDI data for the song ‘Carol of the Bells’ and used that as the light controller data for the project’s Arduino brain. Studio One sends out the song’s MIDI data, handled via the Hairless MIDI to serial bridge, to the Arduino which in turn sets the corresponding bit to on or off. That gets passed along to three 74HC595 shift registers — and their three respective relay boards — which finally trigger the relay for the string of lights.

From there, it’s a matter of wiring up the Arduino shift register boards, relays, and connecting the lights. Oh, and be sure to mount a speaker outdoors so passers-by can enjoy the music:

Continue reading “A Very MIDI Christmas Lightshow” →

Home-made Soldering Station For $15

February 1, 2017 by Richard Baguley 28 Comments

A proper soldering iron is one of the fundamental tools that a good hacker needs. Preferably one that has a temperature control so it can handle different types of solder and connectors.

Decent soldering stations aren’t cheap, but [Code and Solder] show you how to make one for about $15 in parts. This uses a cheap non-temperature-controlled USB soldering iron, an Arduino and a few other bits that they got from AliExpress. The plan is to add a thermocouple to the soldering iron, and let the Arduino control the temperature. A rotary dial and LCD screen control the set-point, and the Arduino switches the feed to the heating element on and off through the FET.

It’s not the cleanest build in the world, and these USB soldering irons aren’t suitable for large joints or long soldering jobs, but it’s a neat little hack for the builder on a budget. We’ve seen teardowns of these rather neat little USB soldering irons before, but this is an interesting way to expand its capabilities.

Continue reading “Home-made Soldering Station For $15” →

33C3: How Can You Trust Your Random Numbers?

February 1, 2017 by Elliot Williams 49 Comments

One of the standout talks at the 33rd Chaos Communications Congress concerned pseudo-random-number generators (PRNGs). [Vladimir Klebanov] (right) and [Felix Dörre] (left) provided a framework for making sure that PRNGs are doing what they should. Along the way, they discovered a flaw in Libgcrypt/GNUPG, which they got fixed. Woot.

Cryptographically secure random numbers actually matter, a lot. If you’re old enough to remember the Debian OpenSSL debacle of 2008, essentially every Internet service was backdoorable due to bad random numbers. So they matter. [Vladimir] makes the case that writing good random number generators is very, very hard. Consequently, it’s very important that their output be tested very, very well.

So how can we test them? [Vladimir] warns against our first instinct, running a statistical test suite like DIEHARD. He points out (correctly) that running any algorithm through a good enough hash function will pass statistical tests, but that doesn’t mean it’s good for cryptography.
Continue reading “33C3: How Can You Trust Your Random Numbers?” →

Self-Lacing LEGO Power Shoe

February 1, 2017 by Steven Dufresne 12 Comments

Here’s a blast from the past, or future, reminiscent of the self-lacing shoes from Back to the Future Part II. [Vimal Patel] made his own self-lacing shoe using LEGO “bolted” to the shoe’s sole. We think these are cooler than the movie version since we get to see the mechanism in action, urging it on as the motor gets loaded down pulling the laces for that last little bit of tightness.

The electronics are all LEGO’s Power Functions parts. A Dremel was used to make holes in the soles to hot glue LEGO pieces for four attachment points. The attachment points are permanent but the rest can be easily removed. In case you want to look them up or make your own, he’s using the using the 8878 rechargeable LiPo battery box, the 88003 L-motor, the 8884 IR receiver, and the 8885 IR remote control. That’s right, these shoes are laced up under command of an IR remote control, well, provided the battery box is powered on. There’s a 1:24 worm gear reduction to get the needed torque.

This was a quick build for [Patel], done over two afternoons. He initially tried with the winding axle behind the heel but that didn’t work well so he moved the axle adjacent to the laces instead, which works great as you can see in the video after the break.

Continue reading “Self-Lacing LEGO Power Shoe” →