New Year Habits – What Do You Do For Data Storage?

January 20, 2020 by Bob Baddeley 109 Comments

2020 is a year of reflection and avoiding regret, and one of the biggest practices we all know we should do better is back up our data. Inevitably there will be a corruption or accident, and we mourn the loss of some valuable data and vow to never let it happen again, and then promptly forget about good data retention practices.

I believe life is about acquiring memories, so it makes sense to me to try to archive and store those memories so that I can reflect on them later, but data storage and management is a huge pain. There’s got to be a better way (cue black and white video of clumsy person throwing up arms in disgust).

Nice Cloud You Have There; Shame if Something Happened to It

The teens of the century saw a huge shift towards cloud storage. The advantages of instantly backing up files and using the cloud as the primary storage for all your devices is appealing. It’s now easier to transfer files via the cloud than with a cable. With Google Docs and WordPress we have our most important documents and writing stored as database blobs on someone else’s servers. Facebook and Google and Flickr record all of our memories as photo albums. Unlimited storage is common, and indexing is so good that we can find photos with a vague description of their contents.

These things are instantly accessible, but lack permanence. Gone are newspaper clippings and printed photos discovered in a shoebox. When we aren’t in control of those services, they can disappear without any warning. Even some big offerings have packed up shop, leaving people scrambling to back up data before the servers were shut down. Google Plus is closed, Yahoo Groups is closed, MySpace lost all content created prior to 2016, GeoCities closed in 2009, and Ubuntu One closed in 2014. It’s safe to say that no online content is safe from deletion. It’s also safe to say that cloud storage is a difficult location from which to extract your data.

With the risk of data leaks and privacy violations occurring daily, it’s also safe to say that some of your files should probably not be stored in the cloud in the first place. So, how do we do it well, and how do we get in the habit of doing it regularly?

Continue reading “New Year Habits – What Do You Do For Data Storage?” →

DIY Photo Backup In The Field

November 20, 2019 by Bryan Cockfield 29 Comments

They say a file isn’t backed up if it isn’t backed up twice. This is easy enough to do if you have access to your computer and a network, but if you’re a photographer you might end up in a place without either of these things and need a way to back up the files you just created. For that you’ll need a specialized photo backup tool which you can easily build yourself.

While commercial offerings are available which back up files locally from a camera’s SD card to another medium, they suffer from a high price. [André]’s solution can be had for a fraction of that cost. Using a Raspberry Pi Zero, a tiny USB hub, and a high capacity jump drive, a photographer can simply plug in an SD card and the Pi will handle the backups with varying levels of automation. The software that [André] made use of is called Little Backup Box written by [Dmitri Popov] and can be used typically as an automatic backup for any other device as well.

This is a great solution to backing up files on the go, whether they’re from a camera or any device that uses an SD card. Removable storage is tiny and easily lost, so it’s good to have a few backups in case the inevitable happens. Raspberry Pis are an ideal solution to data backup, and can even be battery powered if you’re really roughing it for a few days.

Ask Hackaday: What’s In Your Digital Bugout Bag?

April 9, 2018 by Dan Maloney 119 Comments

Your eyes pop open in the middle of the night, darting around the darkened bedroom as you wonder why you woke up. Had you heard something? Or was that a dream? The matter is settled with loud pounding on the front door. Heart racing as you see blue and red lights playing through the window, you open the door to see a grim-faced police officer standing there. “There’s been a hazardous materials accident on the highway,” he intones. “We need to completely evacuate this neighborhood. Gather what you need and be ready to leave in 15 minutes.”

Most people will live their entire lives without a scenario like this playing out, but such things happen all the time. Whether the disaster du jour is man-made or natural, the potential to need to leave in a big hurry is very real, and it pays to equip yourself to survive such an ordeal. The primary tool for this is the so-called “bugout bag,” a small backpack for each family member that contains the essentials — clothing, food, medications — to survive for 72 hours away from home.

A bugout bag can turn a forced evacuation from a personal emergency into a minor inconvenience, as those at greatest risk well know — looking at you, Tornado Alley. But in our connected world, perhaps it pays to consider updating the bugout bag to include the essentials of our online lives, those cyber-needs that we’d be hard-pressed to live without for very long. What would a digital bugout bag look like?

Continue reading “Ask Hackaday: What’s In Your Digital Bugout Bag?” →

Solar Power In A Can!

March 2, 2018 by James Hobson 17 Comments

When spending time camping, people often bring lanterns, flashlights, and the like — you might even bring along a solar charger. Instructables user [bennelson] is combining all your electrical powered needs by cramming solar power into a can.

Already designed to resist the elements, [bennelson] is using a 50cal. ammo can for a portable enclosure. Inside, he’s siliconed a 15AH, 12V lead-acid battery in the centre to maintain balance and to leave room for the wiring and storage. One cardboard mockup later, he laser-cut the top panel from 1/8″ plywood and secured a 20A solar charge controller, a four-in-one socket panel, and two banana plugs on its top face.

[bennelson] is using 12 AWG wire to match the 20A rating of the solar charge controller — including a fuse for safety — and lever lock-nut connectors to resolve some wiring complications. Industrial velcro keeps the top panel in place and easily removed should the need arise. When he’s out camping, he uses an 18V, 1A solar panel to charge, but can still use a DC power adapter to charge from the grid. Check out the full build video after the break!

Continue reading “Solar Power In A Can!” →

Fail Of The Week: GitLab Goes Down

February 2, 2017 by Jenny List 57 Comments

Has work been a little stressful this week, are things getting you down? Spare a thought for an unnamed sysadmin at the GitHub-alike startup GitLab, who early yesterday performed a deletion task on a PostgreSQL database in response to some problems they were having in the wake of an attack by spammers. Unfortunately due to a command line error he ran the deletion on one of the databases behind the company’s main service, forcing it to be taken down. By the time the deletion was stopped, only 4.5 Gb of the 300 Gb trove of data remained.

Reading their log of the incident the scale of the disaster unfolds, and we can’t help wincing at the phrase “out of 5 backup/replication techniques deployed none are working reliably or set up in the first place“. In the end they were able to restore most of the data from a staging server, but at the cost of a lost six hours of issues and merge requests. Fortunately for them their git repositories were not affected.

For 707 GitLab users then there has been a small amount of lost data, the entire web service was down for a while, and the incident has gained them more publicity in a day than their marketing department could have achieved in a year. The post-mortem document makes for a fascinating read, and will probably leave more than one reader nervously thinking about the integrity of whichever services they are responsible for. We have to hand it to them for being so open about it all and for admitting a failure of their whole company for its backup failures rather than heaping blame on one employee. In many companies it would all have been swept under the carpet. We suspect that GitLab’s data will be shepherded with much more care henceforth.

We trust an increasing amount of our assets to online providers these days, and this tale highlights some of the hazards inherent in placing absolute trust in them. GitLab had moved from a cloud provider to their own data centre, though whether or not this incident would have been any less harmful wherever it was hosted is up for debate. Perhaps it’s a timely reminder to us all: keep your own backups, and most importantly: test them to ensure they work.

Thanks [Jack Laidlaw] for the tip.

Rack server image: Trique303 [CC BY-SA 4.0], via Wikimedia Commons.

How To Backup And Restore Your IP Camera Firmware

November 11, 2014 by Rick Osgood 8 Comments

[Filipe] has been playing around with custom firmware for inexpensive IP cameras. Specifically, he has been using cameras based on a common HI3815 chip. When you are playing around with firmware like this, a major concern is that you may end up bricking the device and rendering it useless. [Filipe] has documented a relatively simple way to backup and restore the firmware on these cameras so you can hack to your heart’s content.

The first part of this hack is hardware oriented. [Filipe] cracked open the camera to reveal the PCB. The board has labeled serial TX and RX pads. After soldering a couple of wires to these pads, [Filipe] used a USB to serial dongle to hook his computer up to the camera’s serial port.

Any terminal program should now be able to connect to the camera at 115200 baud while the camera is booting up. The trick is to press “enter” during the boot phase. This allows you to log in as root with no password. Next you can reset the root password and reboot the camera. From now on you can simply connect to the phone via telnet and log in as root.

From here, [Filipe] copies all of the camera’s partitions over to an NFS share using the dd command. He mentions that you can also use FTP for this if you prefer. At this point, the firmware backup is completed.

Knowing how to restore the backup is just as important as knowing how to create it. [Filipe] built a simple TFTP server and copied the firmware image to it in two chunks, each less than 5MB. The final step is to tell the camera how to find the image. First you need to use the serial port to get the camera back to the U-Boot prompt. Then you configure the camera’s IP address and the TFTP server’s IP address. Finally, you copy each partition into RAM via TFTP and then copy that into flash memory. Once all five partitions are copied, your backup is safely restored and your camera can live to be hacked another day.