[Matthew’s] recent blog post does a good job explaining the basics of the Raspberry Pi’s file system. The Linux operating system installed on a Pi is generally installed on two different partitions on an SD card. The first partition is a small FAT partition. All of the files on this partition are used for the initial booting of the Pi. This partition also includes the kernel images. The second partition is the root file system and is generally formatted as ext4. This partition contains the rest of the operating system, user files, installed programs, etc.
With that in mind you can deduce that in order to backup your Pi, all you really need to do is backup all of these files. [Matt] has written some scripts to make this a piece of cake (or pie). The first script will simply copy all of the files into a gzipped archive. You can save this to an external SD card, USB drive, or network share.
The second script is perhaps more interesting. This script requires that you have one free USB port and a USB SD card reader. The script will automatically format the extra SD card to contain the two critical partitions. It will then copy the “boot” files to the new boot partition and the root file system files to the new SD card’s root partition. When all is said and done, you will end up with an SD card that is an exact copy of your current running file system.
This can be very handy if you have multiple Pi’s that you want to run the same software, such as in a Pi cluster. Another good example is if you have spent a lot of time tweaking your Pi installation and you want to make a copy for a friend. Of course there are many ways to skin this cat, but it’s always fun to see something custom-built by a creative hacker.
Forgive me for missing the point, but how is this better than just making a full image of the SD card with either win32diskimager, or dd?
Because it is powered on ? And yes powered off would be better.
I copied the harddisk contents, with dd, on a running system to another harddisk the other day. This was a fully booted Ubuntu install. Sure, it’s a tiny bit risky, but in generally it works. (the old harddisk was failing, and I didn’t want to spend too much time on it)
clonezilla
I usually just image the entire sd card, too. If you zero out the free space first, it compresses really well. However, I see the benefits of the article’s technique, too. That method allows you to create backups while the pi is running, which means no downtime and/or automated regular backups. It’s just an alternative, they both have pros and cons.
Hi, I’m the author here (@sysmatt). #1 using dd you need to resize, my method does not, #2, this can be done online (if you stop services that write to databases etc), #3, you dont waste any time reading/writing zeroed blocks, #4 You can easily customize the destination copy before booting it.
To turn on your computer, press the power button with your finger. You can use a pencil instead of a finger so that it won’t hurt that much.
If your using this to make a “clone” to use in another pi, be sure to generate new keys and change the hostname.
If you are running a database server, then you should use one of their utilities to back up the database, esp. if the database is running when you want to make a backup.
Of course it should be stated that you really can’t trust your backups unless you have actually successfully restored them. There are lots of stories out there about people who dutifully went through the motions of backing up their data only to discover that there were issues with restoring it, only discovered in the dire moments of disaster recovery.
I think that a good use of this would be to run it as a CRON job so that if your file system becomes corrupted by an abrupt power down you would have a recent copy to use to restore it. It would be nice to be able to choose where you would like to boot from, the default or the backup, but a quick SD swap would do the same thing. With my luck I would lose power in the middle of the CRON job and have two corrupted SD cards.
Warning:
Choose the devices carefully this is only an example, otherwise you will overwrite your data!
Hi,
as mentioned before cloning an sd-card is also a good choice to backup that specific state – two sdcards similarly sized.
da0(original) -> da1(target)
under FreeBSD you can do “cp /dev/da0 /dev/da1”
also “dd if=/dev/da0 of=/dev/da1 bs=8M”
both solutions will copy the data rather quickly, please mind putting the bs/block size
parameter to 4-16M byte otherwise the transfer speed will be very slow.
under Linux you can only “dd if=/dev/sda of=/dev/sdb bs=8M”
These solutions work also if you have two – similarly sized – harddisks and want to clone them, on harddisks bs should be 16-32M.
This method will clone everything including gpt/mbr/part-table
dd over ssh is my preferred approach in situations like these where the only available image is on a running system.
On our digital signage system we just clone the whole card, on boot a scrips checks if the serial has changed and does the changes required for the system to work on the new raspi, when the system requires updates (we use a simple timer) the serial is always supplied to the webservice (among other things such as uptime, screen resolution and orientation, etc), this allows us to individually control all raspis on the network with them all using the same image, it also means that if one card somehow gets corrupted you can just clone any other card on the store and the contents displayed will always be right as they are dependent on the serial
That sounds like an awesome solution! you wouldn’t happen to have anything you would be willing to share, I have been looking to set something similar up for myself.
simplified version of our .profile (machine set to auto login)
http://pastebin.com/qVcWiVsz
I figure you need a couple of python files too Yehh ?
The python files are a custom configuration menu (cfgmenu.py) so users can change some basic app and system settings without calling support (screen rotation, wifi network, etc) and the main program updater (update.py) that also runs the main program once finished.
Note that the update.py updates the main python program, it self and the dependencies, the presentation files are downloaded and stored in a cache by the main program, this 2 shot update allows us to update the whole OS if necessary (whole network update) or just the images and movies (individually or in group controlled by serial).
We have raspis from October 2012 that where never touched after being deployed and they are running our latest software.
We also have some nice things going at the back end:
* We have a local cache in each store so we don’t have 10-20 raspis per store hammering the network trying to download from central.
* The local servers are synced using a torrent inspired system (multi TB cache), this way the central server doesn’t carry the whole network load
* Each new raspi sets itself up and automaticaly registers itself on the system, at initial (auto) configuration we use the address from the DHCP server to identify in witch store the raspi is located, we could set static IPs at this point but decided against.
* All raspis have local cache, it was originally made to lower bandwidth and allow for network failures, but the stores started to use this for out of door events without network connection.
* If possible errors are reported to our central alert system and shown to the whole team responsible, think off it like a error reading a file, could mean that a card died or more usual that the raspi it’s using a crappy psu
TL;DR: proud father
nelsontb, sounds like exactly what I’ve been trying to do, though my limited programming skills have been slowing me down.
So tell me more about this cat skinning?