Backing up images with the Raspberry Pi

People who take pictures when they travel frequently want to have a large memory storage for saving images. Commercial providers offer a series of relevant solutions as WiFi hard drives. These solutions are often well equipped on the hardware side but lacking when it comes to firmware. A self-built storage device based on the Pi 3 lets the user take complete control over their storage device.

The goal of this project is to make a system that is as easy to operate as possible. From the moment you plug the camera memory card into the Raspberry Pi, image copying should begin automatically and function smoothly and quickly. At the end of the process, the Pi should shut down cleanly. Unfortunately, the Raspberry Pi is missing a few pieces of basic equipment for this kind of undertaking. There is no display screen to report the current status and it also needs a keyboard before it will shut down automatically.

Luckily there are simple add-ons for the Raspberry Pi baseboard that provide the assistance that you need. Some time ago, the Raspberry Pi Foundation formulated standards for add-ons. A HAT (Hardware Attached on Top) is an add-on board that conforms to these rules. The requirements for our applications are fairly moderate. A few LEDs inform the user about what is currently running. Buttons are used to initiate shut down.

If you are handy with a soldering gun, you'll have a clear advantage when working on this project. There are some nifty and economical components, like Berry Clip [1] and Fish Dish [2], that have the board and all of the necessary parts like LEDs, resistors etc. ready for soldering. I am a pure software guy and so prefer a solution that comes pre-assembled.

This project uses the very simple Pibrella HAT [3] from Pimoroni. Display-O-Tron [4], which is also from Pimoroni, is a good alternative. (We talked bout the Display-O-Tron in issue #20 Of Raspberry Pi Geek – check out the "Less is more" article [5]). However, first we'll address the primary goal which is to automatically copy data from the memory card to the hard drive.

Installation

In terms of requirements, you will need a third generation Raspberry Pi with a connected hard drive, although a Pi 2 will do as well. You will also need the operating system Jessie Lite. The system should be installed as usual on an SD card. You should then connect and partition the hard drive (Listing 1, line1).

$ sudo fdisk /dev/sda
[...]
$ sudo mkfs.ext4 /dev/sda1
$ sudo mkdir /data
$ sudo mount /dev/sda1 /data
[...]

Create a single partition that contains all available storage space on the disk. You should bear in mind that any data already on the disk gets lost during this process. The [W] key rewrites the data and closes Fdisk. Format the data, create a mount directory and mount the drive directly on it (Listing 1, lines 3 to 5). Add the lines from Listing 2 to the /etc/fstab file to make sure the system will automatically mount the disk in future.

/dev/sda1 /data ext4 user_xattr,noatime,acl 1 2

If this all seems too complicated, skip to the directions outlined in the Automatic Installation info box. Since both the data and the operating system are on the hard drive, Raspbian runs somewhat more smoothly than it would from an SD card. Install the Rsync utility from the Raspbian package sources for copying with

$ sudo apt install git
$ git clone https://github.com/bablokb/apiinst
$ sudo apiinst/bin/apiinst -i /pfad/zu/20160527-raspbian-jessie-lite.zip -B /dev/sdb -t /dev/sdc -D 285G

sudo apt install rsync

Automatic Copying

After inserting a memory card using an SD card reader, the system should save the data to the hard drive itself. It will need two things in order to do this. The first is a Udev rule and the second is the actual copy script. The Udev manager operates in the background and monitors the system. It automatically executes predefined actions when devices are plugged in and unplugged.

The one-liner in /etc/udev/rules.d/99-usbcopy.rules the file from Listing 4 tells the daemon that it should execute the script /usr/local/sbin/copy_img for new devices with names that begin with "sd". The script receives the device names (in "$kernel") via parameters. Typically it is not necessary to restart after setting up an udev rule.

KERNEL=="sd?", SUBSYSTEMS=="usb", ACTION=="add", RUN+="/usr/local/sbin/copy_img $kernel"

You can see the script itself in Listing 5. It has been reduced to the bare essentials. A more elaborate version which includes tests and data logging is available on the my project page [6]. The script filters out the hard drive /dev/sda which is used for data storage because its name begins with "sd" (line 3). Copying is performed using the Rsync command on line 7. Lines 5 and 9 are explained in the following sections.

01 #!/bin/bash
02 [ "$device" = "sda" ] && exit 0
03 /usr/local/sbin/hatctl.py AS
04 mount "/dev/${device}1" /mnt
05 rsync -a --no-owner --modify-window=1 /mnt/DCIM/*/* "/data"
06 umount /mnt
07 /usr/local/sbin/hatctl.py AE

$ sudo apt-get update
$ sudo apt-get install python-pip
$ sudo pip install pibrella