Tags


Syncing files from Android to GNU/Linux

First written on August 4, 2018
Last updated on September 13, 2021

Overview

Backups of mobile phone content have been a problem I wanted to solve for years but unfortunately the tools to do a clean job have always been inadeguate. Also, I certainly didn’t want to rely on third party services such as Dropbox, Google, etc…

The first solution was to use bluetooth with an APP I can’t even remember the name of. The problem was its slow speed and non-secureness.

So I started using micro SD cards and manual copying… with the result of losing several times all media on the card because its filesystem got corrupted.

After searching intensively I finally found a reliable and automatic way for mobile phone backups. There is this neat APP called Primitive FTPD that includes an SFTP server. The advantages of SFTP instead of using other systems are:

Steps

#!/usr/bin/env bash

# The full path of the configuration file
CONFIGURATION="${1}"

{ command -V rsync && command -V bash && command -V mail \
    && command -V printf && command -V sync; command -V fusermount3; } \
    || exit 1
{ [ ! -f "${CONFIGURATION}" ]; } \
    && printf "%s\n" 'check configuration file' 2>&1 && exit 1

. "${CONFIGURATION}"

# See https://gist.github.com/mfellner/5743990
RSYNC_MAIN_OPTS='-rv --size-only --no-group --no-owner --no-perms --no-times --whole-file --inplace'

START=$(date +%s)

# Mount only if not already mounted.
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
if [ -z "${exists_mount}" ]; then
        echo "${PASSWORD}" | sshfs -o password_stdin -p "${PORT}" "${USERNAME}"@"${ADDRESS}": "${MOUNT_DIR}"
fi

# Might not be mounted if device not in range or if its server is offline.
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
if [ -z "${exists_mount}" ]; then
        echo ""${ALIAS}" not connected"
        exit 1
fi

rsync ${RSYNC_MAIN_OPTS} "${MOUNT_DIR}"/* "${DST_DIR}"
ret_rsync=${?}
sync
fusermount3 -u "${MOUNT_DIR}"

FINISH=$(date +%s)
if [ ${ret_rsync} -eq 0 ]; then
    if [ "${LOG_TO_FILE}" = 'true' ]; then
        printf "%s\n" "[$(date '+%Y-%m-%d, %T') OK backup_mobile_phone] "${ALIAS}" ; \
$((${FINISH}-${START}))s" >> "${LOG_FILE}"
    fi
    if [ "${LOG_TO_EMAIL}" = 'true' ]; then
        printf "%s\n" "Backup on "${ALIAS}" took $((${FINISH}-${START}))s" \
            | mail -r "${EMAIL_SENDER}" -s 'Backup complete' "${EMAIL_RECIPIENT}"
    fi
else
    if [ "${LOG_TO_FILE}" = 'true' ]; then
        printf "%s\n" "[$(date '+%Y-%m-%d, %T') FAILED backup_mobile_phone \
ret_rsync=${ret_rsync}] "${DST}"" \
        >> /var/log/rsync.log
    fi
    if [ "${LOG_TO_EMAIL}" = 'true' ]; then
        printf "%s\n" "Backup error on "${ALIAS}". Error code ${ret_rsync}" \
            | mail -r "${EMAIL_SENDER}" -s 'Backup error' "${EMAIL_RECIPIENT}"
    fi
fi
USERNAME='username0'
PASSWORD='password0'
ADDRESS='1.2.3.4'
PORT='65432'
MOUNT_DIR='/home/user/mountpoints/user0'
DST_DIR='/home/user/backup/user0'

# Friendlier name.
ALIAS='user 0'

LOG_TO_EMAIL='true'
EMAIL_SENDER='Computer <your.email@address.com>'
# You can use aliases if your mailing system is set up correctly.
EMAIL_RECIPIENT='all'

LOG_TO_FILE='true'
LOG_FILE='/var/log/rsync.log'
$ ${PATH_TO_SCRIPT} ${PATH_TO_CONFIGURATION_FILE}

Problems and explanation

(Rsync over SFTP) or (Rsync over directory over SSHFS)

Since Primitive FTPD is an SFTP server, we can’t use rsync directly on SFTP because this is not supported by rsync (see: $ man 1 rsync). For this reason we must first mount the directory using SSHFS and then rsyncit.

(Public key) or (password authentication)

I was not able to use public key authentication because of Pftpd reported some obscure error with it. Initially I wasn’t able to find Pftpd’s log file in the phone’s flash memory but I noticed it after several sync operations. Going through the file didn’t clear the situation at all. Even the project issues didn’t solve the problem. So, the only remaining solution was to use password authentication. Since all the process needs to be automatic, the password needs to be stored plaintext and passed through the password_stdin argument. This is of course not the best setup.

Storage type

Depending on the Android version and device, the type of storage selected from Pftpd might not work and behave differently than expected, so it is kind of a trial and error operation. In my case the option Plain old filesystem worked best, although the location that will be synced will usually be /storage/emulated/0 and cannot be changed to use, for example, the external SD card.

Alternatives

There aren’t many free software alternatives that work like this for Android. What I found either didn’t involve SSH or lack some functionalities. Syncopoli, for example, was another possible option. The readme file states that it is an Rsync-based application optionally working via SSH and public key authentication, however it is a client, not a server.

Using a client avoids the computer polling for the connection, so it’s a plus. The only problem is that I need to do extra operations like maling and writing to log files after the syncronization finishes. This is not possible with Syncopoli.

~

Have fun!


Post a new comment