Rclone is an amazing command line app for syncing files virtually anywhere. With recent added support for SFTP it's become a core part of my daily backup script. You can connect any number of remote storage systems. If you're only working with a few websites, then importing SFTP credentials isn't necessary as you can manually add using the interactive command rclone config
. However with hundreds of websites, automatically importing credentials is a essential.
Each website needs a unique identifier
Using Rclone for backup is just one use case. There are many other potentional uses, like site migration, deployments and restoring. Before automating, each website needs a unique identifier. Within Rclone I use the format sftp-websitename for the name of the remote.
With the website name unique, you can safely add all of your WordPress websites and use across a wide range of bash scripts.
This small example, part of another bash script, moves active plugins from one WordPress site to another. The sshwpe.sh
file is a SSH wrapper for WP Engine's SSH beta. Will cover that in future post.
installnamesource=website1
installnamedestination=website2
activeplugins=`sshwpe.sh $installnamesource "wp plugin list --status=active --field=name"`
for plugin in $activeplugins; do
rclone sync sftp-${installnamesource}:wp-content/plugins/${plugin} sftp-${installnamedestination}:wp-content/plugins/${plugin} -v
done
By itself the following script doesn't do much other then make a new one-liner for adding credentials to Rclone. The simplicity of a one-liner enables other scripts to import credentials directly. Every time I prep a new WordPress installation this script runs. See comments for usage format.
Importing credentials from command line
<?php
##
## Preps new install for rclone via ~/.rclone.conf
##
## Pass arguments from command line like this
## php rclone_import.php install=anchorhosting domain=anchor.host username=anchorhost password=random protocol=sftp port=2222
##
if (isset($argv)) {
parse_str(implode('&', array_slice($argv, 1)), $_GET);
}
$install = $_GET['install'];
$address = $_GET['address'];
$username = $_GET['username'];
$password = $_GET['password'];
$protocol = $_GET['protocol'];
$port = $_GET['port'];
# rclone obscure password
$password = shell_exec('rclone obscure '. $password);
$file_rclone_config = $_SERVER['HOME'] . '/.rclone.conf';
if (!file_exists($file_rclone_config)) {
// Try alternative location
$file_rclone_config = $_SERVER['HOME'] . '/.config/rclone/rclone.conf';
}
$file = file_get_contents($file_rclone_config);
$pattern = '/\[(.+)\]\ntype\s=\ssftp\nhost\s=\s(.+)\nuser\s=\s(.+)\nport\s=\s(\d+)\npass\s=\s(.+)/';
preg_match_all($pattern, $file, $matches);
$found_install = false;
foreach ($matches[1] as $key => $value) {
$prefix = 'sftp_';
$value = substr($value, strlen($prefix));
if ($value == $install) {
$found_install = true;
}
}
if ($found_install != true) {
# Add to .rclone.conf file
$lines = explode( PHP_EOL, $file);
$line_count = count($lines);
# Add new install to end of array
$new_lines = array_slice($lines, 0, $line_count, true) +
array("1n" => "[sftp-$install]") +
array("2n" => "type = $protocol") +
array("3n" => "host = $address") +
array("4n" => "user = $username") +
array("5n" => "port = $port") +
array("6n" => "pass = $password") +
array("7n" => "");
# outputs new additions to file
$new_content = implode( PHP_EOL, $new_lines);
file_put_contents($file_rclone_config, $new_content);
}