The Best Migrations Happen Between Servers

I regularly migrate WordPress sites. As such I’m always looking for the quickest and simplest way to handle the migration. Recently I’ve been handling all of my migrations directly between servers using SSH. This will be a rough overview of what that looks like and why I enjoy it so much. 

Step 1: Make a backup on the existing WordPress site

While some web hosts have an easy to use 1 click backup, not all do. In order to standardize my process I use BackupBuddy, which only requires access to their WordPress backend in order to install and run a full site backup. Within BackupBuddy I make the following tweaks:

  • Enable backup reminders – Uncheck
  • Include ImportBuddy in full backup archive – Uncheck
  • Database – Use separate files per table (when possible) – Uncheck

After the backup is completed I right-click the “Download backup file” and copy the link address which I feed into my import script.

Step 2: Run SSH script on new WordPress site

With SSH you can run a locally stored script on a remote server. This saves you from needing to upload a script file, connecting to remote SSH, assigning execute permissions, and running the script.

Let’s assume I have a script named migrate.sh stored locally (see script file below). Running the following command locally will instruct the remote server to do the migration ssh remote-server-address "bash -s --" < migrate.sh --url="<backup-url>". Here’s an example of what that might look like.

# WP Engine's command prep
commandprep="cd sites/* &&"

# Kinsta's command prep
# commandprep="cd public/ &&"

ssh sitename@sitename.ssh.wpengine.net "$commandprep bash -s --" < ~/Scripts/migrate.sh --url="https://dev.anchor/wp-admin/admin-ajax.php?action=pb_backupbuddy_backupbuddy&function=download_archive&backupbuddy_backup=backup-dev_anchor-2018_04_08-full-z1ywr5zguz.zip" --update-urls

Currently the migrate.sh script works with Kinsta and WP Engine. It could easily be extended for other providers. It starts by finding a private folder to download the given backup URL. If the backup URL is in BackupBuddy’s format it converts to the publicly accessible location. It then extracts and imports files and database.

The site URLs will import as is which is generally what you want during a site migration.  However you can use this script to make a copy of the site to a different location with the argument ---update-urls which will change URLs to match the destination site.

It’s all about speed.

This is particularly useful for large sites. You don’t need to download the backup file as it’s directly transferred between the servers. This happens at lightning speeds. It always make me happy to see GBs of files transferring faster then possible, considering my internet connection, without using any of my own bandwidth.

#!/bin/bash

#
#   Migrate site from backup snapshot
#
#   `migrate.sh --url=<backup-url>`
#
#   [--update-urls]
#   Update urls to destination WordPress site. Default will keep source urls.
#

# Loop through arguments and separate regular arguments from flags (--flag)
for var in "$@"; do

  # If starts with "--" then assign it to a flag array
  if [[ $var == --* ]]; then
    count=1+${#flags[*]}
    flags[$count]=$var
    # Else assign to an arguments array
  else
    count=1+${#arguments[*]}
    arguments[$count]=$var
  fi

done

# Loop through flags and assign to variable. A flag "--email=my-email@my-site.com" becomes $email
for i in "${!flags[@]}"; do

  # replace "-" with "_" and remove leading "--"
  flag_name=`echo ${flags[$i]} | cut -c 3-`

  # detected flag contains data
  if [[ $flag_name == *"="* ]]; then
    flag_value=`echo $flag_name | perl -n -e '/.+?=(.+)/&& print $1'` # extract value
    flag_name=`echo $flag_name | perl -n -e '/(.+?)=.+/&& print $1'` # extract name
    flag_name=${flag_name/-/_}
    declare "$flag_name"="$flag_value" # assigns to $flag_flagname
  else
    # assigns to $flag_flagname boolen
    flag_name=${flag_name//-/_}
    declare "$flag_name"=true
  fi

done

backup_url=$url
backupformat=$( echo $backup_url | perl -n -e '/.+\.(.+)/&& print $1' )

# Store current path
homedir=$(pwd)

run_command() {

  # Find private folder
  if [ -d "_wpeprivate" ]; then
    private=${homedir}/_wpeprivate
  fi

  if [ -d "../private" ]; then
    cd ../private
    private=$(pwd)
    cd $homedir
  fi

  if [ ! -d "_wpeprivate" ] && [ ! -d "../private" ]; then
    echo "Can't find private folder '/_wpeprivate' or '../private'. Migration cancelled.";
    return 1
  fi

  # Verifies WordPress
  wp_home=$( wp option get home --skip-themes --skip-plugins )
  if [[ "$wp_home" != "http"* ]]; then
    echo "WordPress not found. Migration cancelled.";
    return 1
  fi

  cd $private

  if [[ "$backup_url" == *"admin-ajax.php?action=pb_backupbuddy_backupbuddy&function=download_archive&backupbuddy_backup"* ]]; then
    echo "Backup Buddy URL found";
    backup_url=${backup_url/wp-admin\/admin-ajax.php?action=pb_backupbuddy_backupbuddy&function=download_archive&backupbuddy_backup=/wp-content\/uploads\/backupbuddy_backups/}
  fi

  # Generate fresh snapshot directory
  timedate=$(date +'%Y-%m-%d-%H%M%S')
  mkdir -p restore_$timedate
  cd restore_$timedate

  # Downloads backup file (No local file found)
  if [ ! -f "$private/$url" ]; then
    wget --progress=bar:force:noscroll -O restore_$timedate.out $backup_url
  fi

  # Preps local restore file (local file found)
  if [ -f "$private/$url" ]; then
    echo "Local file '${url}' found. Renaming to 'restore_${timedate}.out'."
    mv "$private/$url" "$private/restore_$timedate/restore_$timedate.out"
  fi

  if [[ "$backupformat" == "zip" ]]; then
    mv restore_$timedate.out restore_$timedate.zip
    unzip -o restore_$timedate.zip -x "__MACOSX/*" -x "cgi-bin/*"
    rm restore_$timedate.zip
  fi

  if [[ "$backupformat" == "gz" ]]; then
    mv restore_$timedate.out restore_$timedate.gz
    tar xvzf restore_$timedate.gz
    rm restore_$timedate.gz
  fi

  if [[ "$backupformat" == "tar" ]]; then
    mv restore_$timedate.out restore_$timedate.tar
    tar xvzf restore_$timedate.tar
    rm restore_$timedate.tar
  fi

  # Finds WordPress path
  wordpresspath=$( find * -type d -name 'wp-content' -print -quit )

  # Migrate blogs.dir if found
  if [ -d $wordpresspath/blogs.dir ]; then
    rm -rf $homedir/wp-content/blogs.dir
    echo "Moving: blogs.dir"
    mv $wordpresspath/blogs.dir $homedir/wp-content/
  fi

  # Migrate uploads if found
  if [ -d $wordpresspath/uploads ]; then
    rm -rf $homedir/wp-content/uploads
    echo "Moving: uploads"
    mv $wordpresspath/uploads $homedir/wp-content/
  fi

  # Migrate themes if found
  for d in $wordpresspath/themes/*/; do
    rm -rf $homedir/wp-content/themes/$( basename $d )
    echo "Moving: themes/$d"
    mv $d $homedir/wp-content/themes/
  done

  # Migrate plugins if found
  for d in $wordpresspath/plugins/*/; do
    rm -rf $homedir/wp-content/plugins/$( basename $d )
    echo "Moving: plugins/$d"
    mv $d $homedir/wp-content/plugins/
  done

  # Find non-default root level files and folders
  cd $private/restore_$timedate/$wordpresspath/..
  default_files=(index.php license.txt readme.html wp-activate.php wp-app.php wp-blog-header.php wp-comments-post.php wp-config-sample.php wp-cron.php wp-links-opml.php wp-load.php wp-login.php wp-mail.php wp-pass.php wp-register.php wp-settings.php wp-signup.php wp-trackback.php xmlrpc.php wp-admin wp-config.php wp-content wp-includes)
  root_files=($( ls ))
  for default_file in "${default_files[@]}"; do
    for i in "${!root_files[@]}"; do
      if [[ ${root_files[i]} == "$default_file" ]]; then
        unset 'root_files[i]'
      fi
    done
  done

  # Move non-default root level files and folders
  for file in "${root_files[@]}"; do
    rm -rf $homedir/$file
    echo "Moving: $file to $homedir/"
    mv $file ${homedir}/
  done
  cd $homedir

  # Remove select plugins if found
  wp plugin delete backupbuddy wp-super-cache adminer wordfence w3-total-cache wp-file-cache broken-link-checker yet-another-related-posts-plugin comet-cache-1 woothemes-updater ewww-image-optimizer https-redirection really-simple-ssl hello wordpress-php-info force-strong-passwords --skip-plugins --skip-themes

  # Outputs table prefix and updates if different
  cd $private/restore_$timedate/$wordpresspath/../
  if [ -f wp-config.php ]; then
    cat wp-config.php | grep table_prefix
    table_prefix=$( cat wp-config.php | grep table_prefix | perl -n -e '/\047(.+)\047/&& print $1' )
  fi

  cd $homedir
  current_table_prefix=$( wp config get table_prefix )
  if [[ $table_prefix != "" && $table_prefix != "$current_table_prefix" ]]; then
    wp config set table_prefix $table_prefix
  fi

  # Grabs current privacy settings
  search_privacy=$( wp option get blog_public --skip-plugins --skip-themes )

  echo "Resetting default file and folder permissions"
  find . -type d -exec chmod 755 {} \;
  find . -type f -exec chmod 644 {} \;

  # Secure database
  chmod 600 $homedir/wp-content/mysql.sql

  echo "Found the following database:"
  find $homedir/wp-content/uploads $private/restore_$timedate/ -type f -name '*.sql'
  databases=$( find $private/restore_$timedate/ -type f -name '*.sql' )
  database_count=$( echo -n $databases | grep -c '^' )

  if [[ "$database_count" == "0" ]]; then
    # Expand db search
    databases=$( find $homedir -type f -name '*.sql' )
    database_count=$( echo -n $databases | grep -c '^' )
  fi

  if [[ "$database_count" == "0" ]]; then
    echo "Database not found. Skipping database import.";
    return 1
  fi

  # select first database and import
  database_file=$( echo "$databases" | head -1 )

  if [ ! -f "$database_file" ]; then
    echo "Database $database_file not found. Skipping database import.";
    return 1
  fi

  wp db reset --yes
  wp db import $database_file
  wp cache flush --skip-plugins --skip-themes

  # Reapply search privacy
  wp option update blog_public $search_privacy --skip-plugins --skip-themes

  # Fetch imported url
  wp_home_imported=$( wp option get home )

  if [[ "$update_urls" == "true" ]]; then
    echo "Updating urls from $wp_home_imported to $wp_home"
    wp search-replace $wp_home_imported $wp_home --skip-plugins --skip-themes --all-tables --report-changed-only
  fi

  # Convert MyISAM tables to InnoDB
  wp db query "SELECT CONCAT('ALTER TABLE ', TABLE_SCHEMA,'.', TABLE_NAME, ' ENGINE=InnoDB;') FROM information_schema.TABLES WHERE ENGINE = 'MyISAM'" --skip-column-names > db_optimize.sql
  wp db query < db_optimize.sql
  rm db_optimize.sql

  # Flush permalinks
  wp rewrite flush --skip-plugins --skip-themes

}
run_command