Tutorial: Managing a WordPress installation in Git using WP Bootstrap

In the previous tutorial, we covered how to set up a simple WordPress site using WP Bootstrap, essentially preparing the site for git and git based deployments. In this tutorial we’re taking the next step. We’re going to add all the needed files to a Git repository and after that we’re going to deploy the code into a production environment.

Update 2016-01-11: After this post was first published, I’ve published wp-deployhelper, a tool intended to speed up deployments to shared hosts. If you’re interested in fast shared host deployments I strongly suggest you have a look at wp-deployhelper.

We’ll look at how we can deploy changes to the production environment regardless if you have ssh access with composer and wp-cli available, or if you’re working with a standard shared host using ftp access. Full ssh access to production is going to be better 999 out of 1000 times. But many WordPress developers have little or no influence over where their customers sites are hosted and we want to document a solid work flow even for this (very common) scenario.

The deployment technique outlined here is based on the very most basic git usage with a manual pull in the target environment. There are other more sophisticated ways like Capistrano or Rocketeer to get the job done and those will be covered at another time, in this tutorial focus is on the basics.

Overview

The steps covered in this tutorial are:

  • Initiating a local git repository and adding files to a Github repository
  • Checking out the project files on the production environment and set up the site for the first time
  • Making changes in the development environment and deploying them to production

 

Step 1: Initiating a local git repository and adding files

This step assumes that you have already created a repository on Github. We’re using a repository named “tutorial-simplesite1” for this part. Note that some of the commands below contains both the Github username and repository names, make sure you modify these if you copy/paste directly from this tutorial. You also need to have git installed and setup properly, it’s included by default inside the Vagrant environment we’re using. But apart from installing it, you will also need to set up name and email address and perhaps add an ssh key. Please refer to the help pages on Github on how to get started with Git and how to set up keys.

Make sure you the Vagrant machine is running (vagrant up) and that you are in a shell inside the machine (vagrant ssh) in the path where the development environment is ($ cd /vagrrant).

First, we’ll create a local git repository directly in the development environment;

$ git init

Create a .gitignore with the following

localsettings.json
.vagrant/
staging/
www/
vendor/
wp-cli.yml
composer.lock

Add the config files

$ git add appsettings.json composer.json .gitignore

Add vagrant files

$ git add Vagrantfile vagrant/

Add content and local files

$ git add bootstrap/ wp-content/

Then commit the files

$ git commit -m "initial import"

Set up the connection with Github and push, note that this assumes you are using Github with the same repository name as in this tutorial. Replace hostname, username and repository name to suit your conditions.

$ git remote add origin git@github.com:yourusername/tutorial-simplesite1.git
$ git push -u origin master

 

 

Step 3: Checking out the files on the production server

Note: In your production environment, I strongly discourage keeping the project files directly in the web root folder. In most hosting scenarios, you can keep the project files in a folder that’s not directly accessible via the web. If you have no choice, you must remove all sensitive information from localsettings.json once WordPress has installed for the first time.

Scenario 1. A server you can access via ssh with wp-cli and composer are installed.

In this scenario, we’re assuming that you are are working in your users home directory and that the web server is configured to use /var/www/tutorial-simplesite1. We’re also assuming that you have created a database and have the user name and password available.

Log in to the server and navigate to the folder where you want to keep your local copy of the code. Note that during checkout, a sub folder will be created, so run this command from one level above. Here we’ll create a sub folder projects under the users home directory:

$ cd /home/user/
$ mkdir projects
$ cd projects
$ git clone git@github.com:yourusername/tutorial-simplesite1.git

Then install WP Bootstrap via composer

$ composer update

…and initialize WP Bootstrap for access via Composer scripts:

$ vendor/bin/wpbootstrap wp-init-composer

Next, generate a localsettings file:

$ composer wp-init

Edit the contents of localsettings.json to suit your server:

{
    "environment": "production",
    "url": "www.example.com",
    "dbhost": "localhost",
    "dbname": "wordpress",
    "dbuser": "wordpress",
    "dbpass": "database_password",
    "wpuser": "admin",
    "wppass": "strong_and_long_pass_1234!",
    "wppath": "/var/www/tutorial-simplesite1"
}

Install WordPress, set up plugins and themes and import the content

$ composer wp-install
$ composer wp-setup
$ composer wp-import

That’s all for this phase.

 

Scenario 2. Standard shared hosting with only ftp access

In this scenario, we’re assuming that the production server is perfectly normal shared host. The steps in this section are tested against a simple $3.95/month account on Bluehost but the steps should be easy to reproduce on most shared hosts. To follow these steps you need to have ftp username/password as well as the database credentials, host, database name, username and password.

The next perquisite for this scenario to work is that your shared host allow remote access to the MySQL server. Some hosts have no restrictions for this at all, while Bluehost and many other hosts allow you to open access to the MySQL server for a specific IP address. Bluehost outlines the steps needed here: Remote Database Connection Setup. If you’re on a different host you can probably find help via their support pages.

Using Git to deploy your site to a shared host is based on the following idea:

  • We will use a local vagrant machine to check out the code, just as the first steps above
  • When creating the localsettings.json file, we will use the credentials for the production database on the shared host
  • Then we run the WP Bootstrap scripts to install, setup and import
  • At last, we will transfer the file tree to the shared host using ftp

If you are using the Vagrant skeleton that was suggested in the previous tutorial, you should already have a staging environment in your local vagrant machine that we will use.

First make sure your vagrant machine is up and running (vagrant up) and that you have ssh’ed into it (vagrant ssh). Then at the command line in the vagrant box,

$ cd /srv/staging

The nginx default setting in the Vagrant box is that the staging WordPress installation should be at /srv/staging/wordpress, so create that folder:

$ mkdir /srv/staging/wordpress

Then clone the code

$ git clone git@github.com:yourusername/tutorial-simplesite1.git

Then install WP Bootstrap via composer

$ composer update

…and initialize WP Bootstrap for access via Composer scripts:

$ vendor/bin/wpbootstrap wp-init-composer

Next, generate a localsettings file:

$ composer wp-init

Edit the contents of localsettings to suit the your domain name database credentials for your shared host. Note that we’re still pointing at a local folder for the installation. That’s because we’re setting everything up there first.

{
    "environment": "production",
    "url": "www.yourdomainname.com",
    "dbhost": "box000.bluehost.com",
    "dbname": "your_database_name",
    "dbuser": "your_database_user",
    "dbpass": "secret_password",
    "wpuser": "admin",
    "wppass": "password",
    "wppath": "/srv/staging/wordpress"
}

Install WordPress, set up plugins and themes and import the content

$ composer wp-install
$ composer wp-setup
$ composer wp-import

After this, you should be able to see your site via the local URL to the staging server, but it will be slow and look terrible. The reasons for that is that (1) it’s using the database on your shared host which will be a lot slower that a local and (2) all internal links, including those for css and image files are now pointing to the production URL. We will not adress those issues right now, instead we’re going to do the very last step, to transfer the files to the shared host.

The bulk of the work will be done using a tool named sitecopy. It’s been around for a long time and it’s sole purpose is to ensure that a remote version of a site is identical to a local version via FTP. It’s a good fit four our needs. First we need to install sitecopy.

$ sudo apt-get install sitecopy

Next, we’ll create a folder where it keeps its state and the file that controls the behaviour of sitecopy:

$ mkdir ~/.sitecopy
$ chmod 0700 ~/.sitecopy
$ touc ~/.sitecopyrc
$ chmod 0600 ~/.sitecopyrc
$ nano ~/.sitecopyrc

And add the following

site remote
    server ftp.yourdomain.com
    remote /public_html
    local /srv/staging/wordpress
    username yourusername
    password "secret_pass"
    symlinks follow

Exit save the file using Ctrl+o and then exit using Ctrl+x.
Then we initiate sitecopy and run the first update

$ sitecopy remote --init
$ sitecopy remote --update

Sadly, since we’re dealing with ftp this step is going to take some time, expect 15-30 minutes even for a small WordPress site. But the advantage of using sitecopy is that it’s fairly quick on subsequent updates, at least compared to other ftp tools. If you run the update again, the process should finish in just a few seconds.

Additional notes
On some hosts, Bluehost is one, the database host name in wp-config.php must be ‘localhost’ in order for the site to function properly. But in our WordPress staging installation we have set the host name to ‘box000.bluehost.com’. The best workaround for this is to go in via a standard ftp client and edit the file directly on the server. Then, in order to avoid that this file gets overwritten at future deploys, we exclude it from the scope using a directive called “exclude” in the .sitecopyrc file:

site remote
    server ftp.yourdomain.com
    remote /public_html
    local /srv/staging/wordpress
    username yourusername
    password "secret_pass"
    symlinks follow
    exclude wp-config.php

Scenario 2 summary

As we’ve seen, there is a straight forward way to maintain a WordPress site using proper git version control even if we have to deploy the site to a very simple shared host.

But this approach has a few downsides. Performance is one, even a simple WordPress site can take very long for the first deploy to finish. But even subsequent deploys can be challenging since the database might be updated minutes before all new files have been copied to the server.

Another downside is that this is far from as robust as a normal deploy with git and composer directly on the server. For instance, changes to files in core or in plugins that happen on the server are not automatically detected. So sitecopy might think that everything is in sync between staging and production even if there are major differences. Updating plugins and core on the production server is simply not recommended when using this approach. If that happens and the file trees come out of sync, you need to either manually edit the state that sitecopy maintains in ~/.sitecopy/remote (an xml file) or run the command sitecopy remote –fetch to get the current state from the server.

Alternate approaches

There are a few alternative approaches to keep two file trees in sync via ftp. One is anohter command line tool called lftp that has a pretty powerful feature to mirror two trees, if for some reason sitecopy does not work out for you, lftp might be an alternative.

However, the biggest downside with using ftp for keeping two file trees in sync is that ftp is very slow when it has to deal with large amounts of files and folders. Not because of the connection speed, but because there will be a huge amount of individual ftp requests and each request has a little bit of overhead time.

Some hosts, including Bluehost, offer a limited ssh access to the server, enough to run another file synchronization tool called rsync. Bluehost has an option where you can verify your account with their verification department and after that you can get limited ssh access to your hosting server. I will describe how to set up a rsync based deployment in a later post or tutorial, for now you should at least examine the options on your host. Here’s an article that explains how to get ssh access on Bluehost.

Step 4: Creating changes

Before heading on to the final step in this tutorial, we’re going to make a few changes in development so that we have something to deploy. We’ll make the following changes in the development environment:

  • Modify code in the child themes functions.php
  • Modify the title on the welcome page to say “Welcome (git deploy)”

By making these two simple changes, we’ll cover deployment of changed code, normally pretty easy to transfer. But we’re also covering a change that took place in the database, which often is a little bit harder.

First, make sure your vagrant box is still running and use your browser to surf to the development version of your site http://www.casestudy1.local/wp-admin. Log in to the admin area and make the modification to the welcome page. Change the title and feel free to do some changes in the content we created with Page builder. Save the page when your’re done.

Next we’ll make a little change in the child theme. Edit the file [project_folder]/wp-content/themes/vantage-child/functions.php and add the following code under the existing code:

add_filter( 'the_title', 'vantage_child_the_title',10, 2);

function vantage_child_the_title($title, $id = null)
{
    return $title . " [I'm added via code]";
}

Save the file and go have a look at http://www.casestudy1.local/, you should see that the title has changed, both the changed you did in the page editor as well as the text [I’m added via code]”. Once you have confirmed this, let’s bring up the terminal. Make sure to ssh into the Vagrant machine (vagrant ssh) and go to the correct folder:

$ cd /vagrant

Then we’re going to create a new export so that our database changes are stored in files in the bootstrap folder:

$ composer wp-export

Next, we’ll use git to find out what’s changed (output shortened):

$ git status
Changes not staged for commit:
	modified:   bootstrap/config/wpbootstrap.json
	modified:   bootstrap/manifest.json
	modified:   bootstrap/media/camera-1080/meta
	modified:   bootstrap/posts/attachment/camera-1080
	modified:   bootstrap/posts/ml-slider/new-slider
	modified:   bootstrap/posts/page/welcome
	modified:   bootstrap/taxonomies/ml-slider_manifest.json
	modified:   wp-content/themes/vantage-child/functions.php

WP Bootstrap regenerates all files in the bootstrap folder at wp-export, and if the internal ID’s changed during the last import process, chances are that there are small modifications in many files. We see our modified functions.php as well as the welcome page in the list of modified files, so it seems reasonable. We’ll commit these files and push them to the remote repository.

$ git add bootstrap/ wp-content/
$ git commit -m "A few changes"
$ git push

And that’s all for this part, all our changes are now stored safely in Git.

 

Step 5: Deploying the changes to production

So, in the last step in this tutorial we’re going to deploy these changes into production.

Scenario 1. A server you can access via ssh with wp-cli and composer are installed.

Log in to the server with ssh and navigate to the folder where you want to keep your local copy of the code (the folder where localsettings.json is located). Then do a git pull go get all the changes:

$ git pull

Then, apply any changes in themes or plugins (none in our case) and finally import the modified content

$ composer wp-setup
$ composer wp-import

And that’s actually it. You might want to create a small bash script to run both commands at once, or you might prefer to run them separately as we did above. In this case, we could have just run the wp-import command since we didn’t add any new plugins, but I’ve taken it as a habit to always run them both.

 

Scenario 2. Standard shared hosting with only ftp access

In this scenario, we’ll do pretty much the same things as above, but we’re going to run it from within our staging environment. Make sure you’re inside your vagrant box and navigate to staging:

$ cd /srv/staging/tutorial-simplesite1

Get the latest version of the code:

$ git pull

And then run the commands to apply changes;

$ composer wp-setup
$ composer wp-import

Now, all the database changes are in place on the production server, we need to push over the modified files as well. First, let’s look at what sitecopy want to transfer:

$ sitecopy remote --flatlist
sitestart|remote
sectstart|changed
item|wp-content/config/wpbootstrap.json
item|wp-content/uploads/2015/12/camera-1080-1080x380.jpg
item|wp-content/uploads/2015/12/camera-1080-720x380.jpg
item|wp-content/uploads/2015/12/camera-1080-768x512.jpg
item|wp-content/uploads/2015/12/camera-1080-150x150.jpg
item|wp-content/uploads/2015/12/camera-1080-960x480.jpg
item|wp-content/uploads/2015/12/camera-1080-1024x683.jpg
item|wp-content/uploads/2015/12/camera-1080.jpg
item|wp-content/uploads/2015/12/camera-1080-272x182.jpg
item|wp-content/uploads/2015/12/camera-1080-436x272.jpg
item|wp-content/uploads/2015/12/camera-1080-300x200.jpg
item|wp-content/themes/vantage-child/functions.php
sectend|changed
siteend|changed

Besides the functions.php file we know we changed, we’ll also see a bunch of other files that sitecopy think have changed. This is because how the import process works where several files get touched even if they didn’t in fact change. It’s a fairly small extra overhead, so we’ll get going with the final step:

$ sitecopy remote --update

 

Some more comments on the ftp/sitecopy approach

When writing this tutorial, the update of the above files took a total of 10 seconds. Most of that time was spent on transferering files that hadn’t actually changed. This means that the time between the database has been updated and all the files are in place was 10 seconds. During this time, there’s a fair chance that your site is in an “undefined” state, so we want to do what we can to fix this. Sitecopy offers a way to use checksums rather than the file modification date as detection method. That creates a little bit of overhead before the transfers can start, but it’s often worth it. If you have a lot of media files that is managed using WP Bootstrap, i suggest you add “state checksum” to the .sitecopyrc file:

site remote
    server ftp.yourdomain.com
    remote /public_html
    local /srv/staging/wordpress
    username yourusername
    password "secret_pass"
    symlinks follow
    exclude wp-config.php
    state checksum

Final thoughts

Combining Git with a “content aware” tool like WP Bootstrap is a fairly powerful combination. By defining precisely what parts of WordPress we need to manage, we can leave all the other stuff untouched. Updating the menu structure never conflicts with existing posts, comments or other content on the production site. So developers can take care of code and certain parts of the site, while writers and editors can manage the content that is just pure content.

Ideas? Questions? Feeling angry or happy? Please share your thoughts in the comment section below.

2 Replies to “Tutorial: Managing a WordPress installation in Git using WP Bootstrap”

  1. Very cool project! A couple of questions on the article:

    1) In the .gitignore file you have defined “staging/” folder to be excluded, what is it meant to contain?

    2) What is the idea behind exluding “www/” from the git repository?

    3) Based on “git add bootstrap/ wp-content/” “wp-content/” folder seems to be at the same level as “www/” folder, isn’t it normally located at “www/wp-content/”? Have you used symlink or how did it get placed at that location in the file system and why that was done?

    1. Hi Aki,

      Sorry for not replying sooner. The email notifications was apparently broken so I missed you comment.

      1. The staging/ in .gitignore is a side effect of how the Vagrant is configured. I have a separate URL set up for the Vagrant machine that allows me to run a staging environment in the same virtual machine.

      2. Anything that goes into the www/ folder is “just dependencies” meaning that they are fetched from wordpress.org’s repos at the time of setup. Since WordPress itself and any standard plugins will be exactly the same everytime you fetch them, it doesn’t make any sense to have them in your own repo. Now there are cases when you want a certain version of WP or any of the plugins but wp-bootstrap handles that fairly well (with room for improvements)

      3. Yes that is exactly true. When running the wp-setup, any plugin or theme that is unique for your project (and defined in appsettings.json) will be symlinked into the proper location. By doing it this way you can keep your child themes and project specific plugins in git while having dependencies being fetched from their sources at install time

      Hope these clarifications helped you. Don’t hesitate to ask again if you need a better explanation. Won’t keep you waiting 6 weeks next time 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *