How We Test WordPress Updates

Maybe the most innovative feature on Seravo.com, certainly one we’re most proud of, are our state-of-art tested updates. This article will delve into more details on how we make automated tested updates work.

WordPress Automatic Background Updates

WordPress is open source software that’s actively developed by the community. As WordPress users we’re extremely lucky to get bug fixes and new features on a regular schedule. It’s especially important we get security patches due to WordPress’ massive popularity and unfortunately the interest of black hat hackers that comes with it.

Every once in a while, a 0-day vulnerability shows up in the open and thanks to the quick work of WordPress maintainers, we usually expect a security update within a few hours of it being reported.

For the reasons above, WordPress has offered automatic background updates since version 3.7, which are enabled by default for the WordPress core minor version updates. Thanks to this feature, WordPress security in general has improved quite a ways since 2009-2012, while it still had a bit of a bad reputation amongst web makers.

Plugins and themes are however still left unupdated by default, but auto-updates can be easily enabled with this piece of code:

define('WP_AUTO_UPDATE_CORE', true);
add_filter( 'auto_update_plugin', '__return_true' );
add_filter( 'auto_update_theme', '__return_true' );

Problems with Automatic Updates?

Most, if not all people who’ve worked with WordPress, have had at least one bad experience with manually updating your WordPress site. It’s not too uncommon for some feature on your site to completely stop functioning after updating a plugin. The worst case scenario even being your entire site gets the dreaded white screen of death after an unsuccesful update run.

This is especially apparent with automatic updates, where the failed update can cause issues without anyone even noticing, sometimes in the middle of the night. Nobody likes waking up to an angry call from a client who discovered their site is down. Especially if nobody even touched it.

To be fair, it’s extremely rare for an automatic WordPress core update to cause any problems. Instead, it’s usually an incompatibility with a 3rd party plugin or a theme, which causes the issue. This is the main reason people are often hesitant to update plugins unless it’s absolutely necessary and also why WordPress has disabled automatic plugin and theme updates by default.

Our Solution

As part of our service, we take care of all updates, including WordPress plugin and theme updates on your site. We also make sure your public-facing site is always in working condition, which means updates simply cannot cause any visible issues to users.

How then do we make sure updates never break stuff on your site? – It’s simple. We test all updates in a sandbox before we update your live site.

For this purpose, we’ve developed our own automated update procedure, which always makes a copy of the public site into a sandbox, which can then be updated and tested without fear of affecting the live site.

For maximum reliability, sites are also always backed up before running any updates in case our tests miss something and we need to revert to a previous state.

Tests are automated using a headless browser calledn PhantomJS, which runs a set of pre-defined acceptance tests like a user would, using a regular browser. These tests make sure things like Javascript works properly, and links and buttons do what’s expected. The default tests include:

• Does the front page load normally?
• Are stylesheets present?
• Are there any Javascript errors?
• Can you log in to the WordPress admin and see the admin dashboard?

The PhantomJS browser also takes screenshots of different parts of the site, which are then compared to the state before the update to determine whether something has significantly changed after the update. We refer to this as visual regression testing.

As Seravo.com customers, you can also define your own acceptance tests to make sure critical functionality is preserved.

The tests reside under the /data/wordpress/tests/ -directory on Seravo.com instances.

The acceptance tests are written using the easy to read Ruby Rspec library. For more information on writing your own acceptance tests, please refer to our developer documentation.

We also offer our expert services for writing tests for your site’s needs.

Once the tests have determined everything works fine after updates, we can then safely replicate the update procedure on the live site.

In cases where our update procedure fails, a Seravo.com administrator gets notified and looks into the issue in detail.

For existing Seravo.com customers, all phases of the update procedure are logged under /data/log/update.log.

The end result is always a well maintained and functioning WordPress site.

In Conclusion

Have you ever run into issues after updating your site? Would you like to know for sure your site doesn’t break after updating? We’ll gladly help you.

Contact us, and let us take care of updating your site.