The Next-Generation WordPress Testing System

Part of our premium WordPress hosting and upkeep service is testing and updating our customers’ sites for errors and otherwise reduced functionality. Previously the testing has been done with the help of our own wp-test command that has been available to our customers as well via SSH. Now we’re excited to announce that in the coming weeks a next-generation testing system will be rolled-out according to the following schedule:

• Now: Every customer has access to the trial version of the new testing system by running the command wp-test-nginside their container.
• Starting from 22nd of January 2019: All new WordPress sites delivered by Seravo will feature the new testing system, as the scripts/run-tests file and the tests/rspec folder utilized by the old testing system have been removed. Sites delivered before this date will also use the new testing system if the file scripts/run-tests has been deleted.
• 15th of February 2019.: Significant improvements to our service will be announced in February. These improvements include, but are not limited to, site deployment, upkeep, updates and what is included in our service.
• 31st of March 2019: The old testing system is phased out entirely. By this date we aim to notify all customers appropriately via customer service and other channels to ensure that no custom rspec tests are in use. These custom tests must be converted to the Codeception format used by our next-generation testing system.

The old wp-test integration testing system we built has been in use since 2015 and it is based on the PhantomJS browser and Ruby’s Rspec tests. During the last year the testing system has come to the end of its road, as the PhantomJS webkit has not been included in the Google Chrome browser anymore. This resulted in the tests (and especially the JavaScript functionality) differing from actual user experience by a wider and wider margin, making it necessary for us to rebuild our testing system.

Our brand new next generation integration testing system is based on headless Chrome technology, meaning that a true Chrome browser is run on the server. Naturally, this is done without any actual graphical output, as our servers are not connected to monitors. This new technology allows for the testing system to gain a view of the site that is nearly identical to what regular user would see when visiting the WordPress site on their Chrome browser. We’re thankful to all customers who have been trying out the new wp-test-ng tool ever since we introduced it in our Seravo Developer Newsletter.

While we were at it, we made other rather significant changes to our system and how it operates. For one, we removed the tests folder from under the /data/wordpress/ path. The folder is from now on only required for custom tests built for the specific site. This means that the version control used by our customers is no longer interfered with in any way by our centralized testing infrastructure, making it easier than ever for developers to work on the sites in our WordPress environment. The only test code within the customer’s git project from now on will be site specific testing code developed by the customer themselves or by Seravo as part of our expert services.

Another important change is that tests are now written without the use of the old system’s Ruby language or the Rspec and Capybara libraries. Based on our experience, our customers rarely utilized these, and so we made the switch to using PHP and the Codeception library in our new testing system. It includes a few changes made by us here at Seravo, to make it easier and more efficient to use with WordPress sites. We believe that as a whole it is a far more enticing solution for developers to start using. Codeception is an open source library that is used for website and code testing more frequently than Capybara. This wider utilization means that more documentation, guides and support is available online, which is something developers will appreciate.

The next-generation testing system has been in test use for a long time now. The extensive testing period has been necessary, as, in theory, the tests could return a unique response for each website due to a difference in their codebase. For the same reason it will not be released directly to everyone, but instead it will be rolled-out according to the schedule above.

For those developers who wish to make use of the new testing system before the scheduled roll-out date, fear not, as you can use the command rm /data/wordpress/scripts/run-tests to turn on the new testing system on sites that don’t utilize any custom-built integration tests. An example of removing custom tests and the old testing system inside a version controlled project can be seen in the public Linux-tuki.fi GitHub repository.

And for those of our customers who are not developers and don’t wish to meddle with their testing system, don’t worry, we will roll-out this next-generation testing system to everybody without any work required on our customers’ part.

Testing WordPress sites is a key part of an updating process, and this new testing system will greatly improve the performance of our update system. Other changes to our update system are also coming in the near future. Subscribe to our newsletter and blog updates to stay in the loop of our latest news and updates!