Is Your Website a Hit? Make Sure the Full Page Cache Works

Do you expect your website to get a lot of visitors? No worries, if your site is hosted by Seravo, you are already well prepared. However, if you want to be extra well prepared and to make sure your WordPress site loads as fast as possible, you should check that the HTTP caching works for your site.

The caching done on the HTTP level stores a copy of the entire HTML document your WordPress PHP code produces, and if that copy can be used instead of fetching the HTML document from PHP again, it will mean that the HTML page itself can load in just a few milliseconds. The HTTP level caching is based on common web standards and special cache controlling HTTP headers, so it is not Seravo specific in any way. Also, visitors browsers and any intermediate network proxy server will store the HTML document and help your visitors get the pages loaded faster.

HTTP Headers

Let’s have a look at how HTTP headers look like. Below is an example of what HTTP headers the Swedish WordPress community website emits. The front page of wpsv.se is fetched with a command-line tool called curl, which is incidentally made by also Swedish open source developer Daniel Stenberg and at Seravo our admins use curl on daily basis to test customer sites.

$ curl -IL https://wpsv.se/
HTTP/1.1 200 OK
Date: Wed, 25 Apr 2018 17:47:59 GMT
Content-Type: text/html; charset=UTF-8
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Link: <https://wpsv.se/wp-json/>; rel="https://api.w.org/"
Link: <https://wpsv.se/>; rel=shortlink
X-Proxy-Cache: MISS P: A: N:

There is a whole bunch of HTTP headers emitted and we have omitted a few ones for brevity. On the first line we see that the page returned code “200 OK” which tells everything went fine. The fields Date, Content-Type, Expires, Cache-Control and Link are all standard HTTP headers. The X-Proxy-Cache is a special field emitted only by Seravo’s servers. The fields Expires and Cache-Control tell the browser (and all intermediate HTTP proxies) what the lifetime of the contents is and how it can be cached. In this case they state that the contents has expired in the past, and that caching is forbidden. HTML documents sent with these HTTP headers will not be cached anywhere, ever.

At Seravo the HTTP headers are emitted either from WordPress PHP code (using the PHP function header()) or from the customer specific Nginx configuration at /data/wordpress/nginx/.

Now it turns out this site had define('WP_DEBUG', true); set in wp-config.php, and when the debug mode is on, WordPress also emits cache forbidding headers. Removing that stanza in this case was the solution to get rid of the cache bursting headers. Below are the headers after the change.

$ curl -IL https://wpsv.se/
HTTP/1.1 200 OK
Date: Wed, 25 Apr 2018 18:14:41 GMT
Content-Type: text/html; charset=UTF-8
Link: <https://wpsv.se/wp-json/>; rel="https://api.w.org/"
Link: <https://wpsv.se/>; rel=shortlink
X-Proxy-Cache: HIT P: A: N:

Note also that the value of the header X-Proxy-Cache. Now it says it is a HIT. This means that the response was served from the HTTP proxy and not from the PHP backend. As a developer, your goal is to make sure your visitors can get HIT as often as possible.

The server is not the only party emitting HTTP headers. Also the client side emits headers. In fact, if you have a cookie set in a browser, then the request headers will include a line starting with keyword Cookie followed by colon and a value. If the client HTTP request has a cookie set, then the response from the server is most likely never from the cache, as the cookie is a sign of a logged in user and WordPress does want to serve each visitor unique content, for example a page that renders “Howdy, Otto” in the upper right corner just for one visitor, and not show the same page to everybody. So cookies will burst the cache.

Another client side HTTP request header is Pragma: no-cache. In fact, if you in Chrome of Firefox press Ctrl+F5 to do a deep reload of a page, this is exactly the client header sent to the server. This can be replicated command line with curl with:

$ curl -IL -H Pragma:no-cache https://wpsv.se/
HTTP/1.1 200 OK
Date: Wed, 25 Apr 2018 18:14:54 GMT
Content-Type: text/html; charset=UTF-8
Link: <https://wpsv.se/wp-json/>; rel="https://api.w.org/"
Link: <https://wpsv.se/>; rel=shortlink
X-Proxy-Cache: BYPASS P:no-cache A: N:

Note that here we see again a new value in X-Proxy-Cache. All possible values are HIT, MISS, EXPIRED, BYPASS and STALE. If you have MISS, it means the page does not cache on HTTP level at all. The other words tell about the state of the request cache. As a developer, you want to see that most of the time you get HIT and only occasionally something else.

Curl includes a huge amount of features and it is a great toolbox for a web developer to debug what happens on the HTTP level when the web browser and web server are interacting. One favourite option of ours is to print our the total time it took for the response to arrive.

$ curl -I -s -w "%{time_total}\n" https://wpsv.se/
HTTP/1.1 200 OK
Date: Wed, 25 Apr 2018 18:20:39 GMT
Content-Type: text/html; charset=UTF-8
Link: <https://wpsv.se/wp-json/>; rel="https://api.w.org/"
Link: <https://wpsv.se/>; rel=shortlink
X-Proxy-Cache: HIT P: A: N:
0,080179

$ curl -I -s -w "%{time_total}\n" -H Pragma:no-cache https://wpsv.se/
HTTP/1.1 200 OK
Date: Wed, 25 Apr 2018 18:20:53 GMT
Content-Type: text/html; charset=UTF-8
Link: <https://wpsv.se/wp-json/>; rel="https://api.w.org/"
Link: <https://wpsv.se/>; rel=shortlink
X-Proxy-Cache: HIT P: A: N:
1,072267

This means that the request completed in 80 milliseconds when it came from the proxy cache and when served all the way from PHP it took a second. That is actually quite a lot, and next week we will blog about tools like XDebug that can be used to analyze what keeps so long for PHP that producing the HTML page took a second.

Note that if these tests are run from a laptop, then the figure will include the network lag of your local WLAN network and all hops between the laptop and the server. To always get comparable results, it is recommended to run curl from the server in a SSH terminal connection.

Since this is a common task, we have at Seravo created two handy commands for our customers to test how fast WordPress serves pages: wp-speed-test and wp-load-test. See example below.

wpsv@wpsv_1e5ea3:/data/wordpress/htdocs$ wp-speed-test
Testing speed URL https://wpsv.se...

For an explanation of the different times, please see docs at https://curl.haxx.se/docs/manpage.html

URL TOTAL NAMELOOKUP CONNECT APPCONNECT PRETRANSFER STARTTRANSFER = AVG
https://wpsv.se 1.174 0.125 0.126 0.132 0.132 1.174 1.174
https://wpsv.se 0.478 0.000 0.000 0.000 0.000 0.478 0.826
https://wpsv.se 1.017 0.000 0.000 0.000 0.000 1.017 0.890
...
https://wpsv.se 0.579 0.000 0.000 0.000 0.000 0.579 0.864
https://wpsv.se 1.163 0.000 0.000 0.000 0.000 1.163 0.880
https://wpsv.se 0.594 0.000 0.000 0.000 0.000 0.594 0.865

Test completed. If the values seems too high, please profile your PHP code to find potential bottle necks.
Note that this test tells how fast your site is in the sense of how long it takes for PHP to generate the 
HTML output. To test how much load the site can handle, run wp-load-test.

wpsv@wpsv_1e5ea3:/data/wordpress/htdocs$ wp-load-test
Testing capacity of URL https://wpsv.se...

URL COUNT ELAPSED TIME RESPONSE TIME
https://wpsv.se 1 1 1.130
https://wpsv.se 2 2 1.040
https://wpsv.se 3 3 0.995
...
https://wpsv.se 38 29 0.571
https://wpsv.se 39 29 0.568
https://wpsv.se 40 30 0.567

Test ended after 30 seconds and 40 requests with an average of 1.3 requests per second.
This test only uses a single PHP worker and it bypasses the edge cache. The actual site 
will be capable of handling much more traffic. If the response time of a single PHP 
request is much above 0.500 seconds, please try to optimize the PHP code and run wp-speed-test. 
If the response time is much below 0.100 seconds, then this test is likely to trigger the 
flood protection and server will yield 429 responses.

You can also add the parameter –cache to see how fast the page loads if fetched from the outside where the HTTP proxy is effective:

wpsv@wpsv_1e5ea3:~$ wp-speed-test --cache
Testing speed URL https://wpsv.se...

Warning: invoked with --cache and thus measuring cached results. This does not measure actual PHP speed.

For an explanation of the different times, please see docs at https://curl.haxx.se/docs/manpage.html

URL TOTAL NAMELOOKUP CONNECT APPCONNECT PRETRANSFER STARTTRANSFER = AVG
https://wpsv.se 0.041 0.028 0.030 0.036 0.036 0.041 0.041
https://wpsv.se 0.006 0.000 0.000 0.000 0.000 0.005 0.014
https://wpsv.se 0.005 0.000 0.000 0.000 0.000 0.005 0.008
https://wpsv.se 0.005 0.000 0.000 0.000 0.000 0.005 0.007
https://wpsv.se 0.006 0.000 0.000 0.000 0.000 0.006 0.007
https://wpsv.se 0.005 0.000 0.000 0.000 0.000 0.005 0.007

Both of these commands also accept an URL as a parameter, so you can test a particular page of your site:

wpsv@wpsv_1e5ea3:~$ wp-speed-test https://wpsv.se/?wpsvorg_feed=1
Testing speed URL https://wpsv.se/?wpsvorg_feed=1...

For an explanation of the different times, please see docs at https://curl.haxx.se/docs/manpage.html

URL TOTAL NAMELOOKUP CONNECT APPCONNECT PRETRANSFER STARTTRANSFER = AVG
https://wpsv.se/?wpsvorg_feed=1 0.088 0.028 0.029 0.035 0.036 0.088 0.088
https://wpsv.se/?wpsvorg_feed=1 0.038 0.000 0.000 0.000 0.000 0.037 0.063
https://wpsv.se/?wpsvorg_feed=1 0.037 0.000 0.000 0.000 0.000 0.037 0.054
https://wpsv.se/?wpsvorg_feed=1 0.037 0.000 0.000 0.000 0.000 0.037 0.040
https://wpsv.se/?wpsvorg_feed=1 0.037 0.000 0.000 0.000 0.000 0.037 0.040
https://wpsv.se/?wpsvorg_feed=1 0.037 0.000 0.000 0.000 0.000 0.037 0.040

Now, go test your own site and make sure it sends out the HTTP headers that it’s supposed to. Happy optimizing!