Online Marketing Today: Your guide to Core Web Vitals tactics using Cloudflare and WebpageTest

One of the best ways to be able to perform tactical edits to webpage code and improve Core Web Vitals scores is to set up a comparison, like A/B testing. As a developer you can run Lighthouse in your local development environment and test as you make changes. It is still useful to tests production code, which you have to do anyway when you’re not the developer.

We used a nifty demonstration stack in our Core Web Vitals Site Clinic just last December in the SMX Build: SEO for Developers. We will continue to use it in the upcoming SMX Master Class training, and from now onward with our SEO for Developers series of posts.

Here is how to test Core Web Vitals scores and code changes using Cloudflare Workers as a reverse proxy and WebpageTest. All these services are free and we’re certainly not the first to use them. Patrick Meenhan, WebpageTest developer, thought it all up. We’ll outline how to get started without all the heavy lifting.

Cloudflare and WebpageTest

The Cloudflare Workers app gives us our reverse proxy testbed and transforming code using the proxy environment. While a playground at cloudflareworkers.com exists, fragmented URLs at that address prevent us from testing without a Cloudflare account (the free one works). You do not need a WebpageTest account.

Once you’ve got a Cloudflare account, navigate to Workers and click to create a new worker with the Create a Service button. This will populate a sample JavaScript worker with a UI editor you can access with the Quick Edit button. Each worker you create gets a unique URL. You can rename it at any time. We set one up at: https://sel.deckart.workers.dev for this purpose.

If you navigate, notice the “x-host header” requirement. The requirement limits requests to testing. We use a browser extension to modify requests, add the x-host header needed to supply the script with the host we want to test. Let’s modify requests in your browser so we can view the page, view-source, and run DevTools.

Navigate Tests with ModHeader

We’ll be using ModHeader which supports major browsers. In our case, we installed the Chrome extension and added two custom headers as shown below. The x-host header provides the host we want to proxy for our test, and the x-bypass-transform header switches transformation on and off so we can test the difference.

With x-bypass-transform set to a “true” value, transformation is off. We can then view-source to look for tactics to test. With the x-host header in place as depicted above, you can navigate to the worker URL and you should be able to see Search Engine Land’s homepage, view its source code, and open DevTools.

Setting up your own Worker

The job for the worker is to obtain the header values and process requests accordingly. We’ll summarize the script below, leaving out a few details that are unimportant at this time.

Proxy URLs using the x-host header value.
Transform requests that have an accept header value with “text/html” in the string.
Bypass transformation when the x-bypass-transform header value is true.
Transform the HTML when x-bypass-transform header is missing or the value is false.

If you’ve ever written a conditional control-flow block of code, then these tasks should be pretty easy to imagine writing yourself in JavaScript. The more interesting question then becomes: How are we going to transform HTML? That’s where the magic of HTMLRewriter() is. Copy the basic worker Gist and replace your default worker with the raw source.

Setting up WepageTest for comparisons

The basic worker script performs only one transformation. Preloading the LCP image we bumped its request a couple of places in the queue. That took mobile LCP load time from over 5 seconds to less than 4, about 500ms improvement. To reproduce this, our script needs to keep up with changes. The point is to get you ready for tests with SEO for Developers and your own work.

Now that we can A/B changes in the browser, how did we use WebpageTest to get the difference between scores? We get LCP element references as part of the detailed waterfall chart, which is our most helpful map for viewing the effects of our tactical changes. We look at the request order and plan to change the order in which resources load for improving speed.

Our starting point is going to be WebpageTest with the URL (and not the reverse proxy). This is because Cloudflare Worker URLs have different conditions than the origin host provider. For example, the host might be operating using the older HTTP/1.1 protocol, whereas Cloudflare upgrades to HTTP/2 as part of its service. This first WepageTest report should be used for developing tactics.

Next, we’ll run a scripted test at WebpageTest to supply the custom headers needed for A/B testing our changes using the reverse proxy URL. To simulate Core Web Vitals conditions, WebpageTest has an easy-to-find button. It’s fine to use this for the initial test. You will need to edit settings on subsequent tests and the Core Web Vitals button page doesn’t have the UI for that.

Instead, use the default homepage test and put the origin URL in the test input field. Change the browser dropdown menu to select “Motorola G (gen 4).” Open the Advanced Settings dialog and change the Connection dropdown menu to select “4G (9 Mbps, 170ms RTT).” Click over to the “Advanced” tab and look for the Custom Headers field where we will add the following headers.

We’re not going to bypass transformations at this time, so we set the value false. Continuing onward, we have to script the test so that WebpageTest will disregard that we put https://ift.tt/1BDlNnc in the test field, and instead will fetch from our reverse proxy which is required so that the test can correctly swap the host in the base HTML document. Switch to the Script tab and add the following.

You are going to want to replace all the URL strings to match your own tests once you’ve set yourself up with Cloudflare. Running the scripted test you will be able to get the blow-by-blow request queue with timings and more than enough detail to develop tactics for improving Core Web Vitals with most any webpage. A majority of webpages are easily proxied, but your mileage may vary.

Why we care

Core Web Vitals is a Google User Experience ranking factor. It may not be the strongest boost to your ranking next to, say, optimizing page titles. Google is on record saying that it will improve your rankings depending on your scores. They have stated that you don’t need all Vitals to score Good in order to enjoy the benefit. Your optimal score is 90+ and once you’ve reached the threshold, a higher 90s score isn’t better than 90 itself.