Is a High Abandon Rate Hitting Your Online Marketing Budget?

Could you be wasting 40% of your marketing budget on abandoned site visits? Calculate your abandon rate today

“Half the money I spend on advertising is wasted; the trouble is I don't know which half.”

 

With the huge diversity of digital marketing tools and data collection techniques we have at our disposal today, you’d think this historic quote from 19th century retailer,  John Wanamaker, would be just that: a thing of the past.

 

On the contrary, this phenomenon is still very much occurring, as my experience working for  international corporations has taught me. Many of these companies had a common, but serious, problem: a high abandon rate. An abandon rate is the percentage of visits to your website that are abandoned before the page is fully loaded.

Why should I bother calculating my abandon rate?

  1. A 40% abandon rate means that 40% of what you’ve spent on marketing potentially goes to waste, simply because your page loads too slowly. Calculating your abandon rate can help you identify wasteful investments and alert you to money being channeled into resources without return.
  2. High abandon rates will be punished by Google. This will have a negative impact on your search engine rankings as well as your Google Adwords Quality Score. This results in a cost per acquisition increase, which in a highly competitive market could be fatal for your business.
  3. It can help you evaluate the trustworthiness of your analytics data once you know that it’s only based on 60% of your page views. For example, mobile users will be more likely to abandon a visit because the chances are higher that a mobile connection is slow. This means you actually have much more mobile users than your analytics tool will tell you.

 

If you’re curious about the scale of your abandon rate, this article can provide you with more information. It will include a step by step guide to calculating the abandon rate for your site.

How big is your data gap?

Modern analytics and marketing tools utilise a small Javascript for collecting data. In order to deliver the best possible user experience, these scripts are loaded asynchronously, which means they start carrying out their work after the page is loaded.

 

Having tested brand websites with a smartphone on a 3G connection, most needed around 15 seconds to load. Web performance studies demonstrate that 40% of mobile users abandon a page visit after 3 seconds if the page isn’t yet loaded. We can assume that a large proportion of this traffic comes from a mobile device these days, and therefore, this can mean a huge gap between different data sources.

 

If users decide to leave your page before it’s fully loaded, this means you’ll find a pageview in your server log files but not in your analytics data. As server log files are often not accessible to marketing teams, the problem can often go unnoticed. Before launching your next major online campaign, you need to make sure you can see the full picture. Otherwise, you are likely to waste a substantial part of your digital marketing budget.

 

How can I calculate my abandon rate?

The abandon rate exposes the gap between your server log files and your analytics data. The simplest metric for this is the pageview (some tools may call it page impressions or PI). A pageview means a request to load a single HTML file.

 

Using pageviews, it should be straightforward to generate a report for a specific time range in your analytics tool in order to produce a number.

 

Server logs, on the other hand, are very different. These provide you with a large text file with thousands of lines. Each line represents a request which looks like the following:

 

148.78.133.209 - - [21/Oct/2017:09:14:07 +0000] "GET
/yourpage.com/intl/de/index.html HTTP/1.1" 200 35495 "-" "Mozilla/5.0
 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/60.0.3112.101 Safari/537.36" "-"

Depending on how many assets (images, javascript files, etc) you are loading on a page, a pageview could be represented by many lines. In order to be able to compare these analytics with your server log data, you first have some work to do.

1. Obtain your server log files

Ask your IT or infrastructure team for the complete server logs for the time range you want to investigate. Most large companies use a content delivery network (CDN) to be able to deliver their pages rapidly around the globe. This means it can take some time to aggregate the data from all servers.

2. Make the data comparable

Adapt your time range

In order to compare data within the same time range, you’ll need to know the time zone settings of your web server(s) as well as your analytics tool. If the timezones are different, you will need to adapt the time range in your analytics tool.

Know how your analytics tool works

Make sure your analytics tool is working as expected:

  • Is it integrated on all pages?
  • Are there any other tools (for example security or network tools) which could affect server log files or analytics data?

 

Understand how your analytics tool is filtering data:

  • Are search engine bots being filtered automatically?
  • Do you filter the traffic from your own company network?
  • Are there any other filters active?

Filter your server logs

You’ll need to apply the following filters to your server log files:

  1. Filter all requests which don’t represent a page view, such as Javascript and CSS files, images, Ajax calls, etc.
  2. Apply all the filters your analytics tool is using to filter data.

 

Filtering server logs is no small task and, once again, this will require some help from your IT department. However, there are plenty of tools available for this, such as grep, a small unix tool, or a more sophisticated log file analyzer.

3. Eliminate the noise

In your server logs you’ll most likely find pageviews that aren’t tracked by your analytics tool. This could be for a number of different reasons. Firstly, to calculate an accurate abandon rate, you’ll have to count all the affected pageviews and subtract them from your server log pageviews.

In order to work out these pageviews, you’ll need to write some Javascript which tests whether your analytics tool is working as expected. If your analytics tool is blocked, generate an entry in your server log files. This can be done by sending an Ajax request or loading a transparent image.

 

Choose an explanatory name for the files, such as:

  • /subtract/doNotTrack.jpg
  • /subtract/adobeAnalyticsBlocked.jpg
  • /subtract/noJS.jpg

This will make it easier to filter these requests in your log files, providing you with further insights that you can’t obtain from your marketing tools. Note that it’s important that these requests aren’t cached in the browser because this would prevent you from finding the request in your server log files.

Do not track headers (DNT)

Most modern browsers offer a “do not track” privacy setting. Once activated, the browser notifies the server that the user doesn’t want to be tracked. Technically, this is done by adding the string DNT:1 to the request header. However, since there are no legal requirements involved, the user’s request for privacy is often ignored by the majority of marketers and marketing tools, though some tools can be configured to respect this DNT header, or to ignore it completely.

Therefore, if you decide to respect your users privacy according to their DNT headers, you should count the affected pageviews. On the other hand, if your analytics tool ignores DNT headers, no changes to the process are necessary.

Tracking blockers

Since many users don’t trust marketers to respect their privacy settings, more and more users have started actively blocking tracking scripts using browser extensions such as Ghostery. To quantify the affected pageviews, you’ll need to determine whether your analytics tool is blocked or not. The Javascript for this part can be a little more complex and will be dependent on the specific tool you are using. If your tool is blocked, you can use the method described above in order to generate a server log request.

No Javascript support

Without Javascript support, modern tracking tools will not work at all. This number is estimated to be very minimal (~ 0.2%), since most pages nowadays won’t work without Javascript. However, to make your calculations watertight, you could also quantify these pageviews. The easiest way to track this is to load a small transparent image inside a <noscript> Tag.

4. Calculate the abandon rate

After you’ve made sure your data is fully comparable, calculating the abandon rate is very straightforward:

5. Automate

It’s recommended to automate the calculation since it can take some work to carry out the calculation manually. This can usually be set up within 2-3 days.

6. Share

Calculating this value on a daily basis and adding it to your KPI dashboard helps your team to find out whether there are problems that your analytic tools aren’t aware of.

When do I need to investigate?

Even after working to make your data sources comparable, your abandon rate still won’t be 100% accurate. The reason for this could be bots using fake user agents or caching effects that aren’t possible to filter. Therefore, think of the abandon rate as a kind of warning signal.

 

This means you might start investigating the causes if your abandon rate:

  • is larger than 10%

  • increases after a deployment

 

Both of these are strong indicators for a web performance problem, which could damage your business in multiple ways, for instance:

 

What’s the solution?

Unfortunately, web performance can be influenced by a number of factors. To name just a few, network, latency, file sizes, number of requests, device, and active background processes can all have an effect on your web performance.

This means that the solutions will be different for every project. A good starting point is to test your site with one of these free tools which allow you to check your page for common problems and provide you with some insights on how to fix them. These can often identify some ‘low hanging fruits’ which can be rectified to speed up your site.

 

If you want to deliver high quality user experiences, web performance can’t be an afterthought. It needs to be a central factor in all the decisions you make from the very start of your project. It will influence your technical architecture, design, code, and content.

In upcoming articles, I’ll delve further into web performance related topics such as:

  • The business impact of poor web performance

  • How to create a fast loading site

  • How to accurately measure web performance

 

If you don’t want to miss out on these articles, follow us on twitter: @NetcentricHQ.

Need some help?

Could Netcentric help you to save a big proportion of your marketing budget? Get in touch with the Netcentric marketing technology experts, who will be happy to support you.

 

Notes

 

[1] http://bit.ly/1Dj8Koy

[2] https://www.quantable.com/analytics/how-many-users-block-google-analytics/

[3] http://bit.ly/2c3yoZK

[4] http://bit.ly/1RttJgr

[5] http://bit.ly/1e79j5W

[6] http://www.webpagetest.org/easy

[7] https://developers.google.com/speed/pagespeed/

[8] https://developers.google.com/web/tools/lighthouse/

 

Fabian Krumbholz

Requirements Engineer

Published on

Dec 5, 2017

Want to know more?

Tags

Developer Circle

Continue the discussion!