How to test a new advertising channel


Can you remember way back when Google first brought out Adwords? Early movers were generating traffic for a fraction of today’s costs.

Facebook ads have seen a similar rise in popularity in recent times. So much so that marketing guru Gary Vaynerchuk  in this public address commented:

The most under-priced ad product right now is Facebook Advertising. If you had to sell to feed your children, you would put your money there.

But that won’t always be the case and we’ve seen spend on Facebook ads increase while CPM’s have nearly tripled since the start of 2017.

Mark Ritson recently condemned this strategy in Marketing week commenting “rather than investing heavily in a few ‘superior’ channels, the prudent approach is to invest across as many channels as your budget will allow.” No matter how great of a marketer you are. Or how great the product you have.


Over time your best channels will start to become less effective.

 

Why?

Every good channel becomes used, saturated by competition and returns diminish. Or platforms change the way they work.

Ever heard of Mahalo.com? It was a startup founded by Jason Calacanis. They were generating $10 million in revenue directly from google search results. Well, they were until google updated their algorithm to panda.

Overnight their traffic dropped. Jason had to lay off over 80 staff and shut down the business.

The best time time to find new channels is before you need them.

 

So what can you do?

You can make a start by find emerging channels. If this is daunting, or information overload. You can always look at channels which have seen a resurgence like outdoor advertising. You might not have used or tested it, but in an age of adblock it is driving real returns for brands.

This article will go through practical tips on how you can get the best results when testing a new advertising channel.



Set goals

If you’re going to run any kind of test then you need to have an understanding of what you’re trying to achieve and how you will measure it.

 

To do this, before we run any campaign we always ask and agree on answers to these questions:


  • What are you trying to achieve with this campaign?
  • How will you measure it?
  • What will you consider a success?
  • What will you consider a failure?

 

If you are running a test you need to have a clear and objective idea of:


  1. Yes, this works and we can scale it
  2. This was okay, we’ll try again and this time make x,y,z changes
  3. This isn’t the right channel for now, but maybe in the future
  4. This probably isn’t the right channel

 

Take the following example:

You run an outdoor advertising campaign in two small cities with two different creatives each including a unique web URL.

 

You have the following results:

Campaign A - web traffic increase by 1,000 views, but positive social media mentions skyrocket

Campaign B - web traffic increase by 5,000 views.

 

Which result are you happier with?

If you read the first part, then you you know where I’m going with this...it depends.

 

Positive brand mentions across social media may be important to your goals and you can easily attribute a rough value of mentions to arrive at an ROI, so you may lean towards A. IIf you solely want to drive traffic to your website, where you can use tag manager and analytics to assess ROI then you likely prefer B as an outcome.

We think that it depends on what the objective of your campaign is and what you set out before starting.

 

A few ways you can measure the impact of an outdoor advertising campaign:


  • Organic traffic increase
  • Direct traffic increase
  • Impressions on search engines
  • Social mentions

 

Run a control campaign 

To do a comparative analysis on the results of new channels, you can run a control campaign to compare against. To do this for outdoor you can’t track track website visits as a direct result campaign, but you there are ways to attribute results.

 

Two ways you can do this:


  1. Run the new medium in isolation in a location where you aren’t running other media campaigns.
  2. Or run the medium but keep activity through other channels consistent.

 

An example 

eMoov used Car Quids with bus and radio in Nottingham, while only using Car Quids in Leeds.

They found that brand awareness increased 2x in the city without car advertising. Whereas in the city with car Quids, brand awareness increased a staggering 3x from an 11% base.

Structuring campaigns in this way enables you do to a comparative analysis of campaign outcomes.

eMoov used brand awareness surveys to measure this.

 

Here are a few other tools and methods you can use to track outcomes:


  • Google Analytics for web traffic uplift
  • Search Console to measure organic impressions and clicks
  • Sprout Social for social listening
  • Google Ads to message uplift in PPC performance
  • Internal tools to measure sales and revenue

 

Make it a fair test

The key to a good test is to make sure it is fair as you can. You don’t want to give one medium an additional advantage over another skewing the results.

 

How to conduct a fair test:

  • Control variables

Ensure that you are consistent in factors you can control. We mentioned running in isolation but you should also keep your marketing mix consistent while in testing - If you normally run Google Ads or Facebook ads then make sure this remains.

Budget is a big one here. Tests should be of a comparable size to measure.

This prevents your analysis skewing towards whatever you spent the most on.



  • Avoid bias

It may be your idea and you really want to impress your boss, but don’t let this impact how you look at the results.

Similarly if this test wasn’t chosen by you, don’t let this spoil your opinion if results are delivered.

You might have your doubts. But let the results speak for themselves.


 

  • Distinguish chance from real difference

To reach a statistically significant outcome, you need your sample size to be big enough.

A small campaign may not work out purely because you haven’t scaled it out enough to achieve a statistically significant result. Facebook now requires you to make 50 conversions before optimising your ad placement. Outdoor advertising is exactly the same, you need to reach a large enough audience to assess if this works for your brand.

Similarly seasonality of your industry and timing of the test can impact outcome.



Allocating budget

As a progressive marketer you should already be splitting your budget into different categories to make sure that you stay ahead of your competition.

 

A leading FMCG brand allocates their budget in the following way:



  • 70% of spend on proven core marketing and brand activities
  • 20% on new and emerging channels that are somewhat proven
  • 10% to test new channels.


Check out our post on trusting new forms of media.

 

If your marketing budget is £250,000 this year, this would mean allocating £25,000 towards new channels.

How you divide this across different channels depends on your risk tolerance. This budget allocated to London using Car Quids could create:

Circa 14 millions impressions and reach 27% of the population of London.



Analysis

Finally you’ll want to analyze the results to see if the test has been a success.

 

Some questions you may want to ask:


  • Did it achieve the results I had anticipated? Why?
  • Were there any other factors which contributed to the success or failure?
  • What can I learn from this test?

 

The importance of this is to learn what is working, what isn’t and why. The channel could be a viable option but needs to be scaled out to work effectively. The only way to find out is to repeat, measure and analyze continuously.