We've been doing a lot of attribution modelling at MediaCom North in the past twelve months. For the uninitiated, that means modelling the paths that people take to arriving at your website, via adverts that you're running on different websites.
Imagine somebody sees one of your banner ads, then searches for you on Google and clicks on a paid search ad, then arrives on your website and buys something. Which of the two adverts that they saw, do you credit with that sale?
Originally, I came at this problem from scratch - as an econometrician, rather than a web specialist - and have arrived at a different answer to a lot of the solutions that you'll see reported on industry sites.
The classic approach and the one taken in DoubleClick's (DFA's) new attribution tool, is to say "well we have one sale and a lot of adverts that might have driven it. What proportion of that one sale do we credit to each ad?"
By doing this you end up with a sale that gets split into pieces - maybe 50% to the paid search ad and 50% to the display ad. Maybe a larger proportion to the paid search ad, depending on your model. You can choose from six different ways of splitting up your online sales, within Google's DFA tool.
This method has one massive advantage. When you finish and add up all of your pieces of sales that have been allocated to different ads, you'll get back to the same total as you see on your final sales report.
And it has one massive disadvantage. It's wrong.
For a start and before we get to why it's wrong, giving people six options is downright unhelpful. There is a right answer to the question of attribution and letting people pick their own method is an analyst's equivalent of throwing up their hands and saying "I dunno, could be anything. Pick an answer you like."
Now why is the approach wrong?
To be fair, this isn't an online only problem. In most marketing analysis, we like to pretend that one sale comes from one marketing channel, because it makes life easier. If you've ever seen a nice neat list of return on investment numbers for different marketing campaigns, then this is effectively what you're looking at. Add up the individual contributions of each marketing campaign and you get total sales from marketing. Simple.
Marketing campaigns in the real world don't work like this, even though we like to pretend that they do. In the real world, people see lots of our ads and it needs lots of ads to persuade somebody to respond. Attribution analysis is specifically trying to solve the question of how (online) marketing campaigns work together, rather than individually, but by splitting up a single sale, we're not answering that question.
Take our example again of the customer who saw a display ad, then searched on Google, then bought something. Assume for the moment that they only decided to search for us, because they'd already seen the display ad. We must believe that sort of thing happens, or we wouldn't bother with attribution analysis in the first place.
What happens if you take away just the display ad? That person doesn't ever see the display ad, so they don't do a search and they don't buy our product. We lose one sale.
And if you take away the search ad? The person sees the display ad, searches on Google and doesn't find us. We lose one sale.
1+1 = 2
No we don't lose two sales in total. That would be ridiculous...
This is why, in real life, you can't add up the individual return on investment to each marketing channel. You'll get to a number that's bigger than your total sales.
But it's still the right approach.
When you use attribution to ask "what does this display advert add to my sales?", you're asking the wrong question. The right question - the one we just worked through above, is:
"What would happen to my sales if I took this display advert away?"
You'd lose a sale. Not half a sale, or 25% of a sale. A whole one.
And at this point you can see that this isn't just an academic argument. Your return on investment numbers to individual ads, using the dividing-up-sales method in Google's DFA tool, are too low. You will potentially remove campaigns that are actually working. This is important.
The only way to do attribution analysis is to ask how many sales you would have lost if an advert wasn't running. This is the question that was asked and is the whole point of attribution analysis. To do it any other way, is to pretend that you're measuring how different adverts work together, while actually measuring them individually.
In technical language, your attribution model must be multiplicative, not linear. You're going to need regression analysis on cookie-level data to build it and you're only going to get one answer, not six.
Still with me?
Next step...
Yes, it makes communicating results to clients harder.
It also makes it harder to forecast the impacts of changes to your campaigns. If you don't run search ads, your display ROI will drop. If you don't run display ads, your search ROI will drop. This is the essence of attribution modelling.
You need a forecasting tool, that can take what you've discovered from attribution modelling and project changes in overall marketing ROI, from a change to one campaign. This is what we've been building and what we're continuing to develop. It's been difficult, but very rewarding, and is now in use on live campaigns - it works.
I'm sure Google knows all this, but has chosen not to implement it, because it's more difficult to build and quite a lot more difficult for end users to understand. What you need to understand though, is that any attribution method which splits up a single sale, will encourage you to low-ball your return on investment numbers. And we can all agree, that's definitely a bad thing.
Imagine somebody sees one of your banner ads, then searches for you on Google and clicks on a paid search ad, then arrives on your website and buys something. Which of the two adverts that they saw, do you credit with that sale?
Originally, I came at this problem from scratch - as an econometrician, rather than a web specialist - and have arrived at a different answer to a lot of the solutions that you'll see reported on industry sites.
The classic approach and the one taken in DoubleClick's (DFA's) new attribution tool, is to say "well we have one sale and a lot of adverts that might have driven it. What proportion of that one sale do we credit to each ad?"
By doing this you end up with a sale that gets split into pieces - maybe 50% to the paid search ad and 50% to the display ad. Maybe a larger proportion to the paid search ad, depending on your model. You can choose from six different ways of splitting up your online sales, within Google's DFA tool.
This method has one massive advantage. When you finish and add up all of your pieces of sales that have been allocated to different ads, you'll get back to the same total as you see on your final sales report.
And it has one massive disadvantage. It's wrong.
For a start and before we get to why it's wrong, giving people six options is downright unhelpful. There is a right answer to the question of attribution and letting people pick their own method is an analyst's equivalent of throwing up their hands and saying "I dunno, could be anything. Pick an answer you like."
Now why is the approach wrong?
To be fair, this isn't an online only problem. In most marketing analysis, we like to pretend that one sale comes from one marketing channel, because it makes life easier. If you've ever seen a nice neat list of return on investment numbers for different marketing campaigns, then this is effectively what you're looking at. Add up the individual contributions of each marketing campaign and you get total sales from marketing. Simple.
Marketing campaigns in the real world don't work like this, even though we like to pretend that they do. In the real world, people see lots of our ads and it needs lots of ads to persuade somebody to respond. Attribution analysis is specifically trying to solve the question of how (online) marketing campaigns work together, rather than individually, but by splitting up a single sale, we're not answering that question.
Take our example again of the customer who saw a display ad, then searched on Google, then bought something. Assume for the moment that they only decided to search for us, because they'd already seen the display ad. We must believe that sort of thing happens, or we wouldn't bother with attribution analysis in the first place.
What happens if you take away just the display ad? That person doesn't ever see the display ad, so they don't do a search and they don't buy our product. We lose one sale.
And if you take away the search ad? The person sees the display ad, searches on Google and doesn't find us. We lose one sale.
1+1 = 2
No we don't lose two sales in total. That would be ridiculous...
This is why, in real life, you can't add up the individual return on investment to each marketing channel. You'll get to a number that's bigger than your total sales.
But it's still the right approach.
When you use attribution to ask "what does this display advert add to my sales?", you're asking the wrong question. The right question - the one we just worked through above, is:
"What would happen to my sales if I took this display advert away?"
You'd lose a sale. Not half a sale, or 25% of a sale. A whole one.
And at this point you can see that this isn't just an academic argument. Your return on investment numbers to individual ads, using the dividing-up-sales method in Google's DFA tool, are too low. You will potentially remove campaigns that are actually working. This is important.
The only way to do attribution analysis is to ask how many sales you would have lost if an advert wasn't running. This is the question that was asked and is the whole point of attribution analysis. To do it any other way, is to pretend that you're measuring how different adverts work together, while actually measuring them individually.
In technical language, your attribution model must be multiplicative, not linear. You're going to need regression analysis on cookie-level data to build it and you're only going to get one answer, not six.
Still with me?
Next step...
Yes, it makes communicating results to clients harder.
It also makes it harder to forecast the impacts of changes to your campaigns. If you don't run search ads, your display ROI will drop. If you don't run display ads, your search ROI will drop. This is the essence of attribution modelling.
You need a forecasting tool, that can take what you've discovered from attribution modelling and project changes in overall marketing ROI, from a change to one campaign. This is what we've been building and what we're continuing to develop. It's been difficult, but very rewarding, and is now in use on live campaigns - it works.
I'm sure Google knows all this, but has chosen not to implement it, because it's more difficult to build and quite a lot more difficult for end users to understand. What you need to understand though, is that any attribution method which splits up a single sale, will encourage you to low-ball your return on investment numbers. And we can all agree, that's definitely a bad thing.
1 comment:
Do you know him?
http://econsultancy.com/it/blog/62586-a-day-in-the-life-of-an-attribution-manager?utm_campaign=Argyle+Social-2013-04&utm_medium=Argyle+Social&utm_source=LinkedIn&utm_term=2013-04-22-17-11-50
Post a Comment