Users don't want 'clever' dashboards

You're an analyst. Probably. That's why you're reading Wallpapering Fog.

If not an analyst, then at least inquisitive; if somebody shows you data then you want to dive a bit deeper and understand why the top-line numbers are doing what they're doing.

The more client dashboards we build, the more I'm discovering that while everybody always says they want to dive into the data, most people actually don't. If you give most marketing managers an interactive dashboard, they won't interact with it. They'll look at the screens in their default state, read what they can from them and then stop.

You can try to demonstrate how to interact with the data. When a marketer asks "why is my website bounce rate so high?", you can drill the number down, while they watch, and show which sources are sending the low quality traffic. They'll nod and thank you. Then next week, they'll ask exactly the same question again.

Unless you set up a screen which is specifically designed to answer the bouncing traffic question, without needing to be manipulated. A specific screen called "Sources of bouncing traffic". Then you'll have a happy client.

Who'll think of a different question, that you haven't already built a screen to answer.

I'm not being negative about interactive dashboards. I love interactive dashboards. Especially Tableau ones, but most of our clients aren't like me. If they were they'd be analysts, not clients.

If you're building dashboards for non-analysts, you may well find that they get a better reception if they're not interactive. Usually a marketer only has the same few questions on a Monday morning and if you can set up screens that answer them without being manipulated then you've got a happy marketer.

You could instead build an interactive screen, which with a few clicks will answer lots of questions, which would need multiple different static views to do the same job. Want a week-on-week and a year-on-year view? Just click the drill down button! What's the difference between branded and generic keyword searches? Click into the 'total searches' line and it will show you!

Except your marketer won't click the drill down button. They'll get frustrated that their dashboard isn't showing exactly what they want.

You can get annoyed about this. People don't want your clever interactive screens!

Or you can see the advantages.

We're doing quite a nice line in dashboards of Google Analytics data. The clients have a login to Google Analytics just like we do, but they don't use it because the GA website doesn't immediately show them exactly the stats, for exactly the date breakdown that they need, on one screen. With Tableau and Python to hit the Google Analytics API, it's very easy to set that up and automatically refresh it.

Now your client has a dashboard that they can't do without, that shows them exactly what they want at 9am on a Monday and you know what else? You don't even have to bill them for a server login because they want static views, so PDFs in their email are actually better than a login to an interactive system.

Not for everybody though, and those logins are still important. You can give analysts (or analytically minded people) on the client side some server logins, but I guarantee the Marketing Director doesn't want one. They might say they want one, but they won't use it, which is where we started. If they needed your dashboard login then they'd already be using Google Analytics.

A lot of dashboard software (and certainly Tableau) is really fantastic at two jobs. One, is making data interactive and easy to interrogate and analysts love that. The other is making refreshes of static screens really easy. Think hard when you're designing, because for a lot of people, those static screens are better.

Static screens may also be much harder to get right, even though building them feels a lot less clever. When you can't ask the user to click through to what they want, you have to know what they want before they arrive at your dashboard.

You have to really understand the client and their business. That's the really clever bit.

Are you ready for Real Time Planning?

This is a reproduction of an article I've written for this month's Admap. They've chosen to title it 'Track the data on the dashboard', which I think rather misses the point but there you go. On Wallpapering Fog, I choose the headlines.


Real-time planning is a tactical tool that, through analysis of customer behavioural data, enables the short-term refinement of communications strategy, explains Neil Charles of MediaCom.


Real-time planning is one of those marketing terms that has a danger of meaning different things to different people, so I'd like to start off with a brief definition. For me, real-time planning means adapting marketing schedules on the fly, in reaction to new data about how customers behave.

The challenge that this type of adaptable marketing presents is to process new data and then react quickly enough, to take advantage of opportunities as they are identified. However, too often, marketers expect data on its own to be enough and that deep insights will reveal themselves if only we can bring different data sources together. Analysts have known for a long time that this is rarely the case, but large quantities of consumer data are seductive. Surely we could build a more efficient, more flexible media schedule if we had more up- to-date tracking of consumer behaviour?

Inevitably, the data that has provoked this new marketing philosophy flows from the web. We have faster access to more granular data than ever before, both in terms of marketing response through clicks and traffic tracking, and also the ability to ask questions of large online research panels cheaply, and to see the results in a very short period of time.

In practical terms, the web will largely be the focus for the outputs from real-time planning too. Traditional media - where the creative process and buying deadlines are longer-lend themselves much less readily to the type of quick schedule changes, which allow us to take advantage of new data. This online focus should put real-time planning in context for marketers as an exciting new possibility, but one which must never be allowed to compromise an overall campaign. The Internet Advertising Bureau and PricewaterhouseCoopers put UK internet spend at £4 billion in 2010, accounting for 25% of all advertising spend. So while we may have the ability to monitor consumer behaviour (on the web at least) in almost real-time, only a part of the marketing budget is as agile as the response data that we can monitor. Of course, TV or press schedules can be adjusted, but once a commitment to TV has been made, barring disaster, the ads will run largely as planned.

Crucially, most ads should run largely as planned. We often preach the benefits of consistency in advertising and of seeing a brand campaign through, for its full benefits to be felt. Real-time planning doesn't replace the normal planning process, but is about tactical adjustments to a campaign that has been well planned in advance. If our understanding of new data is allowed to constantly re-shape a brand's proposition then we risk compromising our ability to put across a consistent message to consumers.

So, with real-time planning in context as a tactical, rather than strategic tool, and one that is based on very recent data about our customers, what do we need to do to make it work?

It is easy to generate and to track extremely large volumes of customer data. Over the past few years, dashboard software has become cheap and capable, and for a small IT investment, marketers can easily bring together their sales information every week, their own brands' and competitors' advertising spends, response data from off-line direct marketing channels and web tracking from a count of homepage visitors, right down to the number of clicks on individual Google keywords. We can also incorporate brand mentions and sentiment from social networks, track PR coverage both online and offline and conduct quick consumer research polls.

Collecting this data and visualising it, in the hope that it will provide insight and lead to greater marketing efficiencies usually results in disappointment. Large volumes of data, without analysis, are more of a hindrance than a help and, unfortunately, analytical insights very rarely jump off the page from a single chart.

Even where a relationship is obvious - such as when the number of brand term searches is charted against TV investment - what do we do with this information? It's not enough to know that TV is driving additional customers to search for us on Google. We need to know whether this means we should increase the TV budget, attempt to convert more of the online interest that TV is shown to be generating - both, or possibly neither. After all, the current schedule appears to be working!

Rather than tracking large volumes of data and hoping to generate insights from them that will lead to more efficient marketing, the data that we choose to track should flow from analysis work that has already been completed.We need analysts to identify from the vast quantity of available information, variables which are useful, show how they can be used and then to hand that information to marketers.

A recent client example concerned a business which had no concrete data on overall sales volumes in its market, but many variables that might indicate whether they were rising or falling. Sales in the client's business were rising and they wanted to know whether - as some believed internally - this was bucking the market trend, or following it. The answer would have significant implications for advertising, since if the overall market wasn't getting stronger, then the most likely candidate to have caused the extra sales was a recent increase in marketing spend.

Large volumes of data were available that might provide insight, from a set of total market sales estimates that may or may not have been reliable, through to a number of Google searches for various brand and product terms and government economic data on the health of related sectors. The data contradicted each other and tracking alone raised many more questions than it answered.

A long-term econometric study into the drivers of sales had recently been completed, which identified a few key Google search terms that accurately mirrored market trends. This prior analysis flagged up data that was worth tracking and which could answer the question: No, marketing response didn't appear to have changed, and yes, increasing sales were being led by a market recovery.

The key point here is that the data we track to aid our marketing efforts, and which we aim to use to refine campaigns on the fly, should already have an identified purpose at the point when we decide to track it. Data that we do not yet understand in detail doesn't allow us to plan in real-time, it raises questions, which first need to be answered. Answering those questions is an analysis process that can take from a few weeks, to several months.

Together with data, which is already well understood, a second ingredient is needed for real-time planning to work. We need to know beforehand, what our likely reaction will be to a change in the data.

Marketing dashboards, metrics and tracking should be like the petrol gauges or the speedometer on a car. When they change, we already know why and so we already know what to do about it. When the petrol gauge gets too low, we stop and fill up, to avoid an embarrassing call for a tow from the side of the road.

A lot of information about your car isn't displayed on the dashboard. Not because it isn't useful at all, but because it isn't useful minute-by-minute and would be a distraction from driving. This sort of information - on engine efficiency for example - is checked annually when the car is serviced. Marketing analysis should work the same way, meaning that we track what we already understand and can respond to, and ask mechanics (our planners and analysts) to react to more complex data once or twice a year. What the analysts discover might increase the scope of real-time planning as different data becomes well understood. To stretch the car dashboard analogy, we might gain new warning lights on the dashboard, but we are unlikely to start visualising large quantities of new data.

Without prior analysis, there is a sub-set of data that is always useful and, realistically, this is where a lot of brands already do 'real-time planning', whether it is labelled as that or not. Based on direct response data from clicks or phone calls, under-performing press insertions, search keywords and display placements can be pruned from a schedule in real-time without any need for further analysis. Their budget will be allocated to ads with a better response rate, and so all we need to know is that there is a better ad where we could be spending the money instead. It doesn't matter what is the true return on investment to a display ad - only that we can move budget from an under-performing ad to a stronger one that generates more clicks for the same money.

I should point out here that I'm absolutely not arguing against collecting marketing data. We have incredible quantities of information at our disposal to track and better understand consumers and we should keep them, because we don't always know what will be useful until later. This article is about reacting to that data in real-time and day-to-day, those volumes of data become a hindrance rather than a help. Once we focus only on the data that we genuinely understand, a lot of available data - from follower counts to web traffic - becomes surplus to requirements until somebody can work out why it's useful and what it means when it changes.

Even a measure of total sales or footfall to a store, is of dubious value to a marketer who doesn't know what impact the brand's marketing activity will have on the metric. A drop in sales presents two immediate possibilities - spend more on marketing in order to restore sales to where they were previously, or cut marketing in response to a worsening business environment. The data only becomes useful and real-time planning becomes possible, if econometric models or other in-depth response analyses are already in place. Then it is possible to estimate what marketing can achieve, given the data that we're tracking and to decide on the best course of action.

In summary, I would argue that real-time planning is a tactical, rather than a strategic tool. It creates efficiencies on smaller parts of an over-arching marketing strategy and allows us to quickly remove inefficient parts of the marketing mix, or to take advantage of short-term opportunities. It also allows us to increase the amount of marketing investment when that money is shown to be working harder than usual. The overall marketing plan, though, should be driven by longer term in-depth insight work and certainly shouldn't be compromised by trying to make too many short-term tactical gains.

To make real-time planning work, we need data and we need to have done some prior analysis. Monitoring data series that start a debate when they change can be helpful, but it doesn't allow us to make rapid changes to a marketing schedule. An upfront investment in statistical modelling, so that we fully understand the data that we monitor, allows us to predict the likely outcomes of making a change to the marketing schedule.

Real-time planning is about investing in analysis and preparing for situations that could be faced in the future, and if you haven't done that prior analysis, then you're not ready for real-time planning.

As a small illustration of these principles in action, Brilliant Media has a retail client where analysis has revealed that strong online sales can be generated, by up-weighting search activity against a competitor's television schedule. The competitor TV activity is largely predictable and the benefits of diverting the online interest that it generates have been proven. As a result, competitor TV schedules are closely tracked and search terms up-weighted to take advantage of the spikes in search volumes that they generate. This adaptable schedule has real benefits in terms of additional sales and has arisen as a result of a piece of investigative analysis that identified data that was worth tracking and could be responded to very rapidly.

In the end, I would argue that real-time planning is something of a contradiction in terms. We shouldn't attempt to plan in real-time; we plan and we analyse, so that we can react in real-time.

Reproduced with permission of Admap, the world’s primary source of strategies for effective advertising, marketing and research. To subscribe visit www.warc.com/admap. © Copyright Admap.

The chart you should never see on a dashboard

This chart communicates two data series pretty well. It's not cluttered and it's definitely not a pie chart.

Unfortunately, it's also an almost guaranteed sign of a dashboard that's looking for a purpose and that hasn't been designed with a task in mind. A dashboard that won't help you.

Dashboards are for monitoring data that you already understand. Data, which you very likely already know how you'll react to when it changes. This chart is for analysis - it compares data, looking for a relationship.

Take the term 'dashboard' literally for a minute and think about the data that's displayed on the dashboard in your car. Your speed is shown on there, because you need to know it and because you already know how to make it change when it's too fast or too slow.

You have a petrol gauge too. That's useful data to avoid an embarrassing call to the AA from the hard shoulder because you've run out of petrol. Again, when the gauge drops, you already know what to do.

A lot of data about your car isn't on the dashboard. There's no report on the mix of gasses coming out of your exhaust for example.That's useful information to know whether your engine is running efficiently, so why isn't it there?

It's not there, because if it's out of line, what are you going to do about it? If you're like me, then you probably don't even know what out of line might look like. It would be a distraction from driving at best. There might well be a warning light on the dashboard that says 'see a mechanic' when the exhaust gasses aren't right but that's a very different concept to showing you all of the data and hoping that you draw the right conclusions.

Back to the chart then. It's a classic sign of a dashboard that's trying to be used for analysis, rather than for keeping an eye on indicators that you understand. We had a client recently ask for a chart of their sales vs. the weather and it's very much that kind of chart. The client didn't really want a chart of their sales vs. the weather; they wanted to know if their sales were affected by the weather and they thought that the chart might tell them.

It wouldn't.

You need a mechanic for that. In marketing, we call mechanics analysts.

Dashboarding large amounts of data that you don't already understand in detail, won't help to improve your marketing. It will distract you from driving.

Monitoring a lot of statistics and hoping to spot relationships between them won't work. You need to be a mechanic, or at least to want to learn to be one.

Dashboards are for metrics that you understand. That you already know how to respond to. Everything else is analysis and it doesn't belong on your dashboard.

A bit of context

The UK police have launched a new website this week, which allows you to look up crime statistics at street level. The figures have been available for neighbourhoods for a while but this level of detail is new and now that the site is back up again, good fun to play with.

The methodology and lack of data history have been criticised. As always with a database this big, looking at the outliers doesn't find you interesting facts (much to the newspapers' annoyance) it finds you anomalies in the data. In this case, the 'most dangerous places in Britain' are streets where the postcode is used as a quick location to log all of the crime that happens in a city centre. It's not that 500 people a month are arrested on the same small street for being drunk and disorderly, just that the police need a postcode for their database, so they use that one.

I've recently moved house, up to my exciting new econometrics job in Leeds, so looked up my new neighbourhood.


Wow. It seemed like such a nice, friendly, quiet little town! Now I can see that just in the last month there have been assaults, burglary, theft from cars... I'm scared.

What about my old address in SE London? I lived there for five years and never had a problem (except for a chav nicking my satnav, but I'm an idiot for leaving it on display in the car.)


Suddenly I feel safer in my new home!

I'm sure a policeman would have looked at the Mirfield statistics and pictured a fairly safe little town. I'm not an expert though and any number above zero looks like a crime hotspot.

If you're producing a dashboard for a non-expert audience - whether it's showing crime or click through rates - then context is essential. Is the number you're looking at good? Bad? Indifferent? The old neighbourhood level mapping did give context and in that very important way was the superior tool.

Why a dashboard won't solve all of your problems

Business Intelligence (BI) has come of age. Assuming your IT department know roughly what they're doing and you can afford it, you can have your latest sales, market share, media spend, Google conversion rates and any other metrics you care to mention on your screen on a Monday morning.

More information is incredibly seductive. If you had your sales and media spend histories at your fingertips, you could show the effectiveness of your current advertising campaign. Surely? Couldn't you?

Well actually, no you couldn't. Marketing analysts have had this data for ages and it takes them a couple of months (and a degree in statistics) to work it out.

As analysts, we're often the ones selling dashboards, so lets be honest about what they do well. They show data. So to be useful, you have to be someone who needs to see that data - and I mean really needs to see it. Just the number. Not why the number, or where it came from, or what you might want to do about it.

Anything that goes beyond looking at a number isn't a dashboard, it's insight and analysis. Some pieces of software that can be used to build dashboards also have great tools for generating insight, like Qlik View, or Tableau. They don't do the analysis work for you though and if you haven't got staff with the time or the maths background required to go digging through the numbers then that dashboard's not going to help you make a decision.


Look at it like this; here's a dashboard in the true sense of the word.


It's got all the things on it that you might need to know while you're driving. Even some of the indicators on here might be considered surplus to requirements, but we can agree that knowing how fast you're going and whether you have any fuel left are useful pieces of information to have while you're in a car.

They're useful, because you know what to do about them if they're not right. Out of fuel? Find a petrol station, quick.

A car's dashboard doesn't tell you how to make the car slow down, how to mend it if it won't start or whether you should fit a loud exhaust and a spoiler.

Dashboards can be incredibly useful, but if you want to know how your business works, then they're not all you need.