Users don't want 'clever' dashboards

You're an analyst. Probably. That's why you're reading Wallpapering Fog.

If not an analyst, then at least inquisitive; if somebody shows you data then you want to dive a bit deeper and understand why the top-line numbers are doing what they're doing.

The more client dashboards we build, the more I'm discovering that while everybody always says they want to dive into the data, most people actually don't. If you give most marketing managers an interactive dashboard, they won't interact with it. They'll look at the screens in their default state, read what they can from them and then stop.

You can try to demonstrate how to interact with the data. When a marketer asks "why is my website bounce rate so high?", you can drill the number down, while they watch, and show which sources are sending the low quality traffic. They'll nod and thank you. Then next week, they'll ask exactly the same question again.

Unless you set up a screen which is specifically designed to answer the bouncing traffic question, without needing to be manipulated. A specific screen called "Sources of bouncing traffic". Then you'll have a happy client.

Who'll think of a different question, that you haven't already built a screen to answer.

I'm not being negative about interactive dashboards. I love interactive dashboards. Especially Tableau ones, but most of our clients aren't like me. If they were they'd be analysts, not clients.

If you're building dashboards for non-analysts, you may well find that they get a better reception if they're not interactive. Usually a marketer only has the same few questions on a Monday morning and if you can set up screens that answer them without being manipulated then you've got a happy marketer.

You could instead build an interactive screen, which with a few clicks will answer lots of questions, which would need multiple different static views to do the same job. Want a week-on-week and a year-on-year view? Just click the drill down button! What's the difference between branded and generic keyword searches? Click into the 'total searches' line and it will show you!

Except your marketer won't click the drill down button. They'll get frustrated that their dashboard isn't showing exactly what they want.

You can get annoyed about this. People don't want your clever interactive screens!

Or you can see the advantages.

We're doing quite a nice line in dashboards of Google Analytics data. The clients have a login to Google Analytics just like we do, but they don't use it because the GA website doesn't immediately show them exactly the stats, for exactly the date breakdown that they need, on one screen. With Tableau and Python to hit the Google Analytics API, it's very easy to set that up and automatically refresh it.

Now your client has a dashboard that they can't do without, that shows them exactly what they want at 9am on a Monday and you know what else? You don't even have to bill them for a server login because they want static views, so PDFs in their email are actually better than a login to an interactive system.

Not for everybody though, and those logins are still important. You can give analysts (or analytically minded people) on the client side some server logins, but I guarantee the Marketing Director doesn't want one. They might say they want one, but they won't use it, which is where we started. If they needed your dashboard login then they'd already be using Google Analytics.

A lot of dashboard software (and certainly Tableau) is really fantastic at two jobs. One, is making data interactive and easy to interrogate and analysts love that. The other is making refreshes of static screens really easy. Think hard when you're designing, because for a lot of people, those static screens are better.

Static screens may also be much harder to get right, even though building them feels a lot less clever. When you can't ask the user to click through to what they want, you have to know what they want before they arrive at your dashboard.

You have to really understand the client and their business. That's the really clever bit.

Are you ready for Real Time Planning?

This is a reproduction of an article I've written for this month's Admap. They've chosen to title it 'Track the data on the dashboard', which I think rather misses the point but there you go. On Wallpapering Fog, I choose the headlines.


Real-time planning is a tactical tool that, through analysis of customer behavioural data, enables the short-term refinement of communications strategy, explains Neil Charles of MediaCom.


Real-time planning is one of those marketing terms that has a danger of meaning different things to different people, so I'd like to start off with a brief definition. For me, real-time planning means adapting marketing schedules on the fly, in reaction to new data about how customers behave.

The challenge that this type of adaptable marketing presents is to process new data and then react quickly enough, to take advantage of opportunities as they are identified. However, too often, marketers expect data on its own to be enough and that deep insights will reveal themselves if only we can bring different data sources together. Analysts have known for a long time that this is rarely the case, but large quantities of consumer data are seductive. Surely we could build a more efficient, more flexible media schedule if we had more up- to-date tracking of consumer behaviour?

Inevitably, the data that has provoked this new marketing philosophy flows from the web. We have faster access to more granular data than ever before, both in terms of marketing response through clicks and traffic tracking, and also the ability to ask questions of large online research panels cheaply, and to see the results in a very short period of time.

In practical terms, the web will largely be the focus for the outputs from real-time planning too. Traditional media - where the creative process and buying deadlines are longer-lend themselves much less readily to the type of quick schedule changes, which allow us to take advantage of new data. This online focus should put real-time planning in context for marketers as an exciting new possibility, but one which must never be allowed to compromise an overall campaign. The Internet Advertising Bureau and PricewaterhouseCoopers put UK internet spend at £4 billion in 2010, accounting for 25% of all advertising spend. So while we may have the ability to monitor consumer behaviour (on the web at least) in almost real-time, only a part of the marketing budget is as agile as the response data that we can monitor. Of course, TV or press schedules can be adjusted, but once a commitment to TV has been made, barring disaster, the ads will run largely as planned.

Crucially, most ads should run largely as planned. We often preach the benefits of consistency in advertising and of seeing a brand campaign through, for its full benefits to be felt. Real-time planning doesn't replace the normal planning process, but is about tactical adjustments to a campaign that has been well planned in advance. If our understanding of new data is allowed to constantly re-shape a brand's proposition then we risk compromising our ability to put across a consistent message to consumers.

So, with real-time planning in context as a tactical, rather than strategic tool, and one that is based on very recent data about our customers, what do we need to do to make it work?

It is easy to generate and to track extremely large volumes of customer data. Over the past few years, dashboard software has become cheap and capable, and for a small IT investment, marketers can easily bring together their sales information every week, their own brands' and competitors' advertising spends, response data from off-line direct marketing channels and web tracking from a count of homepage visitors, right down to the number of clicks on individual Google keywords. We can also incorporate brand mentions and sentiment from social networks, track PR coverage both online and offline and conduct quick consumer research polls.

Collecting this data and visualising it, in the hope that it will provide insight and lead to greater marketing efficiencies usually results in disappointment. Large volumes of data, without analysis, are more of a hindrance than a help and, unfortunately, analytical insights very rarely jump off the page from a single chart.

Even where a relationship is obvious - such as when the number of brand term searches is charted against TV investment - what do we do with this information? It's not enough to know that TV is driving additional customers to search for us on Google. We need to know whether this means we should increase the TV budget, attempt to convert more of the online interest that TV is shown to be generating - both, or possibly neither. After all, the current schedule appears to be working!

Rather than tracking large volumes of data and hoping to generate insights from them that will lead to more efficient marketing, the data that we choose to track should flow from analysis work that has already been completed.We need analysts to identify from the vast quantity of available information, variables which are useful, show how they can be used and then to hand that information to marketers.

A recent client example concerned a business which had no concrete data on overall sales volumes in its market, but many variables that might indicate whether they were rising or falling. Sales in the client's business were rising and they wanted to know whether - as some believed internally - this was bucking the market trend, or following it. The answer would have significant implications for advertising, since if the overall market wasn't getting stronger, then the most likely candidate to have caused the extra sales was a recent increase in marketing spend.

Large volumes of data were available that might provide insight, from a set of total market sales estimates that may or may not have been reliable, through to a number of Google searches for various brand and product terms and government economic data on the health of related sectors. The data contradicted each other and tracking alone raised many more questions than it answered.

A long-term econometric study into the drivers of sales had recently been completed, which identified a few key Google search terms that accurately mirrored market trends. This prior analysis flagged up data that was worth tracking and which could answer the question: No, marketing response didn't appear to have changed, and yes, increasing sales were being led by a market recovery.

The key point here is that the data we track to aid our marketing efforts, and which we aim to use to refine campaigns on the fly, should already have an identified purpose at the point when we decide to track it. Data that we do not yet understand in detail doesn't allow us to plan in real-time, it raises questions, which first need to be answered. Answering those questions is an analysis process that can take from a few weeks, to several months.

Together with data, which is already well understood, a second ingredient is needed for real-time planning to work. We need to know beforehand, what our likely reaction will be to a change in the data.

Marketing dashboards, metrics and tracking should be like the petrol gauges or the speedometer on a car. When they change, we already know why and so we already know what to do about it. When the petrol gauge gets too low, we stop and fill up, to avoid an embarrassing call for a tow from the side of the road.

A lot of information about your car isn't displayed on the dashboard. Not because it isn't useful at all, but because it isn't useful minute-by-minute and would be a distraction from driving. This sort of information - on engine efficiency for example - is checked annually when the car is serviced. Marketing analysis should work the same way, meaning that we track what we already understand and can respond to, and ask mechanics (our planners and analysts) to react to more complex data once or twice a year. What the analysts discover might increase the scope of real-time planning as different data becomes well understood. To stretch the car dashboard analogy, we might gain new warning lights on the dashboard, but we are unlikely to start visualising large quantities of new data.

Without prior analysis, there is a sub-set of data that is always useful and, realistically, this is where a lot of brands already do 'real-time planning', whether it is labelled as that or not. Based on direct response data from clicks or phone calls, under-performing press insertions, search keywords and display placements can be pruned from a schedule in real-time without any need for further analysis. Their budget will be allocated to ads with a better response rate, and so all we need to know is that there is a better ad where we could be spending the money instead. It doesn't matter what is the true return on investment to a display ad - only that we can move budget from an under-performing ad to a stronger one that generates more clicks for the same money.

I should point out here that I'm absolutely not arguing against collecting marketing data. We have incredible quantities of information at our disposal to track and better understand consumers and we should keep them, because we don't always know what will be useful until later. This article is about reacting to that data in real-time and day-to-day, those volumes of data become a hindrance rather than a help. Once we focus only on the data that we genuinely understand, a lot of available data - from follower counts to web traffic - becomes surplus to requirements until somebody can work out why it's useful and what it means when it changes.

Even a measure of total sales or footfall to a store, is of dubious value to a marketer who doesn't know what impact the brand's marketing activity will have on the metric. A drop in sales presents two immediate possibilities - spend more on marketing in order to restore sales to where they were previously, or cut marketing in response to a worsening business environment. The data only becomes useful and real-time planning becomes possible, if econometric models or other in-depth response analyses are already in place. Then it is possible to estimate what marketing can achieve, given the data that we're tracking and to decide on the best course of action.

In summary, I would argue that real-time planning is a tactical, rather than a strategic tool. It creates efficiencies on smaller parts of an over-arching marketing strategy and allows us to quickly remove inefficient parts of the marketing mix, or to take advantage of short-term opportunities. It also allows us to increase the amount of marketing investment when that money is shown to be working harder than usual. The overall marketing plan, though, should be driven by longer term in-depth insight work and certainly shouldn't be compromised by trying to make too many short-term tactical gains.

To make real-time planning work, we need data and we need to have done some prior analysis. Monitoring data series that start a debate when they change can be helpful, but it doesn't allow us to make rapid changes to a marketing schedule. An upfront investment in statistical modelling, so that we fully understand the data that we monitor, allows us to predict the likely outcomes of making a change to the marketing schedule.

Real-time planning is about investing in analysis and preparing for situations that could be faced in the future, and if you haven't done that prior analysis, then you're not ready for real-time planning.

As a small illustration of these principles in action, Brilliant Media has a retail client where analysis has revealed that strong online sales can be generated, by up-weighting search activity against a competitor's television schedule. The competitor TV activity is largely predictable and the benefits of diverting the online interest that it generates have been proven. As a result, competitor TV schedules are closely tracked and search terms up-weighted to take advantage of the spikes in search volumes that they generate. This adaptable schedule has real benefits in terms of additional sales and has arisen as a result of a piece of investigative analysis that identified data that was worth tracking and could be responded to very rapidly.

In the end, I would argue that real-time planning is something of a contradiction in terms. We shouldn't attempt to plan in real-time; we plan and we analyse, so that we can react in real-time.

Reproduced with permission of Admap, the world’s primary source of strategies for effective advertising, marketing and research. To subscribe visit www.warc.com/admap. © Copyright Admap.

Dashboard software - why we chose what we chose

Some time ago, I wrote a post that laid out what I want from a piece of dashboarding and data visualisation software. We had just finished going through the process of selecting a software platform and I'd intended to lay out over a series of posts the different options that I looked at and explain how we made our final choice.

Apart from a review of Excel Services (summary: it's sort of ok, but you can do much better for the money), the series didn't get very far. Due to the overwhelming volume of emails Despite a complete lack of emails accusing me of being a lazy so and so who doesn't finish what he starts, I'm going to quickly wrap up the conclusions here.

First, the contenders. Some of these I looked at for longer than others and I can guarantee a few first round knockouts.

Ladies and gentlemen, in the blue corner etc.

Tableau

Spotfire

Qlik View

Excel Services

LogiXML

Microsoft Analysis Services

Pentaho

Microstrategy


We'll start with where I started the process - looking for a cheap but capable solution, that would work for a medium sized business. I'm not looking at six figure software budgets here, or at running the back end for the Tesco Clubcard.

With that in mind, I went looking for an open source solution and found Pentaho. There are other 'free' platforms, including customized Google Docs, but all require programming and we need a drag-and-drop front end that analysts can use to build dashboards. For that requirement, Pentaho seems to be what's available.

First impression is that it's not bad at all. A little rough around the edges as you might expect from an open source project, but I had some simple dashboards up and running on a demo server pretty quickly.

These demos were enough to show a few senior managers and to explain why we should be buying a dashboard platform rather than building our own, but then things started to go wrong. If you're going to do anything more than a few simple charts, then you're going to end up doing some programming, which means as an analyst that you need IT's help. IT's help brings change requests and (long) development time-lines and all the joys of not being able to put data labels on a bar chart by yourself. Scheduled refreshing also looked like it was going to be trickier than it needed to be.

Once you're building and customising with your own IT department, you're incurring extra costs, so it was off to the commercial software world to see if off-the-shelf was a better idea. It was.

When you head for a commercial piece of dashboard software, you find out that there are at least tens available and probably hundreds. Most are rubbish. You also get into some very frustrating conversations with salesmen about licencing costs, but more on that later.

Let's eliminate a few contenders quickly here, pretty much as quickly as I dropped them during the assessment process.

I've already reviewed Excel Services in detail so won't go through that again. For the price, it isn't good enough and it's awkward to use.

Analysis Services is much more capable, but it's got that Microsoft feel. You know what I mean; it makes things hard that should be easy. Options are hidden, it's all tied into SharePoint (which I hate) and you end up writing loads of custom SQL - which will be a nightmare to maintain - in order to work around things that Microsoft either doesn't want you to do or hasn't included as standard. The final dashboards also feel like a tool for analysts rather than a tool for marketing managers and if you're not at least a little geeky, they can be quite intimidating. Rejected.

LogiXML and Microstrategy were very much the also-rans in this race. LogiXML, because it's bloody awful and Microstrategy because it's just not good enough to be up with the front runners.

Here's a LogiXML demo dashboard. Apparently they're proud of it and if you can't see why that's a problem then go ahead and buy a copy! I'm being deliberately harsh because they keep sending me junk email. Petty revenge is sweet.

Microstrategy sounds pretty exciting when you read the blurb on their website. The demos aren't front and centre on the site though and you have to dig a bit to find them. That's a warning sign for me. If a software company knows that it has a good product, you'll find links to the product itself featured prominently. Microstrategy has a lot of claims about what it can do on the homepage and then a few clicks and some searching around takes you here...

I'm sure it works ok, but the looks are five out of ten at best. I did download the demo but it took ages to install and the dashboards are slow to load on a browser too. All in all it just didn't feel like a smooth, slick piece of software and I couldn't see the visuals impressing a marketer. Another one bites the dust.

Now we're onto the serious contenders: Spotfire, Tableau and Qlikview.

Spotfire first, because it didn't make it quite as far as the other two. Spotfire is undoubtedly a tremendously capable piece of software but at Brilliant Media, we're building dashboards for marketers and they need to present data, more than they need to make it available for investigation by an analyst.

With that purpose in mind, Spotfire suffers from some of the same issues as Microsoft Analysis services. It's less awkward than Analysis Services, but in use it feels like it was built for an analyst. It's the sort of software that would be great to sit on top of a loyalty card database for an analytical team to use as their tool for day-to-day interrogation. It's not what we need to present relatively top-line data to marketers.

Take a look at the demos and you'll see what I mean. It's good software, but not for us.

Spotfire has another major problem and that's Qlikview. If I wanted what Spotfire does, then I probably wouldn't buy Spotfire, I'd buy Qlikview. It's an amazing piece of software.

I've been playing with Qlikview demos on and off for a few years and it has a few stand-out characteristics. It runs like lightning and the data engine is fabulous - importing your data quickly and making intelligent guesses about how it's structured. You need ten minutes guidance to get used to how the dashboard front ends work, but once you've got it, then Qlikview is very simple to use.

Admittedly there are a lot of options for customising views and you get to them via large, complex options windows, rather than by a quick right click on the thing that you want to change, but fairly quickly you get used to where to find what you want. The positive side of all of these options is that you can create virtually any view that you might need.

Qlikview was a runner up though, based on cost. Actually it might have been dead last based on cost, because I never managed to find out how much it is. One of my criteria for software was transparent pricing and Qlikview's reps are a nightmare to talk to (I've had a few friends in IT back up that opinion too.) They want meetings; they want to know all about your business; they promise free copies of the software. What they absolutely will not do is give you a figure for how much it's going to cost to deploy the software onto x analysts' desktops and allow them to publish to a server.

It's a shame, but Qlikview was rejected on those grounds, even though I loved the demo.

------------------------------------------------------------------------------------------------------
Edit (22-1-13): Qlikview has moved to transparent pricing since this post was written (pointed out in comments below.) That's great to hear. Prices here.

Initial impression? The product mix looks a little over-complicated and pricier than Tableau. I'm not sure if Qlikview is targeting larger, more complex deployments than we've built, but I wouldn't change the conclusions below, based on anything I've seen over the past year or so.
------------------------------------------------------------------------------------------------------

Which brings me onto our winner. Tableau. Of course it's Tableau.

Tableau seems to be a well kept secret for now among analysts who've done their homework, but I can't see  it staying that way for long. It's a joy to use - simple, fast and powerful.

The software's actually quite restrictive in the way that it will allow you to present data, but that's a very good thing. It's like a best-practice data visualisation engine. No you can't draw 3D pie charts. Dashboard creation is simple drag and drop and the best way I can describe it is as if somebody took Excel Pivot Tables, made them nice to use, made them look good and then gave you the ability to publish them to the web.

Pricing is transparent (and very reasonable.) The desktop dashboard builder software has a simple cost per user. Not an ongoing licence fee, but a one-off cost, although you can pay for ongoing support if you want to. The server also has a one-off cost per user and that's it. No OEM nonsense and easy to work out pricing for scale, buy it once and do what you want with it. Perfect.

Other dashboard vendors really need to take a leaf out of Tableau's book on pricing. It's so much easier to pitch the idea to management of investing in software when you've got a clear idea yourself of exactly what it will cost over the next three years. If the dashboards you build are a runaway success then how much is an extra five desktop licences and twenty server logins going to be?

If you haven't tried Tableau yet, then give it a go. There's a free version for bloggers that's great for trying out some data vis and publishing it to the web. I feel like I'm giving away a trade secret here, but what the hell, you're going to hear about it from somewhere soon anyway. Tableau has very quickly become an essential analysis tool for me alongside Excel and if you try it, I'm willing to bet that it will be for you too.

The chart you should never see on a dashboard

This chart communicates two data series pretty well. It's not cluttered and it's definitely not a pie chart.

Unfortunately, it's also an almost guaranteed sign of a dashboard that's looking for a purpose and that hasn't been designed with a task in mind. A dashboard that won't help you.

Dashboards are for monitoring data that you already understand. Data, which you very likely already know how you'll react to when it changes. This chart is for analysis - it compares data, looking for a relationship.

Take the term 'dashboard' literally for a minute and think about the data that's displayed on the dashboard in your car. Your speed is shown on there, because you need to know it and because you already know how to make it change when it's too fast or too slow.

You have a petrol gauge too. That's useful data to avoid an embarrassing call to the AA from the hard shoulder because you've run out of petrol. Again, when the gauge drops, you already know what to do.

A lot of data about your car isn't on the dashboard. There's no report on the mix of gasses coming out of your exhaust for example.That's useful information to know whether your engine is running efficiently, so why isn't it there?

It's not there, because if it's out of line, what are you going to do about it? If you're like me, then you probably don't even know what out of line might look like. It would be a distraction from driving at best. There might well be a warning light on the dashboard that says 'see a mechanic' when the exhaust gasses aren't right but that's a very different concept to showing you all of the data and hoping that you draw the right conclusions.

Back to the chart then. It's a classic sign of a dashboard that's trying to be used for analysis, rather than for keeping an eye on indicators that you understand. We had a client recently ask for a chart of their sales vs. the weather and it's very much that kind of chart. The client didn't really want a chart of their sales vs. the weather; they wanted to know if their sales were affected by the weather and they thought that the chart might tell them.

It wouldn't.

You need a mechanic for that. In marketing, we call mechanics analysts.

Dashboarding large amounts of data that you don't already understand in detail, won't help to improve your marketing. It will distract you from driving.

Monitoring a lot of statistics and hoping to spot relationships between them won't work. You need to be a mechanic, or at least to want to learn to be one.

Dashboards are for metrics that you understand. That you already know how to respond to. Everything else is analysis and it doesn't belong on your dashboard.

Dashboard & visualisation software: Excel Services review

It’s been a short wait for this first dashboard software review because… well because I’ve been busy building dashboards. Now with a long train journey from London to Leeds, it’s time to take a quick look at Excel Services. I wanted to look at this one first, because it doesn't seem to have been widely adopted and I've come across a few people who build very good Excel dashboards but who don't even know it exists. Excel is still the go-to solution for just about everything, so what about its Services sibling?

When I first came across Excel Services, I have to say my immediate reaction was “what took them so long?” Lots of people, including me, build dashboards in Excel - from simple report tables to large macro driven data explorers – and it’s a logical step from mailing them around a business to publishing them for viewing on a website: One version of the truth, no versioning problems and easier to read for people who aren’t comfortable in Excel.

Excel services sits on top of SharePoint and lets you publish spreadsheets from Excel 2007 onwards (*.xlsx files) to the web. You can then view elements of the spreadsheet in your web browser.

(microsoft don't really do pretty)

First the good, and publishing is fairly easy. You specify the sheets and named ranges that you want to make available and then from Excel, you save the workbook to the server. There’s not much more to it than that. Hopefully you already know how to make a reasonably good looking spreadsheet, so now you can make a reasonably good looking online dashboard.
Live data comes from data connections to pivot tables that refresh automatically when a user visits the dashboard. Not the fastest, but it works.

There are a few things you can’t publish and we’ll come back to the limitations of Excel Services later, as you’ll encounter some of the issues pretty early on. Big things that you can’t use are:

• Macros. Your workbook has to work with worksheet formulas only.
• Any kind of drawing object (including buttons and pictures) so you’ll have to make it look good by colouring in cells and giving them borders.
  

Once you’ve published the workbook, it’s immediately viewable online, but you’re going to want to build it into a proper dashboard with menus for the different screens and some element of user control. For that you use the dashboard builder and here’s where things get a little less smooth. The builder is fine and it works, but it’s not friendly. Creating filters and dashboard screens is very manual, pretty clunky and to be honest a bit of a pain. There’s a weird distinction between different ways of publishing and refreshing items on the server and you’re never sure which one your latest change needs, so you end up always doing all of them.

There’s also the potential to get into a hell of a mess with the builder if you don’t keep your files and working practices organised. One (local) dashboard builder file can contain the elements of several different online dashboards. Get two or three people building and uploading without remembering to refresh from the server and you can find your painstakingly created view has disappeared when somebody else ran an upload. It’s not that hard to avoid but it shouldn’t be possible at all.

For an overall idea of feel, have a crack at the skydrive. They’re not too similar in function, but they both have that hint of “we’re Microsoft, we know what’s best and if you do it wrong or can’t find a setting, then it’s user error”. They're also both slow. You know what I mean - Hotmail’s the same.

User interaction comes from setting up drop-downs on the dashboard and linking them to named cells in your workbook. You populate what’s in the drop-down from a workbook or database connection.

One major - and I mean really major – limitation is that you can’t make the drop-downs drill. So if you’ve got a list of countries in one and a list of cities in another, you can’t only show the UK cities if the viewer selects UK. This one issue alone kills Excel Services for serious dashboard work for me. As soon as you’ve got any amount of data, you have to be able to drill the options or there will be hundreds of items in a list. There are workarounds involving tree based lists but they’re not very satisfactory.

While we’re doing the negatives, here are a few more:

• If you’ve got something that’s not allowed for publishing in your workbook, like a cell with a comment in it, you get no help at all about what’s wrong and have to go over everything with a fine toothed comb until you work it out. One of mine had broken links in a hidden named range (it had disappeared out of the names list – a ‘known issue’ apparently…) and that took ages to solve. There’s a download tool from MS to help, but frankly, it’s rubbish.
  
  
• Viewers can’t download data from the dashboard unless they’ve got Excel 2007 or better. If you’ve got users on Excel 2003, my recommendation is that you don’t touch Excel Services with a bargepole. Getting the data out for viewers who are using 2003 becomes a complete pain.
  

Would I buy it? Well we did at EMI and it was useful. Talk to your IT people and if they’ve invested in the top-end version of SharePoint then you may find you’ve already got it.
For small dashboard tasks, it’s not bad and for my current needs, if we already had the licences then I’d use it to help with a lot of them.

Unfortunately though, we don’t already have the licences and it will cost somewhere around £8k to get them. With the other tools on the market (some of which we’ll come to) that’s too much when it only looks ok and is only ok to use.

Microsoft are being left behind in dashboarding and visualisation. In Excel, they’ve got what is still everybody’s first-stop BI tool, but they just haven’t made creation of an online Excel dashbard easy enough and they haven’t implemented the macros that make Excel dashboards so flexible.

In short, Excel Services is ok, but only ok. If you’ve got it already, give it a try and if you only want to publish small dashboards, have a look at it. For everyone else though, you can do better.

Dashboard & visualisation software: The criteria

Edit: If you're looking for the conclusions of this post, please go here.

I'm coming to the end of an evaluation process for dashboarding software and it's been an interesting journey. Some pieces of software I've loved, others not so much and a theme running through the whole experience has been a serious difficulty with finding information on the pros, cons and pricing of each platform.

I had a rant a while back about how dashboarding software is sold. Coming out of that post, some discussion on linkedin and some very helpful people who got in touch, I've finally got a reasonable picture of what's out there and roughly what the different options cost. Capturing that sort of knowledge is what blog posts are for, right?


Over a few posts, I'll line up the candidates, with reviews and the path to the platform that's come out on top. First though, with so many platforms out there, what was I looking for?

I do econometrics (marketing return on investment analysis) at Brilliant Media, a mid-sized, independent media agency with three offices and 100+ employees. We've got a lot of basic marketing response data - click through rates and monthly spends, that sort of thing - plus more advanced modelling and forecasts coming out of the econometrics. There aren't vast volumes of data (a few million rows max) and it needs to be visualised clearly for our clients, who are marketing directors and managers who may or may not be comfortable with maths, stats and charts. I'm not expecting a dashboard to work miracles, but to be functional, effective and look good.

These criteria are probably fairly typical for a smaller dashboard project where the output will be viewed internally and sold to clients.

1. Off the shelf, not custom.
   I'm not interested in starting a major IT project here, I want something that works out of the box as quickly as possible and can be built and maintained by (SQL capable) analysts with minimal programmer involvement.
   
2. Low cost.
   Always! Our clients aren't huge and dashboard server solutions for ftse 100 companies really aren't happening. We need to get off the ground for a few grand, max.
   
3. Web facing (or at least has the potential to be.)
   We're mailing spreadsheets at the moment and need to be able to step to web-based delivery.
   
4. Better looking than I can do myself in Excel.Which actually sets the bar quite high - Excel's not bad if you put some effort in.
   
5. At least as flexible as Excel.
   
6. Easier than building and then mailing spreadsheets to clients.
   
7. Data stored on our own servers
   There are some quite tempting cloud based solutions emerging, but we need the capability to include confidential data, so they're out from the start.
   

Finally, a word on OEM licences. OEM stands for Original Equipment Manufacturer and it's a world of confusion in terms of pricing. If you want to build a dashboard that you're going to sell to somebody else then you need an OEM licence, if you're building just for your own internal business, then you don't.

The trouble is, OEM licence prices (usually) depend on the scale of your deployment. If you want hundreds of people to be able to log into your dashboard, the software will cost a lot more than if you only want ten, even though it's the same software running on the same server in the same building. That's annoying and it can become absolutely infuriating when you don't know yet how big the market is for your new product.

If you want a giggle, ask your IT department to find out how much Excel Services will cost to deploy externally on your Sharepoint extranet. You'd think it being a recognised tool from a big company like Microsoft and the fact you've already got Excel would make it simple wouldn't you? Give it a try.

So a last criteria...

Transparent pricing. If I still don't know what the year one cost for the software will be after a ten minute conversation with a salesman then, forget it.

Excel services will be up first. Check back for more!

You wouldn't sell any other product like this

I've got a simple problem that lots of other people in marketing analysis have got. It's 2011 and we're still mailing spreadsheet reports to clients when they could be logging onto a web page and viewing the report there. No versioning problems, auto-updating, access on-demand, flexible and less intimidating than a spreadsheet for non-technical users. I want a piece of dashboarding and data visualisation software, but which one?

Lots of software vendors have worked out that there's a market and made a product. There are loads of solutions, from huge database engines powered by SAP that could handle a global airline's booking system, down to Excel Services from Microsoft that gives you a quick way to publish a spreadsheet to Sharepoint.

They all have one thing in common though. Nobody will tell you what any of them cost and you'll be very, very lucky to get a demo copy without talking to a sales rep.


I've been talking to a Spotfire rep this morning. When I said I was looking at making a small number of dashboards that external clients could access and I'd like a ball-park figure for how much the licences might cost, he giggled and said it wasn't that simple. How can it not be that simple? I want to know if I can afford your software - one dashboard builder and a server licence - yes or no?

Let's draw a quick analogy. You go into a furniture shop to look for a new sofa - you've heard they might have nice sofas but that's all you know.

When you get into the shop, there are loads of posters and pamphlets about how great the furniture is, but no prices and no products on the shop floor. A salesman says he can take you into another part of the shop where if you're really lucky, you can actually try sitting on a sofa, but only if you can tell him based on the leaflets exactly which one you'd like to see. He'll also need your name, address, phone number, email address and a short explanation of why you're interested in his sofas before he'll let you see any.

A little bit annoyed, you point out a picture and ask how much that one is, so you can know if it's in your price range before you try it.

"Well that depends," says the salesman. "How many people will sit on it?"

You don't know for sure and can't see why it's important, but hazard a guess at you and your family definitely, plus some friends when they come to visit.

Now he wants to know if more than one person will want to sit on it at the same time and whether your friends all live with each other, because if they visit you from different houses, then it will cost more.

You slap the salesman as he adds that you can't actually buy a sofa, you can only licence one. You don't get anything new after the sofa arrives, but that's just the way it works - next year you have to pay to keep sitting on it.

Oh and by the way, if some visiting relatves stop by unexpectedly this year and want to sit down, you'll owe the shop more money.

Analogy stretched far enough I think, but that's what this morning at work's been like. I don't mind bulk discounts and negotiating prices, but this is ridiculous.

I'm happy to recommend any of the multiple bits of software that will do a job for us, if I can find a developer who will give me a time-limited trial copy without jumping through sales hoops and a straight price for a software licence. No wonder so many people just hire a few few Adobe Flex developers and do it themselves - at least you know what you'll have to pay out in wages and anything they build is yours forever.

Since SAP bought Xcelsius and ruined it, there's a huge gap in the market for a cheap, simple way to publish Excel dashboards to the web. You used to be able to buy Xcelsius with a straight price per copy and do whatever you liked with it. Surely that's not too much to ask.

Take the long view. Except in a crisis.

There are many similarities between marketing mix modelling (econometrics) and weather forecasting. They both use data on the past to try to predict the future - though I'd be the first to admit that weather forecasting is harder - and they both struggle badly when you knock them out of their comfort zone. Neither forecasts well when they're in a situation that they haven't seen before.

Weather forecasts in the UK are, to put it kindly, struggling this week. My lovely new HTC phone brings me weather alerts and on Saturday it went from forecasting not really any snow, to heavy snow, to very heavy snow in the space of a few hours. Half an hour after it made up its mind that we were going to get some proper weather, the heavens opened and London went very white and very pretty in the space of about two hours.


There's a marketing angle coming, I promise.

I'd normally argue against taking a day-by-day view of marketing ROI. It's unproductive and it hides the big picture. Retailers especially have an obsession with yesterday's takings which is brilliant for stock control but terrible for working out whether your ad campaign is effective. Advertising has at least medium term effects and you're really looking to build a brand over the long term, so asking your analysts to spend their day working out why Tuesday was 0.5% down year-on-year is distracting at best.

This short term obsession ties in nicely with social media reporting. We're sure we need to track it - how many new Facebook friends, twitter mentions and positive blog posts - but nobody really seems to be able to explain why. If I've got an up to the minute tweet dashboard, what do I do with it? What do I change in my business?

The weather's just given us an example. Forecasts are unreliable and we're out of our comfort zone. What the hell is going on? Is the snow headed our way?

#UKSnow knows.


#UKSnow takes a hashtag on Twitter, a postcode and a report of how heavy the snow is, then maps all the tweets for you. Fabulous! It's doing better today than the Met Office rainfall radar; it updates faster and is reporting snow where the Met Office isn't (in places where it's definitely snowing!)

Most of the time the long view is the best one, but just occasionally, when our models get knocked out of their comfort zone there's a real use for social media dashboards. They can bring us information far faster and more accurately than a model ever could. Provided every day isn't nominated a crisis day, social media tracking is a powerful analysis tool.

What happened to instructions?

Have you noticed how nothing needs intructions any more? When you turn on a new mobile phone for the first time, it gives you a little tutorial on how to use everything and I can't think of a successful web application that needs an instruction manual. Facebook or Google Mail with instructions? The idea is ridiculous, but that doesn't mean you can't do a hell of a lot with them.

I'm an occasionally keen photographer, but as much as I like my new compact, it's got features that you can only understand if you read the instructions. They're not intuitive and so I don't use them. Full auto works, so full auto it is.


Media software is being left behind in this new world. Our agency's timesheet system is so complicated that you need to be shown how to use it, which is a farce (thanks again DDS.) If you've ever tried to get TV ratings from Nielsen Addynamix or MMS, then you'll know that working it out for yourself is hard. Why? It's not like you're trying to do anything difficult. Pick an ad or a programme; pick an audience; show me the ratings. Not difficult.

We don't need instructions any more because we've learned the rules. I know I can right click things to get options, so I try it. I know what save buttons do. When we build software and dashboards, it's essential to build on that existing knowledge that users have - to put menus where the user expects to find them. This means an intuitive understanding of how people work and then a lot of user testing.

Where I don't know the rules, I need instructions. A mechanic can service my car with just a toolbox but I need a toolbox and a Haynes manual and a lot more time. And then a mechanic to fix the mess I've made, because I've got no experience. Making up new rules so that you need a manual is a serious undertaking - is what you're building important enough to junk all that built up experience and make users learn a new way of working? Really?

I'm applying a new principle to the dashboards, spreadsheets and tools that we build for clients: The ten minute demo. If you can't work out for yourself how to use everything after a ten minute demo, then it's too complicated. You might learn a lot of new things after that demo, but it should be straightforward enough that you can work them out for yourself with a minumum of effort.

As a rule, if it needs instructions or training, then it's too complicated.