Fighting IT? Prototype an Executive Dashboard in Excel

by Jorge

I am a moderately advanced Excel user. This means “a dangerous person” for the IT department, but I like this daily fight, and Excel dashboards are among my preferred weapons. Let’s see how they can be used.

Excel is the best tool for executive dashboard prototyping, because of its flexibility and development costs. Creating a fully functional prototype is not hard and it should be available for user feedback in a matter days. So, make sure that, every time you spot a dashboard project, a prototype in Excel is included.

Since most business intelligence applications are notorious for their lack of basic chart formatting options, it shouldn’t be hard for you to create a simply set of charts that the IT is unable to implement. If needed, use some advanced Excel charting techniques (including dummy series), but make sure they add real value to the user experience. Interactive features like visual what-if analysis are always cool and the users love them.

When presenting your project, do your best to convince your audience that you are technology-agnostic and all you care about is to create the best answer to users needs.

IT will try to change your project, naturally. Try to avoid the “security bomb” (their favorite). You know how poor their expensive BI toys are, and you should know what they can and can’t do with them. Minor concessions can earn you some points. When they tell you they can’t implement your core ideas be prepared to fake genuine surprise, compare costs (again) and emphatically say that their options clearly don’t meet the organization’s needs.

Pissing off the IT department is one of the most enjoyable games in corporate life, but be a gentleman and don’t make them look stupid. They don’t usually have a good sense of humour and take their quest to conquer the world very seriously. If you really want to implement the dashboard, don’t make it an island if you can avoid it (connect it to the tables in the IT infrastructure, instead of copy/pasting data). 

Seriously: Excel is a great tool for dashboard prototyping. You can easily create multiple alternative user interfaces, get feedback from users or find design flaws. The end result should be much better than trying to capture some ill-defined requirements and send them to the IT, where user interface design usually ranks very low in their priorities list.

{ 16 comments }

Can You See the Chart Under the Notes?

by Jorge

Annotating your chart helps your audience to understand the reasons behind some patterns or outliers. But, please, please, don’t bury the data under boxes and arrows and busy grid lines, like this one on the right does (from WTRG Economics).

How can you improve a chart like this? First of all, the series must be visible… :) Don’t create a competition for the readers’ attention. Notes should be in the background, like the axis, labels and other chart objects.

A note is not a marker. Add a note to explain a specific behaviour (“this happened because…”). If you have a time series, create a timeline outside the chart and add markers for relevant contextual events (there are several other design options).

It is easy to come up with something cleaner:

There was enough notes about quotas to create a new chart, with new insights. All the other notes are there, but they only murmur. I don’t feel confortable with using two axis, but that’s a topic for another post.

By the way: there is (at least) a perceptual error in my chart. Can you find it?

{ 1 comment }

Is Your Chart Really Answering Your Question?

by Jorge

We are so busy creating sexy charts to illustrate some random data that we often forget to check if our chart really answers the question. Heck, most of the times we don’t even have one. Chart first, ask questions later.

One of the major differences between tables and charts is this: a tables says “here is your data, now go find the answers (they must be here, somewhere)”, while a good chart says “here is your answer”.

The more precise and clear our question is, the easier is to select the right data and the right chart. Let me give you a recent example. Robert Kosara, at EagerEyes, discusses the “swing states”. Several readers contributed with great alternative displays but, as I commented, there is a fundamental issue: if we want to see “the swing” that’s what should be displayed, not the election outcomes. This:

 

 

is different from this:

 

 

In the first chart, we can see that the Republican candidate won in Alaska in 1968. In the second chart, we know that, in 1968, in Alaska, there was a different outcome – a “swing”.

Sure we can infer the swing from the first chart, and our answer about the “swing states” is there somewhere, but only the second chart can provide a clear and concise answer.

It is prudent to keep all the data (who knows what the future will bring, right?), but we should always be aware of our loss aversion tendency, and make sure that our chart is displaying what it was designed for. Edit your chart without mercy, and let redundant or plainly useless data go. That’s the only way to highlight the patterns we are looking for.

Now, you may want to reevaluate your question to allow for a broader answer. That’s ok, but do it carefully. Add detail without breaking the pattern. For instance, we may want to know more about the direction of the swing:

 

 

Bottom line, make sure that what your chart says is aligned with what you asked. If you can use your question in the chart title that’s a good sign that you are on the right track.

{ 2 comments }

The Inner Beauty of Business Charts

by Jorge

Naked womanCan a picture of a nude person improve your decision-making processes? (Please don’t say “yeaaaaah”.) Probably not, but if you need a good attention grabber a picture of a naked body is your best bet. Make sure you’ll add one to your next sales report.

Because, if you are using those glossy 3D pie charts from Crystal Xcelsius (or Dundas, or…), you are applying the same principle, safe-for-work version. Your underlying message to your audience is “you are so dumb that you don’t even understand a simple chart with a clear message. I have to use charts that obfuscate the message, but they grab your attention and that’s all that matters. Let me take my shirt off too.”

I’m a business analyst I usually try to create charts that can support the decision-making process. I am not a graphic designer, trying to illustrate a story and get reader’s attention.

When you are in a corporate environment you can enjoy the attention of your audience (the organization is paying for it…). Also, information is shared among people with similar professional profile that at least know what the basic concepts are.

On the contrary, in a magazine, your readers don’t know or may not care about your subject. How do you grab their attention? Your best option is to add a photo of a naked male/female. Can you justifiably use it to illustrate the story? Do it. You don’t? OK, try other attention-grabber devices, like a nice, glossy pie chart (not as satisfying, though).

These are different needs, but we, the so called “visualization experts” often fail to aknowledge that.

Eye-Catching Charts vs. Decision-Support Charts

Eye-catching charts are used to get the reader’s attention by providing some sort for light entertainment. Their primary focus is on the format. They use many colours and and large textured surfaces. Because of that, their data density is low and context is almost absent. A 3D pie chart is the typical eye-catching chart.

Decision-support charts  focus on the data and should be “invisible” (the audience sees the patterns, not the chart). There are no textured surfaces and colors are used to highlight specific details. The display real estate can be filed with context data, maximizing data density. The typical decision-support chart is, obviously, the scatterplot.

Charts for Analysis and Charts for Communication – Not Anymore?

This is the traditional split. After the analysis stage, the analyst should prepare his/her findings for the communication stage. But vendors like Microsoft and Business Objects have been short-circuiting this process, selling the idea that all you need is form, not content, and whatever stage you are in, you must have a nicely textured 3D chart.

These charts are sold as “professional-looking” and let’s accept that for a moment. They are professional-looking from a graphic designer perspective, but they are completely useless in a corporate environement where you have masssive amounts of data to deal with. I’m sorry to say, but the more textured charts you have the dumber you look.

Pin-Up Charts Don’t Belong Here

I don’t really care about pin-up charts (charts that the media pin up on their pages…).  Sometimes they are amusing (not sexy, unfortunately) but they just don’t belong in a corporate environment. If you need attention, make better use of your data to find its inner beauty or use a photo of a proper pin-up.

{ 6 comments }

Data Visualization: Who Needs a Reality Check?

by Jorge

According to Stephen Few, the founders of Tableau Software made some assumptions about visual analytics’ adoption that we can summarize in a single sentence: analysts want to find hidden insights in large and complex data sets using new visual paradigms. Later on, they discovered that these assumptions were somewhat flawed, and that what people really want is to save time in their daily routine when analyzing small and simple data sets, using familiar formats. Reality check, anyone?

We all make some wrong and costly assumptions. I wrote a blog on data visualization in Portuguese for about a year and then I had to give up, because no one seemed to care. I’m selling a tutorial on how to create Excel dashboards that I am proud of, but I should have started with a simpler version that delivers similar results (I’m working on that, by the way…).

Many of these assumptions are powered by what Chip and Dan Heath in Made to Stick call the “curse of knowledge” (“the better we get at generating great ideas—new insights and novel solutions—in our field of expertise, the more unnatural it becomes for us to communicate those ideas clearly”). Our wishful thinking makes us to believe that the knowledge gap is narrower than it really is, and some basic notions that we take for granted are not.

I’d love to write a blog on data visualization using higher-end tools like Tableau or Spotfire, but you can’t tell people “ditch Excel, use these great tools instead”. They have their (growing) market, but an overwhelming  proportion of business charts are made in Excel because that’s the only tool people have access to. Excel is good enough to teach sound visualization principles, so visualization experts should start by saying “you can do it in Excel; here is how”. At some point the newly acquired knowledge would make people move up, if needed. In information visualization, we have the (graphic literacy-wise) rich and the poor. Now we need a solid middle class. Accessible learning tools is one of the answers.

(This is what I am trying to do with pie charts: instead of banning them, I’m trying to show how to create acceptable pie charts. At some point people will realize that they will need something better. I may be wrong, but the other options don’t seem to be working, either.)

If we fail to communicate this simple message (“you can do it in Excel; here is how”) do you know what we’ll get? A new Dundas/Crystal Xcelsius user.

{ 7 comments }

14 Misconceptions About Charts and Graphs

by Jorge

Misconception #1: A Better Chart Starts With… the Chart

Wrong. It starts by asking yourself if you really need one. Perhaps a statistical measure of some sort is good enough, perhaps you should use a table. If your job is to find patterns in a data set and build shared knowledge about it, what really matters is how efficiently the message is sent, and how efficiently it is received by the audience (two different things).

Misconception #2: You Should Master the (Technological) Tools of the Trade

No, you don’t. Just because you know how to create a chart in Excel it doesn’t mean that you know how to create a chart. If you use Microsoft Excel as your charting software then yes, you should learn more Excel (to spend more time with the kids). But you must go beyond technology, or else you end up creating some very stupid charts. Please note that a vast majority of Excel training courses will not teach you what it should (best practices). It will only tell you how to make “cool” graphs, like a 3D exploded pie chart.

Misconception #3: Defaults are good enough

They aren’t. Each chart must be tailored to the specific data set, audience and message. For instance, try to create a graph that clearly displays a large number of series and you’ll fail if you use the defaults (but can do it with clever color coding). And if you use recognizable defaults, like the Excel 2003 charts, you’ll look very, very, lazy (at best).

Misconception #4: Vendors obviously implement the very best templates

(I’ve heard this one recently, and I found it so incredibly naive that I had to write about it.) They don’t. About 90% of the Excel 2003 chart gallery is junk, and you must heavily reformat the remaining 10% to get something useful. Select other tools, like Crystal Xcelsius and the scenario is even worse. And I am unable to create in Cognos something that remotely resembles a chart (people tell me that version 8.4 is a little better).

Misconception #5: Better charts are just “prettier” charts

I hear this all the time. A good chart may look “prettier”, but that’s just an unintended consequence of a design that communicates better. In information visualization, prettiness must be a by-product of function. The very concept of a “better communicator” is sometimes difficult to comprehend, and trying to explain it is a waste of time, because people need to see it in action. You must take the user by the hand and guide him/her. You must force comparisons: “what can you learn about x using this chart?” and “what can you learn about x using that chart?” “how long did it take you to learn x using this and using that?”.

Misconception #6: It’s All About the Wow Factor

It is not. Many marketers and graphic designers fail to understand this. Marketers are hopeless in their relentless search for the wow factor and the eye-catching, “professional-looking” graphs, and graphic designers should know better, but they prefer to sacrifice data on the altar of Beauty (form is everything, data is a nuisance).

The dominant view among visualization experts (namely Tufte and Few) is that “form follows function“: every ornament in a graph should be eliminated, every object must serve a clear purpose, efficiency should be maximized (labeling series instead of using a legend, for instance). Given the extremely low graphic literacy levels among the general population, this may not always be the best approach.

Misconception #7: A good chart displays the actual values

No. If you label each data point you get a useless table over a useless chart. Labels are not only a distraction but often actually hide patterns in the data. Short labels and annotations can, and should, be used to identify or explain outliers or other interesting data points and circumstances. If your audience expects to see the underlying data then add a link to the table.

Misconception #8: Good Charts Should Be Read at a Glance

No, they don’t. The more complex, the longer it takes. It really doesn’t matter if it takes a second or an hour. What matter is how efficiently the graph  communicates. If a chart takes for ever to be read look for bottlenecks: the series are not easily identifiable, patterns are hidden, demands on the working memory are high, etc.

Misconception #9: The More Detail the Better

What we see as detail can be seen by someone else as clutter. Clutter is the natural child of loss aversion and is is very difficult to remove. If you have 12 competitors your audience will want to see the market share for each of them, even if it doesn’t make any sense. Tufte says “to clarify, add detail”, and yes, 12 competitors in a line chart can be made clear and useful, but you must know how to categorize them and provide a framework to help the user (you can use a large number of categories in a pie chart, for instance).

Misconception #10: It’s All About Selling Your Point, No Nuances

In The three laws of great graphs Seth Godin says that “there is no room for nuance [in a presentation]” and your charts should reflect that. Maybe it is just me, but I hate it when I am not allowed to draw my own conclusions because the data made available by the presenter is too biased towards his/her own points of view. Depending on the situation, a clear path that is supported by a lot of details is much better than a yes/no pie chart.

Misconception #11: You Have to Have Color, Lots of Color

Wrong. Color is a very difficult subject. Large surfaces of primary colors like we often see in presentations should be avoided because they are hard on the eyes and, because everything stands out, nothing stand out. A good option is to use grays for non-data elements like grid lines, and pale colors for color-coding. As a rule of thumb, color should always carry some meaning. Use primary colors to highlight a data point or some other small detail.

Misconception #12: A Single Chart is Enough

It is not. We live in an increasingly complex world, and traditional charts are very simple tools. While we wait for a new set of charts to be invented, we can use interaction (see below) and multiple charts to create a richer picture. That’s why scatter plot matrices, small multiples or trellis displays, and specially those multiple variations of executive dashboards are much more powerful than a simple chart.

Misconception #13: Charts Are Interchangeable

They aren’t. You can use a column chart or a line chart to display a time series, but while a line chart performs better than a column chart when reading trends, it is easier to compare data points using a column chart. Most visualization experts will tell you that you should use a bar chart instead of a pie chart (also because it is easier to compare data points), but a pie chart gives you the perception of a whole that is absent in a bar chart. Every graph has its own strengths, and you should select the one that suits your needs.

Misconception #14: Create It and Forget It

Don’t. Making sense of your data is a process of exploration and discovery. A pattern in a subset may be hidden by a noisy background. Different measures may lead to more complex insights. Creating a chart that the user can interact with should always be your primary goal. Unfortunately, that’s beyond the skills of an intermediate Excel user (if you what to learn about interactive charts my Excel dashboards may be a good starting point).

*

This post lists 14 widespread misconceptions about charts, but probably is a very incomplete list and you may not agree with all of them. What misconceptions would you add/remove?

*

[Update: Jon has been writing extensively about Excel 2003 and Excel 2007 (by the way, it's a great resource that helps us to see through the marketing noise). I said in the comments below that I prefer to use Excel 2007 charts to post images in this blog. He doesn't agree and he tries to prove in his last post that charts in Excel 2003 are actually better. He uses good examples to prove his point but I still believe that this (Excel 2007):

looks better than this (Excel 2003):

Yes, probably there is an "overaggressive anti-aliasing", but the line in Excel 2003 is too "crispy" for my taste. Again, it is just a matter of creating images for a blog, not exactly for serious work...]

{ 22 comments }

How Many Computer Scientists Does It Take to Screw Up a Pie Chart?

by Jorge

You can add  silly 3D effects to a pie chart, you can explode all the slices, you can compare multiple pie charts, you can use a legend instead of labeling the slices directly. This will probably render your graph useless, and make you look kind of dumb, but it is not the end of the world-as-we-know-it. But when making a pie chart there is something that you should never ever do, a capital sin that will make you burn in the hell of information visualization: using more than one variable in a single graph.

Well, since we are witnessing the end of the world-as-we-know-it, computer scientists at the University of Utah decided to give a little push, visualization-wise. They are designing a computer application “they hope eventually will allow news reporters and citizens to easily, interactively and visually [analyze] election results, political opinion polls or other surveys”. They boldly state that they “have developed new techniques for exposing complex relationships that are not obvious by usual methods of statistical analysis” (press release). And what are those new techniques? A doughnut chart:

The outer ring labels the series and the inner ring displays the data. Apparently you may add as many series as you wish and you can filter the results by socio-demographic characteristics. There is a video demonstration here [via].

This is the kind of joke that I would expect to be related to April Fool’s Day, but they seem to be serious about it. No one told them that showing part-of-a-whole is one of the few strenghts of circular charts, that when people see 52,7% they see a pie cut in half, not a quarter, that “whole” mean 100%, not 200% or 300%.

Regular readers know that I rarely utter such harsh comments on visualization ideas and applications (I even tried to create a dashboard using Crystal Xcelsius), but this is the stupidest idea of the year. They should know better (here are some tips).

By the way, I found this through a post by Sarah Perez at ReadWriteWeb. She writes: “unfortunately, the poll-analysis software isn’t quite ready for prime time. What a tease!” Fortunately, it is not! And judging from other posts, they could use an information visualization consultant. 

Well, perhaps I’m missing something. Am I?

{ 9 comments }

Spending Time Deciphering Time Utilization Charts

by Jorge

I really dislike stacked bar charts… Let’s see a new bad example.

Steve Rubel shares with us how he spent his time online over the last two weeks. He uses the Firefox add-on PageAddict to monitor the time. He writes:

As you can see almost all of my time online is work related. Still I can see that I need to shrink my social network usage a little bit and increase my time with documents, web applications while also keeping RSS contained. I also need to go through the undefined section to see if there are big groups of sites that can be tagged.

I would say that it is hard to see what Steve Rubel wants us to see. It is not his fault, of course, he is just pasting a chart from the application. I do see something interesting: since he spends “the vast majority of [his] computing time” using Firefox (let’s say 80%) he’s using his computer around 4.5 hours a day only. That’s nice… :)

If I wanted to visually track my time online (I should…) these are some of the options I’d like to have:

  • Color-code work/non-work related categories;
  • Label the x axis with dates, not “days ago”;
  • Remove non-working days;
  • Use small-multiples to track each category;
  • Use weeks instead of days;
  • Annotate outliers;
  • Show planned vs. actual time spending;
  • Minimize the “undefined” category;

I’m installing the add-on. Hope I can have an interesting dataset to share by the end of October.

Do you use these tools? Do you like their reporting functionalities?

{ 6 comments }

Information Visualization Week in Review: September 28

by Jorge

{ 2 comments }

How to Make Better Pie Charts with On-Demand Details

by Jorge

This article goes much against conventional wisdom about pie charts (and doughnut charts) by answering these two simple questions:

  • Can we use a large number of categories in pie charts? (Yes, we can.)
  • Can we make a productive use of the apparently useless doughnut chart? (Yes, we can.)

Disclaimer (Sort of…)

Let me start by declaring this: I believe that the analysis of simple proportions is, by its very nature, very limited. It only scratches the surface of the data and it is useless for serious, decision-making processes.

A circular chart is poor because the underlying message is poor. If you can run a business using pie graphs to make sense of your data please let me know what market are you in, because I want to be there too (well, not really…).

Pie chart belong to the media and to some simple presentations. Leave them there. And don’t make the charts you see in the media your role model.

The part-of-whole issue

That said, one must recognize that proportions are so pervasive and hard-wired into our brain that escaping them is almost impossible.

A circular chart conveys perfectly the idea of part-of-whole relationship. You can’t use a bar chart to show this relationship because the whole just isn’t there! Yes, you can use percentage scales, yes you can say it in the title, but it isn’t the same thing, is it?

As I wrote in my previous post on loss aversion, each chart answers a question from a different perspective. Charts are not interchangeable.

Often pie charts are used just because they may look better (this is, of course, in the eyes of the beholder) but what the user really wants/needs to know would be better answered by a bar chart. This is a problem of graphic literacy and information management. It has nothing to do with the intrinsic qualities of pie charts.

The limit of 4 to 6 categories in pie charts

There is a widespread believe that you should not use more than four to six categories in a pie chart.

That’s is wrong or, at the very least, very incomplete.

In fact, you can use as many categories as you want, and still get meaningful insights from the chart. Problem is, you must know what to do with your data (graphic literacy and information management, again), and a large number of bad charts come from this simple fact: people don’t know what to do. Garbage in, garbage out.

“The Secret Strenght of Pies”

Here comes the fun part. In an article published back in 1991 by Ian Spence and Stephan Lewandowsky, titled “Displaying Proportions and Percentages” the authors write:

“the pie chart outperforms the bar chart for complicated comparisons, suggesting that the perceptual addition and comparison of components is inherently easier with the pie chart than the bar chart.” (emphasis added)

(By the way, the authors also say that this advantage will be lost if you “explode” the slices.)

Stephen Few, in his “Save the Pies for Dessert“, cites this article and writes about “the secret strength of pies”:

It is not difficult to believe that it is somewhat easier to sum the areas of slices in a pie than it is to imagine the combined heights of bars stacked on one another.(…) Regardless, the fact remains that a comparison of two sets of summed parts is rare in the real world. But, by all means, should you ever need to display data for this purpose, a pie chart would serve you well.

Please note that Stephen Few, in his highly regarded book “Show me the Numbers” says:

“I don’t use pie charts, and I strongly recommend that you abandon them as well.”

Few acknowledges that pie charts “could serve you well” in a very limited set of circumstances (“a comparison of two sets of summed parts is rare in the real world”).

Is it really rare? It may be, but that’s because people don’t know what to do with their data (again). Let’s see.

You have 10 or even 20 categories and you want to use them all (your loss aversion tendency?). Because 20 ungroupable categories are rare in the real world, you should be able to visually group them, using a color (hue) for each group and a different saturation for each category. By doing this, you are adding layers of detail, and the reader will be able to select the level of detail that suits his/her needs. This works best when using an interactive chart because you don’t have to label everything (just use your mouse to identify on-demand the more relevant detail categories) but even a static chart can be used (in this case, label only the relevant details).

The Consumer Expenditure Chart

I used this methodology to design the consumer expenditure chart above, with living expenditure (on the right) and discretionary expenditure (on the left).  As you can see, living expenditure accounts for almost 60% of the total. That’s something you can’t easily see with a bar chart.

Then, there is a second level of detail, where you have categories like Housing (more than half of living expenditure) or Transportation. And finally, you could use your mouse to identify those detailed categories in the outer gray ring.

I’ve added some arcs to compare the profile of total consumer units to consumer units with five or more persons. Each arc always starts at the same degree of the corresponding slice. Different proportions lead to gaps or overlaps. Please note that this is not a core feature of this chart. Just wanted to play a little with comparisons (an obvious issue: since the first arcs are closer to the center, a gap between them is different than a gap between the last arcs).

The Secret Strength of Doughnut Charts

As we saw above, pie charts are better than bar charts when comparing proportions. But, as soon as you add a second pie chart you are trying to compare proportion A1 with proportion A2, not proportions A and B of the same pie. There is a shift in the analysis and the pies become useless (use bar charts instead).

Just because you can merge both pie charts in a single doughnut chart it doesn’t mean that you gain efficiency, because the essential problems remain in place.

For many, a doughnut chart is a bad mutation of a bad chart. But if, just if, two bad’s become on good? Could a doughnut, if correctly use, become a kind of pie chart on steroids?

Let me emphasize this: never use a doughnut chart to compare series. I don’t, and I strongly recommend that you should avoid it as well… Always use a doughnut chart to add detail to a series. That’s the secret strength of doughnut charts.

And please, please, could someone write an article on doughnut charts for the English Wikipedia?

I made this chart in Excel

In case you are wondering, you can make the Consumer Expenditure chart in Excel, 2003 or 2007. Instead of the default theme colors, I used some of the colors that will be available in Chart Tamer (thanks, Andreas!).

Conclusion

Pie charts do not deserve their bad reputation. They seem to be more efficient than bar charts in some very specific tasks, like  comparing combined proportions. We should take advantage of that by adding multiple levels of detail. We shouldn’t be afraid of using a large number of categories, provided that those levels of detail are clear and meaningful.

The doughnut chart is the most misunderstood of our chart toolbox. It is seen as completely useless because two series should not be compared using circular charts, but that’s not what doughnut charts should be used for. They should be used to extend the power of pie charts, managing efficiently the level of detail that we need to add to create more insightful charts.

Is this a good way to use pie and doughnut charts? Please share your thought in the comments.

[Update: If you want to know how to create this chart (with a bonus hole-remover...) Jon has a detailed explanation here.]

{ 12 comments }

← Previous Entries

Next Entries →