Posts tagged as:

Data

Stephen Few, Data Visualization, Eye Candy and the Pie

by Jorge

Stephen Few left a comment in my post “Is Data Visualization Useful? You’ll Have to Prove it“. We all have much to learn with Steve, so instead of leaving the discussion buried in an old post, I thought it would be interesting to make it more visible. Please read the comment then come here and join the discussion. Here is my answer.

Steve, sorry if I sound provocative, that’s not my intention. You are the leading expert in data visualization for business, you are doing a remarkable work with your books, with your blog, with your forum, with your patience to answer posts like mine. I have to be  thankful for that. And I do agree with 95% of what you write. But you don’t want to be surrounded by people who fully agree with you, do you?

The Effectiveness of Data Visualization

You say “the effectiveness of data visualization is well established by a large body of empirical evidence”. I want to believe that too. However in this study Jarvenpaa writes:

“Graphical charts are generally thought to be a superior reporting technique compared to more traditional tabular representations in organizational decision making. The experimental literature, however, demonstrates only partial support for this hypothesis.”

And J.-A. Mayer adds:

“This study refutes the general superiority of visual information in improving the decision quality (‘naive superiority hypothesis’). The choice and design of visual presentation is determined by information structure, decision environment, the decision-maker and the task decision. (…) The successful use of visual information depends substantially on its acceptance by the manager and the environment.”

What do these authors tell us? First, we cannot be 100% sure about the effectiveness of data visualization. Second, there are many other variables at play. And third, managers must accept it. This is a critical factor. Managers love impression management, and making a good impression using the dreaded “professional-looking charts” is the path of least resistance.

Data Visualization Success Stories

I have no doubts that you could share with us many success stories. When I write about an “admission of impotence” I am not questioning your ability to create/lead/mentor successful data visualization projects. But if you want to use those projects to inspire the average person I think you’ll fail most of the time, unfortunately.

Let me tell you how the layman looks like in my part of the world. He makes charts like this:

He believes that a 3D pie chart “looks more precise” and he doesn’t know that Excel chart defaults can be changed (more advanced laymen are able to switch to more “impactful” colors like reds, yellows and bright greens). In my part of the world, a layman doesn’t even know what “data visualization” is about (and they don’t even care). (Here are some more profiles.)

If you are preaching to the choir your conversion rate may be high. But the layman is not easily impressed. You must convert one at a time, and that’s something many of us can’t afford. Can you? He’ll keep making those pie charts because that’s what his manager requires him to do, he doesn’t know better, he’s lazy or you fail to convince him of a causality effect between better charts and better results.

The Layman Must Like Your Charts

In a business environment, charts don’t have to be memorable, only results do. But if you want to change behaviors, your audience must like the new behavior and accept the unavoidable pain. Likable charts help conversion.

You say “I do not discount people’s emotions”. I don’t see it, I’m sorry. The way I see it, you sacrifice everything to the altar of “chart effectiveness”. I don’t find a single one of your charts where the use of color is not purely functional. You say “you should support your claim with concrete examples”. I do have lots of examples: all your charts!

Let me reemphasized this: I agree with you. Chart effectiveness is what we should aim at. But I’m part of the choir. I’m not the layman. I don’t use pie charts.

Pie Charts Again

Unlike most people, I don’t think pie chart addiction is a disease. It is a symptom of a much more serious problem: low numeracy and poor data management skills. Address this problem and pie charts will virtually disappear.

How do you address this problem? “I don’t use pie charts, and I strongly recommend that you abandon them as well.” Researchers like Ian Spence and Stephen Kosslyn don’t think pie charts are as bad as you paint them. Even if they are, it’s very hard to talk people out of an addiction with purely rational arguments.

Perhaps this is my European soul speaking, but I do prefer a gradual approach (“this is acceptable, for the time being”) whereby people (hopefully) start to develop a sensibility to the perceptual issues.

By the way, how come we keep telling people that charts are about trends and patterns, not about the precise figures and then we argue that pie charts are bad because we can’t tell the difference between a 13% slice and a 14% slice? It doesn’t make sense (I’m exaggerating).

We must find more compelling arguments. I don’t like pie charts just because they are a waste of space (low data density) and can only answer very basic questions, better answered using a table. These arguments are good enough for me. I don’t care if we humans are bad at calculating areas and angles. That’s an academic argument that is irrelevant in the real world (I’m being provocative now…).

To Sum Up

You have  a very consistent approach to data visualization and you practice what you preach. You believe that you can convince people using rational arguments.

Mine is a much more comfortable place. I know that eye-candy is a can of worms that shouldn’t be opened. I know that we should protect the layman from himself. I know that simple rules with no exceptions work better than complex rules no one bothers to learn or understand.

But I like the gray areas. I like to protect the poor and the oppressed pies and I try to find their small role in the world of data visualization. The same with eye-candy. The same with emotions. The right amount can get your foot in the door. What is “the right amount”? I don’t know. I’m still searching.

{ 9 comments }

Is Data Visualization Useful? You’ll Have to Prove It.

by Jorge

Great data visualization is hard to measure: you can’t prove you have a good chart. Unless you can convince your employer to deploy at least two different formats/layouts and are able to compare results, you can say “this is a good chart” but that’s an act of faith, not an act of science.

It’s True Because It Rhymes

Information visualization experts like to evaluate a chart based on its compliance to some more or less accepted standards (Tufte’s data-ink ratio, for example). That’s like saying “it must be true because it rhymes”: the truth is defined by the language itself, not by the real world. Now, please close the curtains of our ivory tower…

I know, it’s not easy to assess the efficiency and effectiveness of good displays. They look natural and obvious, undeserving of praise and, probably, boring and uninspiring. Compare these charts:

Bubble charts

This is a true story: users wanted to evaluate sales territories, one at a time. Color-coding each bubble (Example A) was pointless, while Example B provided context without distractions. Guess what chart they would choose if they were allowed to… (happy ending: they reluctantly accepted Example B). (A word of advice: if you are looking for a promotion, a kindergarten chart variety always outperforms a “serious” chart.)

If your chart is doing a good job at helping people, no one will actually be aware of the chart’s role at making sense of the data. That’s why it is so hard to find good examples of data visualization using standard charts. If people actually like them, they like them because of their usability and/or interactive features.

When Stephen Few asks the readers “true stories about the benefits of data visualization” that’s almost an admission of impotence.  He should have hundreds if not thousands of good examples to share with us, right? Well, I know there are many examples out there, but I can give you none, sorry. Is data visualization some kind of astrology? I know it works. Why? Because I have faith. (On second thought, he is not asking for good data visualization examples. It really doesn’t matter if you use Tableau or Xcelsius, and that’s a relief.)

Opening the Pandora Box

Ultimately, what makes a good chart is how it resonates with your audience. Assuming that your are not unethically distorting the data, a chart that forces people to act is better than another one that only makes people aware of the subject.

If a single chart can save the world, it will not be a Few’s or Tufte’s 100% compliant chart. It will be a glossy Xcelsius pie chart.

(Wow, that’s depressing…)

If you read this blog that’s a clear sign of intelligence and sophistication :) . Unfortunately, you are not representative of the typical data visualization user and/or producer. The real world loves pie charts and doesn’t understand scatter plots.

Here is my Pandora box: give the audience what it expects and understands, even if that hurts your data visualization soul (OK, give it 90% of what it expects and use the remaining 10% to educate it.)

Cultural Relativism? Not So Fast.

Please don’t misrepresent these arguments. I’m not saying that all charts are born equal. There is a reference point and some misconceptions should be avoided A chart that maximizes insights, removes clutter, uses color wisely and clearly shows the patterns hidden in vast amounts of data, that’s probably a good chart and that’s what you should aim for. And yes, you should avoid pie charts.

If you present some sophisticated charts to your unsophisticated audience you’ll lose it. Relax. Draw a line but don’t forget the candies. You can take a horse to the water, but you can’t make him drink, unless you give him some sugar cubes…

{ 13 comments }

How to create a dashboard in Excel

by Jorge



Excel dashboards and executive reports are powerful, fairly easy to design and a great way to improve your Excel and data visualization skills. Because of its flexibility, you can virtually design any dashboard in Excel exactly the way you, or the users, imagined (and then send project to IT for implementation).

Once you know what will the Excel dashboard be used for and what kind of metrics users expect, there are three major areas that you must address when creating it:

  • how to bring data into your spreadsheets;
  • how to manage the data and link it to the dashboard objects, like charts and tables;
  • how to design the dashboard report.

Let’s take a look at each of them.

How to bring data into your Excel dashboard

Yes, Excel is a very flexible tool, but to create an Excel dashboard you can’t just paste some data and add a few charts, can you? You must maintain and update it, and if you want to decrease the cost associated with those tasks you must impose some structure to your data.

Usually the data should not be entered directly into the spreadsheet. You may copy /paste the data, but the best option is to connect the spreadsheet to the data source. There is a standard way to communicate with external databases called ODBC. You can use it to connect your dashboard to a table in Access or Oracle, for example. [Update: go to the members area, create a free account and watch the sample video on how to connect Excel to an Access table.]

Once you have that connection established, every time the data changes at the source it also changes in the spreadsheet, after refreshing. Keep in mind that a good practice is to minimize the amount of data you bring into your spreadsheet. If you don’t need it, don’t use it. If there are calculations to be performed, try to do them at the source.

Data can be imported using two basic structures: a flat file and a pivot table. As a rule of thumb, I would say that a pivot table tends to create larger files but the calculations will be faster. The flat file will be smaller but, if you need complex formulas to select the data, performance will suffer (you must test this in your own project).

How to manage the data and link it to the dashboard objects

If you use pivot tables you can extract the data using the GETPIVOTDATA function. If you use a flat file, there is a vast array of formulas you can use, like database formulas (DSUM, DGET), lookup (VLOOKUP, MATCH, INDEX) or math formulas (SUM, SUMIF, SUMPRODUCT).

Having many formulas and calculations is dangerous to your dashboard integrity. You can decrease this by using pivot tables. That’s one of the reasons I choose pivot tables by default.

Named Ranges

Do I have to tell you that you should document your work? Ok, I’ll tell you: yes, you should. But you can simplify that boring task by using named ranges. I am sure this is a familiar example: “=Sales-Expenses” is better than “=$G$4-$H$5”.

Named ranges not only help you making your formulas cleaner and easier to read, but they also play a key role in data management for Excel dashboards. Example: let’s say that the next update adds a new column to your table. If you don’t use named ranges you’ll have to change every single formula and add that extra column. With dynamic named ranges, the extra column is immediately available. This means less and more reliable work.

Dynamic Charts

I am a truly believer in dynamic charts, and dashboards that the user can’t interact with don’t make much sense to me. All my dashboards have controls to change markets, change products, change sales territories, whatever. What is important is that the user must be able to interact with the dashboard and build his/her knowledge by exploring the available data. Some managers prefer a static dashboard (perhaps just a sheet of paper, or a PDF file) but even then you should implement interaction, because it will be easier to create those static reports (you can automate them).

If you are creating interactive charts you will probably need dynamic ranges. You can do it using the OFFSET() function.

Macros

OK, you are not a programmer (I am not) but you should consider using some macros to speed up some repetitive tasks. Also, some cool things that you can add to your dashboards can only be done using macros, like greeting the users and selecting the right profile when they open the dashboard (you may want to read the discussion around the use of VBA in Excel dashboards).

How to design the dashboard report

If you like the kind of charts that you find in marketing-oriented tools like Crystal Xcelsius let me tell you this: they don’t work. There is a first wow! and then their uselessness starts creeping in.

This is not a matter of opinion only: I tried (and failed) to replicate my Demographic Dashboard (see below) using Xcelsius and detailed the process in a series of posts (1, 2, 3, 4, 5), so I know why these tools are useless for serious work. Don’t try to use Excel to mimic them.

Excel 2003 chart defaults are ugly and you should avoid them like the plague (and make sure you select the acceptable formats). Throughout this blog you’ll find many charting tips, and tips to improve your Excel dashboard, so I will not elaborate much on this. The basic premise is this: a chart is used to discover actionable patterns in the data, and you should do your best to show those patterns.

This means that you should remove everything that has no added value (3D effects, backgrounds, etc), create a hierarchy of focus and context data (make relevant data stand out, using color carefully) and add at least a basic form of interaction (let users select the series, for example).

Books

The reference in dashboard design if of course Stephen Few’s book, Information Dashboard Design. Other recommended reading are John Walkenbach’ Excel Bible (2003 or 2007) and Excel Charts (2003 or 2007); Tomothy Zapawa’ Excel Advanced Report Development (2003 or 2007); Michael Alexander’s Excel 2007 Dashboards and Reports for Dummies and Jon Peltier’s blog.

An Excel Dashboard Example: The Demographic Dashboard

This is an example of dashboard reporting using many advanced functions:

As I said above, it is difficult to put everything to work together in an Excel dashboard project. With the Demographic Dashboard, I wanted to show how to connect the dots and create a fully functional dashboard. So, I used a very interesting dataset from the US Census Bureau (population by sex, age and country, 1996-2050) to design it.

Actually, there are three dashboards… I took advantage of Excel’s built-in flexibility to create three versions of the same dashboard. They all look the same, but each one uses a different set of techniques:

  • The VBA version: This was the first one. It uses Visual Basic for Applications and pivot tables. It is faster than the other ones, but it relies on VBA to perform some tasks, and that may put off some users (I’m just using recorded macros, nothing too complex);
  • The VBA-Free version: After some discussion on the merits of using VBA (Excel Dashboards: Do You Need VBA?) I decided to add a VBA-free Excel dashboard; It mainly uses pivot tables and the powerful GETPIVOTDATA function. It is a well balanced version: it is slower than the previous version but since it doesn’t use VBA some users don’t have to leave their comfort zone;
  • The LookUp Table: It uses a spreadsheet database (basically a table that you paste into a sheet) and lookup functions (VLOOKUP, MATCH, OFFSET) to get the data for the dashboard report. Personally I don’t feel very comfortable with spreadsheet databases (more maintenance, more error prune, slower) but it is a good starting point to learn advanced Excel techniques (and sometimes is it your only option).

For a deeper discussion on the pros and cons of each version you may want to read the post Performance vs flexibility in Excel: Demographic Dashboard Lookup Edition.

Wrap-Up

Excel is a great tool for any type of business executive reporting. It allows for fast, flexible and cost-effective dashboard implementation (or prototyping). Intermediate users should be able to design and implement an executive dashboard in a matter of days (or even hours, for the first draft), provided that they know how to select the right techniques.

An Excel Dashboard is the Perfect Excel Learning Tool

You can read a million tips and still miss that wow! moment when you see all those small pieces working together. That’s why designing a dashboard is one of the best ways to learn Excel. If you want to get the details but also the whole picture, get my Demographic Dashboard Tutorial and start your Excel wow moment now!

{ 39 comments }