Posts tagged as:

excel charting

Stephen Few, Data Visualization, Eye Candy and the Pie

by Jorge

Stephen Few left a comment in my post “Is Data Visualization Useful? You’ll Have to Prove it“. We all have much to learn with Steve, so instead of leaving the discussion buried in an old post, I thought it would be interesting to make it more visible. Please read the comment then come here and join the discussion. Here is my answer.

Steve, sorry if I sound provocative, that’s not my intention. You are the leading expert in data visualization for business, you are doing a remarkable work with your books, with your blog, with your forum, with your patience to answer posts like mine. I have to be  thankful for that. And I do agree with 95% of what you write. But you don’t want to be surrounded by people who fully agree with you, do you?

The Effectiveness of Data Visualization

You say “the effectiveness of data visualization is well established by a large body of empirical evidence”. I want to believe that too. However in this study Jarvenpaa writes:

“Graphical charts are generally thought to be a superior reporting technique compared to more traditional tabular representations in organizational decision making. The experimental literature, however, demonstrates only partial support for this hypothesis.”

And J.-A. Mayer adds:

“This study refutes the general superiority of visual information in improving the decision quality (‘naive superiority hypothesis’). The choice and design of visual presentation is determined by information structure, decision environment, the decision-maker and the task decision. (…) The successful use of visual information depends substantially on its acceptance by the manager and the environment.”

What do these authors tell us? First, we cannot be 100% sure about the effectiveness of data visualization. Second, there are many other variables at play. And third, managers must accept it. This is a critical factor. Managers love impression management, and making a good impression using the dreaded “professional-looking charts” is the path of least resistance.

Data Visualization Success Stories

I have no doubts that you could share with us many success stories. When I write about an “admission of impotence” I am not questioning your ability to create/lead/mentor successful data visualization projects. But if you want to use those projects to inspire the average person I think you’ll fail most of the time, unfortunately.

Let me tell you how the layman looks like in my part of the world. He makes charts like this:

He believes that a 3D pie chart “looks more precise” and he doesn’t know that Excel chart defaults can be changed (more advanced laymen are able to switch to more “impactful” colors like reds, yellows and bright greens). In my part of the world, a layman doesn’t even know what “data visualization” is about (and they don’t even care). (Here are some more profiles.)

If you are preaching to the choir your conversion rate may be high. But the layman is not easily impressed. You must convert one at a time, and that’s something many of us can’t afford. Can you? He’ll keep making those pie charts because that’s what his manager requires him to do, he doesn’t know better, he’s lazy or you fail to convince him of a causality effect between better charts and better results.

The Layman Must Like Your Charts

In a business environment, charts don’t have to be memorable, only results do. But if you want to change behaviors, your audience must like the new behavior and accept the unavoidable pain. Likable charts help conversion.

You say “I do not discount people’s emotions”. I don’t see it, I’m sorry. The way I see it, you sacrifice everything to the altar of “chart effectiveness”. I don’t find a single one of your charts where the use of color is not purely functional. You say “you should support your claim with concrete examples”. I do have lots of examples: all your charts!

Let me reemphasized this: I agree with you. Chart effectiveness is what we should aim at. But I’m part of the choir. I’m not the layman. I don’t use pie charts.

Pie Charts Again

Unlike most people, I don’t think pie chart addiction is a disease. It is a symptom of a much more serious problem: low numeracy and poor data management skills. Address this problem and pie charts will virtually disappear.

How do you address this problem? “I don’t use pie charts, and I strongly recommend that you abandon them as well.” Researchers like Ian Spence and Stephen Kosslyn don’t think pie charts are as bad as you paint them. Even if they are, it’s very hard to talk people out of an addiction with purely rational arguments.

Perhaps this is my European soul speaking, but I do prefer a gradual approach (“this is acceptable, for the time being”) whereby people (hopefully) start to develop a sensibility to the perceptual issues.

By the way, how come we keep telling people that charts are about trends and patterns, not about the precise figures and then we argue that pie charts are bad because we can’t tell the difference between a 13% slice and a 14% slice? It doesn’t make sense (I’m exaggerating).

We must find more compelling arguments. I don’t like pie charts just because they are a waste of space (low data density) and can only answer very basic questions, better answered using a table. These arguments are good enough for me. I don’t care if we humans are bad at calculating areas and angles. That’s an academic argument that is irrelevant in the real world (I’m being provocative now…).

To Sum Up

You have  a very consistent approach to data visualization and you practice what you preach. You believe that you can convince people using rational arguments.

Mine is a much more comfortable place. I know that eye-candy is a can of worms that shouldn’t be opened. I know that we should protect the layman from himself. I know that simple rules with no exceptions work better than complex rules no one bothers to learn or understand.

But I like the gray areas. I like to protect the poor and the oppressed pies and I try to find their small role in the world of data visualization. The same with eye-candy. The same with emotions. The right amount can get your foot in the door. What is “the right amount”? I don’t know. I’m still searching.

{ 9 comments }

Animation, Interaction and Dynamic Excel Charts

by Jorge

If you want to add animation to your charts that’s a clear sign that you have too much free time. Go out and play with the kids instead. :)

Yes, animation is a powerful attention-grabber, even more powerful than a glossy 3D pie chart in Crystal Xcelsius. And yes, it can actually be helpful (from time to time). But.

A good example of animation in data visualization is the famous Hans Rosling’s TED presentation, where a long-term pattern is clearly seen (at min 4:00):

This works well because the trend is easily identifiable and you don’t care much about the details. It would be much more difficult to make sense of this data if there were multiple trends and short-term variability. (More on the use of animated charts.)

After watching this presentation people often ask me: “Wow! Can we do that in Excel?” Wrong question. The right question would be “Wow! Can our data do that?”.

You can always create a simple animation in Excel, but it’s hard for a non-programmer to get a smooth transition effect. Here is an example from my dynamic Excel dashboard:

Although you can see a pattern emerging, you would need to add a complex interpolation routine to make it look better (read Jon’s post to see how a simple interpolation can be used).

Animation is better used if there is a pattern to be discovered, but you need something more: the ability to interact with the data. You must be able to stop, go back, get the details. While you can create a simple animation effect with a for/next loop in VBA, interaction with the chart is much easier using a Google spreadsheet with a motion chart:

(This is just an image. Click here to play with the chart.)

Dynamic Charts in Excel

In Excel, instead of creating a VBA routine, consider using a scroll bar linked to the value you want to change (year, for example):

Using a scroll bar adds some level of interaction because you can scroll back and forth, pause and examine the details for a specific year. Obviously, you need to change the chart data source dynamically.

Let’s recap how to create a dynamic chart in Excel. You can do it by changing values or by changing the data source itself (using a different range).

Option #1: Copying the Data

The range A1:E11 is our data set and we are comparing regions. Moving the scroll bar at the bottom changes the year. Column F is our data source. You can enter this formula there:

=OFFSET($A$1,ROW()-1,MATCH($A$13,$B$1:$E$1,0),1,1)

ROW() gets the current row and MATCH() returns the position of value 2002 (A13) in the range B1:E1. So, the formula in cell F2 reads something like this: start at A1, go down one row, go right three columns and get the data in that cell (range of width 1 and height 1). When the user selects a different year the data is copied to column F.

Option #2: Using a Dynamic Named Range

The second option is to create a dynamic named range. Create a named range and, instead of entering a fixed range, enter this formula:

=OFFSET(Sheet1!$A$1,1,MATCH(Sheet1!$A$13,Sheet1!$B$1:$E$1,0),10,1)

As you can see, it is very similar to the one above, but now the number of rows down is fixed (1) and it returns a range of 10×1 cells (because we have 10 regions, but if the number of regions varies you may use COUNTA(A:A)-1 to count the number of regions, excluding header – and don’t forget to move the year to a different cell…).

When you verify your range this is what you get:

You just need to use this range in your chart (when entering the range you must add the workbook name: “=Book1!SourceData”). You’ll also need a range for the category axis labels:

=OFFSET(Sheet1!$A$1,1,0,10,1)

You can download the spreadsheet here.

Take Aways

If you want to animate your charts make sure you do it because it adds value, not because you want to show off your skills. You can do it in Excel but it looks much better in Rosling’s Trendalyzer or in the Google spreadsheet gadget.

To create animations your data source must change dynamically, and that requires some work (and skills). I advise you to shift your focus from animation to interaction and use all that work to design a better user experience. Do you prefer a “wow!” (animation) or a “wow, thank you!” (interaction that actually helps the user)?

{ 3 comments }

Better Color Palettes for Your Excel Charts [Video Tutorial]

by Jorge

You need a better color palette for your Excel charts, but you are a mere mortal and your artistic skills are less than stellar. Hell, you can’t even choose the right tie for a suit! So, what do you do? (hint: watch the video below)

Maybe we could ask Edward Tufte for advice. In Envisioning Information, he writes:

What palette of colors should we choose to represent and illuminate information? A grand strategy is to use colors found in nature (…). Nature’s colors are familiar and coherent, possessing a widely accepted harmony to the human eye (…). A palette of nature’s colors helps suppress production of garish and content-empty chartjunk.

Better said than done, right? Well, let me tell you a secret: it’s easier than it seems.

How to create a new Excel color palette

I’m going to show you how to create a palette of colors found in nature ( you can create a palette of colors not found in nature, if that suits you better). Here are the steps:

  1. Select a photo you like. If you happen to live in a big city, finding nature can be challenging. Google for “[your location]+park” and you may get lucky. Go there, take some photos and download them to your computer. If that looks like a lot of work, you can try Flickr. Search for “autumn colors” and you’ll get some useful results. I’ve picked the one on the right;
  2. Upload it to an online palette generator. There are many, so choose whichever you like. I like Genopal because it is clean (no ads) and extracts exactly the number of colors we need (eight).
  3. Convert color codes to RGB. Copy the color codes to Excel and use the HEX2DEC() function to get the RGB values.
  4. Create the Excel palette. In Excel 2003, replace the existing colors with the new ones. In Excel 2007 create a new color theme.

This is it. You can try it in your next chart.

A new Excel palette: the making of

If you need the details, here is a step by step tutorial:

Is this good enough?

As you can see, the results are heavily dependent on the photo you choose, so you should try different photos and test the resulting palette in a chart. If you find it hard to come up with a good color scheme, you can use this method to create a basic color palette that you can tweak to meet your needs.

Other Excel and Chart Color Resources

Jon Peltier wrote two interesting posts about managing the Excel color palette (here and here) and Stephen Few shares some Practical Rules for Using Color in Charts (PDF). Bonavista’s Chart Tamer includes a professionally designed palette.

Photo credit: Xavier Fargas

{ 5 comments }

Are Excel Charts Hurting Your Business? 10 Mistakes You Should Avoid.

by Jorge

Are Excel charts, and Excel in general, a commodity, with no competitive advantage? Only if you want it to. Why? Because a vast majority of users:

  • Have the data analysis skills of a toddler (or less);
  • Can’t go beyond chart defaults;
  • Functions? They know how to click the SUM() button;
  • Don’t know what a dynamic chart is;
  • Think pivot tables are too complex;
  • VBA? No way!

I believe you can squeeze some competitive advantage out from Excel if you avoid some basic mistakes everyone else makes. Here is a very short list (some generic, some chart-specific).

#1. Assuming that Excel Can Do It.

Because a spreadsheet is such a loose environment, people often assume that all things numeric can be done using Excel and that no specific skills are needed. Using Excel as a database tool is an obvious example. If you can’t stop singing Ode to Joy every time you see one million rows in Excel 2007 that’s a clear sign that you are on the wrong track.

#2. Assuming that Excel Can’t Do It.

Most people aren’t aware of how powerful Excel is and use it almost as a pocket calculator. If you routinely have to manage quantitative data, learning a little more always pays off. As an example, you can use it to create complex executive dashboards or, at least, as a dashboard prototyping tool.

#3. Not Having a Go-To Person.

Not everyone needs to be an expert, but having someone that understands the business and proactively tries  to find better ways to perform common (or not so common) tasks should be a requirement.

#4. The Excel Islands.

If you have a data source you can connect your sheet to, do it. Don’t make the mistake of copy / pasting the data into your sheet. That’s a bad practice and it can undermine your organization’s entire information system.

If you connect your spreadsheet to the data source (via ODBC, for example) you are ensuring the integrity of your data and minimizing maintenance costs. If you are creating a new table, use a simple database tool like Access. And if you really, really, need to use Excel, make sure you create a table that can easily be exported and used by a different tool.

#5. The Uncharted Archipelago of Excel Workbooks.

Search for “book*.xls” in your hard drive. How many do you find? Get rid of all those useless files now.

#6. Bypassing IT.

We all have our little fights with IT, but we must understand their role and the reasons behind their actions sometimes make sense… Excel users often lack knowledge to understand data structures, data access, documentation, security (IT 101, basically). If your IT don’t want to conquer the world and actually helps users, you should work hand in hand with them.

#7. Not Using VBA.

I’m sorry to break this to you, but if you spend your time analyzing data and Excel is your primary tool, then you must use VBA. I’m not telling you should be a programmer, far from it. But you should be able to record a macro and make some simple tweaks. There are many repetitive tasks that can easily done using three lines of code.

#8. Excel is the only charting tool.

There are no neutral tools. They force upon you a certain way of doing things that you may not be comfortable with (or you shouldn’t). The Chart Wizard is one of the stupidest wizards I’ve ever met. If you need to spend time removing defaults there is something wrong with the defaults. Excel charts emphasize:

  • Stupid Defaults versus Cleaner Formatting Options
  • Static versus Dynamic
  • Marketing versus Insights
  • Disposable versus Reusable
  • One versus Multiple
  • Large versus Small
  • Island versus Continent
  • Presentation versus Discovery

I’ll detail this in a future post but, like PowerPoint, Excel charts do have their own cognitive style. If you want to do things differently you can, but you are choosing a path of endless pain. So, give some users the option to use a different charting tool or, as a bare minimum, use a better wizard.

#9. Forgetting the 3R’s.

Reduce, reuse, recycle. Yes, I’m talking about Excel charts. How many disposable charts are cluttering your hard disk? Go green when creating your charts:

  • Charts are typically much larger than needed, so you can safely reduce their size;
  • When you create a chart make sure that it can be reused when the data updates;
  • A chart for Market A is probably very similar to a chart for Market B. By recycling the chart from market to market you don’t need do create new charts and pollute your hard drive.

#10. Not Making Things To Work Together

Lists like this one are useless, unless you know how the basic system works. Try to see the big picture and it will be much easier to understand where each piece belongs to:

  • If a function needs a range address, you will know that another function can provide the address (first step to create dynamic charts);
  • If you are repeating the same stupid task, you’ll find a way to automate it;
  • etc.

Bottom line: make sure you know what you should use Excel for, find inefficiencies and give power users the option to use specialized tools.

{ 11 comments }