From the category archives:

Better Charts

Charts Are for Sissies (Grumpy Old Man’s Guide to Charts)

by Jorge

Making a chart is so simple that even a chimpanzee can be trained to do it – press F11 and get the banana (that would explain the poor quality of many business charts and presentations – and the raising banana consumption).

To prove that they are better than chimpanzees at making charts, humans invented the eye-candy and its epitome, the glossy 3D pie. Some well-known data visualization experts believe they are poor and useless, nothing more than lipstick (on a pig?).

A don’t agree. A think they are rich and very informative: there is no better chart to tell us that the author hasn’t the slightest idea of what to do with the data. (I am sure there is a strong inverse correlation between 3D pie charts and scatter plots. The more you love one, the more you hate the other.)

This is not just another rant about 3D pie charts. It’s about charts in general, even the good ones. If your only data analysis / communication strategy is to pollute the air with yet another chart then you are fully immersed in the sissy world, and lipstick is all over the place. Charts can help reduce information overload, but chart overload is not better.

A chart is just one of several tools you can use to make sense of your data. You need text, and plain figures, and statistical measures, and tables and yes, some charts. The best results come from the right blend of all those tools.

How do you know if you are a sissy (chart-wise)? Here is a simple clue: if you know how to use and interpret a box-and-whisker plot then you’re on the right track (extra points if you can do it in Excel). If not, do yourself a favor and find a good entry-level statistics manual.

{ 3 comments }

Better Color Palettes for Your Excel Charts [Video Tutorial]

by Jorge

You need a better color palette for your Excel charts, but you are a mere mortal and your artistic skills are less than stellar. Hell, you can’t even choose the right tie for a suit! So, what do you do? (hint: watch the video below)

Maybe we could ask Edward Tufte for advice. In Envisioning Information, he writes:

What palette of colors should we choose to represent and illuminate information? A grand strategy is to use colors found in nature (…). Nature’s colors are familiar and coherent, possessing a widely accepted harmony to the human eye (…). A palette of nature’s colors helps suppress production of garish and content-empty chartjunk.

Better said than done, right? Well, let me tell you a secret: it’s easier than it seems.

How to create a new Excel color palette

I’m going to show you how to create a palette of colors found in nature ( you can create a palette of colors not found in nature, if that suits you better). Here are the steps:

  1. Select a photo you like. If you happen to live in a big city, finding nature can be challenging. Google for “[your location]+park” and you may get lucky. Go there, take some photos and download them to your computer. If that looks like a lot of work, you can try Flickr. Search for “autumn colors” and you’ll get some useful results. I’ve picked the one on the right;
  2. Upload it to an online palette generator. There are many, so choose whichever you like. I like Genopal because it is clean (no ads) and extracts exactly the number of colors we need (eight).
  3. Convert color codes to RGB. Copy the color codes to Excel and use the HEX2DEC() function to get the RGB values.
  4. Create the Excel palette. In Excel 2003, replace the existing colors with the new ones. In Excel 2007 create a new color theme.

This is it. You can try it in your next chart.

A new Excel palette: the making of

If you need the details, here is a step by step tutorial:

Is this good enough?

As you can see, the results are heavily dependent on the photo you choose, so you should try different photos and test the resulting palette in a chart. If you find it hard to come up with a good color scheme, you can use this method to create a basic color palette that you can tweak to meet your needs.

Other Excel and Chart Color Resources

Jon Peltier wrote two interesting posts about managing the Excel color palette (here and here) and Stephen Few shares some Practical Rules for Using Color in Charts (PDF). Bonavista’s Chart Tamer includes a professionally designed palette.

Photo credit: Xavier Fargas

{ 4 comments }

Cooking Better Charts

by Jorge

Templates and defaults are very useful when you are not a subject-matter expert. You don’t have to know much, but if you choose the wrong templates you are on the wrong track.

Cooking is a good example. I don’t know how to cook and, frankly, I don’t want to learn. But my wife is coming late from work, I’m tired of take-away food and we must feed the kids. So I guess I have to add cooking to my daily routine.

Here in Europe, Thermomix (also known as “Bimby”) is a very popular kitchen appliance (specially among men, so I’m told). You just add ingredients, press three buttons (temperature, speed and time) et voilà, diner is ready (sort of). Follow the recipe  and what you get what you expect. That’s good enough for me.

This is a gadget most people love or hate. A friend of mine hates it and tried to persuade me that I should learn how to cook using the traditional pots and pans. Only ill-informed people buys it, he says.

Yes, but he’s single. No kids, no blog, loves to cook. How can he possibly understand my motives? I just want to get this thing done with minimal fuss, no random results, small learning curve. Some good templates, that’s all I want.

I had to buy one. I just did.

OK, but what about making better charts?

Apparently, most people love making charts as much as I love cooking. Even if they need them on a daily basis, there are so many things fighting for their attention that the need for a better data visualization can easily be overlooked and pushed to the bottom of their list.

Junk food, junk charts. Some times it’s great fun to use them. And it’s very easy to take them for real food, real charts. But they are unhealthy, and if you use them often you and your business will pay a price, sooner or later.

It’s not about food or charts, it’s about business and personal health (or wellness). You don’t have to know the details, you don’t have to know how to use the tools. But you have to have some kind of framework to guide you. Fat is bad, sugar is bad, 3D effects are bad. Then you need to find some templates that implement that framework and tools that use them by default (if you make charts in Excel, using its defaults is not an option).

We played with the machine over the weekend and I’ve very pleased with the results. We eat better than we usually do and saved money and time. Not bad. Now, do yourself a favor and find your own charting machine. If you’ll excuse me, mine is calling me. Dinner is ready.

{ 0 comments }

Weltanschauung, Lies and Charts

by Jorge

It’s very easy to use charts to support false arguments, distortions, omissions or outright lies. But you can use words and statistics too. If you want to deceive nothing will stop you. (Required reading: How To Lie With Charts and How to Lie with Statistics).

Simple lies are often easy to spot and not very interesting. More interesting are our biases. Germans call it weltanschauung (“world view”) and without it we wouldn’t be more than boring rational machines. Our biases help us to select the data and interpret it the way it makes sense to us, reinforcing our believes.

Lying with charts, if done properly (!), is more an act of omission (what you hide) than an act of commission (what you show). To better understand the differences, let me give you an example of how data visualization amateurs lie with charts.

In a post titled “charts can be deceiving”, E. D. Kain writes:

I’m not a huge fan of charts because I think they’re usually just used to create illusions and sales pitches.(…) Numbers don’t lie, but how we present them can make all the difference in the world.

Then he goes on and offers an example of how deceiving charts can be. I’ll recreate them for you. Chart A is the original chart, Chart B is his:

Jon writes about the flaws in both charts, so I’m not going to discuss them here.

It’s funny to see how a lack of action (the original chart accepts the Excel default scale) induces an over reaction (an absurd “theoretical” scale). Manipulating the y-axis scale is “How to Lie With Charts 101”.

Yes, charts can be deceiving. Words too. Numbers don’t lie? Bullsh*t. The political discourse is full of “illusions and sales pitches” and carefully selected and biased numbers. Yes, charts can be deceiving. It takes one to know one, I guess.

Deconstructing Lies with Charts

The original chart reveals a clear act of omission: how can you conclude anything relevant if you have no reference to compare the trend to?

So, let’s try to answer this question with the available data: are the wealthiest 1% of households getting a more favorable tax treatment, or not? First, we need some contextual data:

So, the wealthier you are the more taxes you pay; a downward trend is also visible across quintiles (although the highest quintile shows a slight increase over the last two years).

And what happens within the highest quintile?

Well, this is interesting: tax rates increased, but not for the top 1%. But the general rule is kept: the higher the income, the higher the tax rate.

What if we chart, not the tax rate but the change, assuming 1993=100? Again, some contextual data:

Tax rates for the lowest quintile declined sharply. And here is another general rule: the higher the tax rate, the less it changes.

Here is the detail chart for the highest quintile:

Well, it seems that at the very top some rules don’t apply, after all (surprise, surprise). The top 1% households did get a more favorable tax treatment after 1996, when compared to the top 5% and the top 10% households.

Takeaways

The world is never black and white, and your own shade of gray is as unique as your fingerprints. If you want to use charts to support your arguments, please don’t resort to scale tricks and make sure you add enough detail and contextual data.

There is no intrinsic objectivity in a chart, but if you want to support your story you should cover your bases and make sure it’s hard for someone else to come up with a different narrative. This is valid for charts but also for words and numbers.

{ 6 comments }

Seth Godin Is Getting It…

by Jorge

From time to time, Seth Godin comes to visit our little field of information visualization, and I’m pleased to note that he is learning…

Today’s post, “How to make graphs that work” is remarkably better than “The three laws of great graphs” or “How to make a PowerPoint chart”. Today he warns  us against Excel and PowerPoint defaults and templates, invites us to tell a story, by following some simple rules and braking some other rules. It’s clearly a post on the safer side…

But I do have some remarks. Godin says:

when you show me something exactly like something I’ve seen a hundred times before, what do you expect me to do? Here’s a hint: Zzzzzz.

Right. That’s the problem with defaults and templates. But it’s a problem with all defaults and templates, no matter how good they are. So, what can you do? First choice: be relentlessly creative. Can you do it? Good for you, I can’t. Second choice: make your charts invisible. Show the message, not the chart.

Pie charts are spectacularly overrated. If you want to show me that four out of five dentists prefer Trident and that we need to target the fifth one, show me a picture of 5 dentists, but make one of them stand out. I’ll remember that.

I actually prefer the Presentation Zen style. If you have two slices, you just need a percentage. And be careful with the pictures you choose. In this case, I see a female dentist that doesn’t prefer Trident on Wednesdays…

You can animate, but only if you have a note from your doctor.

You can animate if there is a pattern inside (Hans Rosling, anyone?).

So, what do you think? Is Seth Goding ready for serious information visualization?

{ 0 comments }

Charts: Monthly Unemployment Rates by State 1976-2009

by Jorge

Here are two ways to display a relatively large dataset, montly unemployment rates by state since 1976. The first one is perfect to see the overall patterns, the range from the lowest to the highest, the outliers and the slopes. An interactive version would allow the user to highlight specific series.

A small-multiple version allows the user to focus on specific states, compare them to the normal band, etc. States are ranked by labor force size and, as you can see, in the first row seven out of ten are above the US average in April. In the last row, only one is above the US average. You can also see that Michigan was not well (unemployment-wise) long before the current crisis, or a spike in Luisiana (Katrina). It pays to study this chart carefully.

Bottom line: try to see the same data from different angles. There will always be semething interesting to find.

What do you think? How would you improve these charts? Would you use a different display? Share it in the comments! (here is the data file)

Update: I usually stay away from Excel’s surface charts, but I’d like to add this one:

Also check Michael’s Horizon chart.

{ 17 comments }

Sub-Prime Charts: Should Data Visualization Be Boring?

by Jorge

In a recent article for the New York Times, Paul Krugman, the 2008 winner of the Nobel Prize in Economics, writes:

“The banking industry that emerged from that collapse [the Great Depression] was tightly regulated, far less colorful than it had been before the Depression, and far less lucrative for those who ran it. Banking became boring, partly because bankers were so conservative about lending (…).Strange to say, this era of boring banking was also an era of spectacular economic progress for most Americans.”

Now that history is repeating itself, I believe that this applies to data visualization too. The 3D pie chart with pseudo-realistic textures, charting tools like Dundas, Crystal Xcelsius and Excel 2007’s charting engine, they all share the same spirit of the times that nurtured the sub-prime lending mess and all that followed. The spirit of the times that rewards illusory short-term results and effectively dismisses consistent, well-founded, long term strategies.

Can’t We Learn?

We may be scared of the future, but are we scared enough? Krugman again:

“Despite everything that has happened, most people in positions of power still associate fancy finance with economic progress. Can they be persuaded otherwise? Will we find the will to pursue serious financial reform? If not, the current crisis won’t be a one-time event; it will be the shape of things to come.”

Many business managers still associate fancy charts with serious decision-supporting tools. This is the right time to change. Eye-candy, “professional looking” charts are sub-prime charts, and if you take them seriously, they’ll do to your business what sub-prime lending is doing to the world economy.

Take a Chart Stress Test

Good charts are invisible. If your audience’s first comments go to your chart format and design, that’s a sure sign that something is wrong. Get back to your charting tool and create a new chart. Do it as many times as necessary. The audience must see and comment the data patterns only, not the chart.

Charts don’t have to be boring. ”If the statistics are boring, then you’ve got the wrong numbers” says Tufte. If you need your daily adrenaline shot, get it from the insights a good chart provides, not from the chart design.

What do you think? Is this crisis creating a serious “back to the basics” spirit that will influence the way organizations optimize their resources, including the time they spend creating useless charts and presentations?

Photo credit: Steve Kay

{ 5 comments }

Focus + Context (a Bar Chart Is Not a Skyscraper)

by Jorge

poverty-ratios-skyscraperTextures. 3D. Pie charts. Primary colors. Trends hidden behind labels. Backgrounds. Pie charts again.

Clear signs of a bad chart, right? Right. It is so easy to spot a badly designed chart that you can use a computer to do it. Don’t waste your time.

Let’s stop discussing the obviously wrong and start discussing the useless right. Like this chart here. (I’ve borrowed the dataset Nathan used in one of his visualization challenges – some interesting entries and great discussion there, by the way).

There may not be anything really, really wrong with this chart, but it reflects a bureaucratic way of thinking about data and data presentation where every single data point must be clearly shown and labeled. Just like a table.

Listen, unless you work for a statistics office, you should never create a chart like this. I know, it’s irresistible to check how well my state ranks, but identifying each and every data point in a virtually limitless bar chart makes no sense in most cases.

Do you read the labels between the top five and the bottom five? Charts like this encourage look up of individual data points, and for that a table is probably a better option. If anything, a skyscraper bar chart is a clear sign of loss aversion.

A Flexible Bar Chart: Introducing the Accordion Bar Graph

How do you graph a categorical variable with more than, say, 20 data points without creating a skyscraper? This is what I have in mind:

  • You must retain the overall pattern, so you can’t remove data from the chart;
  • Create one or more focus area (top five and bottom five, for example);
  • Gaps between bars should be larger in these focus areas, so that labels can easily be added.
  • Minimize the height of the remaining bars and remove the labels;

The chart should look like this:

focus-context-bar-chart

 

I like the accordion metaphor and I’m playing with it. An interactive version could use a simple event to create a focus inside the context area, so when the user moves the mouse the bar is enlarged and the label is shown.

What do you think? Do you agree that skyscraper bar charts are (almost) useless or should we focus on reducing the number of data points instead? How would you improve this design? Please share your comments and charts below.

Update

Well, if you want to know how to do this in Excel and read a great discussion about it, Jon wrote Accordion Chart for Jorge. He not only discusses some of the options but also shares the Excel file with us. Thanks Jon! And Dick, over the Daily Dose of Excel wants to make sure that your state is automatically highlighted (Ego Charts). Nice “quarter step”!

{ 22 comments }

Can Edward Tufte Do Business Charts?

by Jorge

1. Tufte, the Father of Eye-Candy Charts

Tufte’s The Visual Display of Quantitative Information, published in 1983, is probably the most influential book in the history of data visualization, and it is likely to remain so for some more time.

In his book, Tufte outlines for the first time a consistent theory of how a chart object should look like and why it should look like that. His guidelines are easy to understand and very quotable, not buried under six feet of abstractions. Think of well-known concepts like “data-ink ratio”, “data-density” or “chartjunk”: they all come from The Visual Display…

However, too often these principles are taken as self-evident, somehow “discovered”, not invented. A fundamental clarification must be made: these are aesthetic principles that Tufte transposes (brilliantly) from Ludwig Mies van der Rohe’s minimalism to the field of data visualization. These are not universal principles backed up by scientific evidence. Some studies find them helpful, some studies say they are irrelevant, but their effectiveness is hard to measure and they should not be taken as indisputable laws (I call this the “what-would-tufte-say syndrome“).

Unlike other authors (Jacques Bertin, Tukey, William Cleveland), Tufte recognizes that only an aesthetic framework can structure the image (color management, the role of non-data objects, how to emphasize/de-emphasize elements in a chart…). This is clearly the realm of graphic design.

Using aesthetics to improve function is probably the major contribution of Edward Tufte to the display of quantitative information. Unfortunately, this idea that a chart can be an aesthetically pleasing object (“Beautiful Evidence”, the title of his latest book, says it all) went astray and gave birth to a whole industry of eye-candy visualization tools.

From Tufte’s positivist point of view, a chart is defined by how well it makes a pattern stand out. It may be boring but, if that is the case, then “you’ve got the wrong numbers”. His faith in human rationality is both charming and frightening…

2. Patterns, patterns, patterns. And something else.

There are so many misconceptions  to discuss about data visualization that we often forget to emphasize this simple true: data visualization is about pattern discovery, finding useful, actionable visual patterns hidden in the data and make them stand out. Let me repeat: it’s all about visual patterns.

Tufte would agree, but here is the fun part: there is nothing wrong with using 3D effects, textures, and all the decoration in the world. Use them! It is your good taste against Tufte’s. You don’t have to like minimalism. Add color, clipart, anything that you think can engage your audience.

I am not kidding. It’s you, not Tufte, who defines your aesthetic program. Almost anything goes. But, whatever you do:

  • Don’t design technically incorrect charts: do not distort a circle, do not use more than one series in a pie chart, do not make an object variate in two dimensions when you are using a single series, etc. Just common sense, really. And, of course, if you want to break the rules, know them first.
  • Don’t hide the patterns: find the patterns and make them visible. Remove everything except the series themselves. Now start embellishing your chart. But remember: every little thing you add multiplies the clutter and makes the patterns harder to see. You’ll have to find that point where the impact of eye-catching decoration on pattern visualization goes beyond an acceptable threshold.

Please note that minimalism was not randomly chosen. Not only it makes pattern discovery much simpler but also creates a framework to evaluate what belongs to the chart and what doesn’t belong. You can reject it, but if you don’t have a different framework you must decide on an ad hoc basis. Unless you are an accomplished graphic designer (and even then), a minimalist approach is a good start and it should help you to find your own style.

3. Emotions, Emotions, Emotions

Let’s face it: you don’t have much choice. If you do not want to sacrifice patterns, the amount of of decoration that you can actually use is very limited.

So, what do you do with that limited amount of decoration? Essentially you’ll try to create the right emotional response. This is not what you would expect from a over-positivist chart that you end up with by choosing the minimalist path.

Refusing to acknowledge the role of emotions in data visualization is a bizarre thing, considering that you can’t remove aesthetics from the equation, and we all have an emotional sense of Beauty. What many hardcore Tufte fans may consider chartjunk can actually keep the audience from turning the page.

4. Edward Tufte and Excel

Throughout his books, Tufte often refers to the higher resolution of paper, and how it outperforms the current screen resolutions. His sparklines are meant to be printed, because only then the fine details can be observed.

In Edward Tufte’s vision, each chart is unique, and deserves the attention of a work of art. He despises PowerPoint and hardly mentions Excel. His charting tool is Adobe Illustrator, where he is in full control of each small detail. He admonishes against patronizing the readers, but he never really discusses the audience as something that should be taken into account when designing a chart.

5. Knowledge Is Built by the User

matrixpermutator

Much as changed in the last 27 years and you may think that Tufte’s The Visual Display… emphasizes the use of paper just because the extraordinary changes in information technology were still in their infancy back in 1983.

Thing is, that’s not the reason. The real reason is that Tufte always thought of a chart as a final product to be printed and handed to the audience, not something that could be manipulated by the audience.

There is a striking difference between Edward Tufte and Jacques Bertin. Bertin’s “reorderable matrix” is dynamic by definition, and and one of my preferred quotes summarizes perfectly his views:

“It is the internal mobility of the image which characterizes modern graphics. A graphic is no longer ‘drawn’ once and for all; it is ‘constructed’ and reconstructed (manipulated) until all the relationships which lie within it have been perceived.”

This was written in 1967, long before the PC was even imagined. Edward Tufte wants to design an efficient but elegant chart, Bertin wants to solve a business problem. There is no contradiction, one is not better than the other. They just serve different masters. (The image above is from Bertin’s Graphic Semiology and shows how a “dynamic chart” looked in 1967…)

Forty years have passed, but a vast majority of data users have no access to dynamic charts, either because they don’t have access to the right charting tools or they are unable to create those charts using their current tools (it is not that easy for a beginner to create a dynamic chart in Excel).

6. The Life Span of a Business Chart

In his essay “The Cognitive Style of PowerPoint” Edward Tufte argues that the tool itself is intrinsically flawed. I agree with him. Tools are not neutral. They can be forced to do things against their will, but that’s never easy. You can create a dynamic chart in Excel, but it is difficult. You can even force Excel to work like Tableau, but that’s like reinventing the wheel. You can create good chart in Crystal Xcelsius, but that’s against its nature.

The point is, you can apply Edward Tufte’s principles by the book, but that means spending hours perfecting a chart in Illustrator and then printing it. I’d love to. Unfortunately, that’s not exactly how things work in a business environment. The life span of a business chart is short and the time to create it, even shorter. We cannot use Illustrator to create business charts.

7. Take-Away Points

Break away from Edward Tufte, but make sure you know why. Add emotion to your charts (rationally). Decide if the level of eye-candy your audience needs goes beyond what you are willing to add. Other things been equal, an interactive chart should need less eye-candy than a static one. Above all, show the patterns (but make sure your audience wants to see them).

{ 20 comments }

The Inner Beauty of Business Charts

by Jorge

Naked womanCan a picture of a nude person improve your decision-making processes? (Please don’t say “yeaaaaah”.) Probably not, but if you need a good attention grabber a picture of a naked body is your best bet. Make sure you’ll add one to your next sales report.

Because, if you are using those glossy 3D pie charts from Crystal Xcelsius (or Dundas, or…), you are applying the same principle, safe-for-work version. Your underlying message to your audience is “you are so dumb that you don’t even understand a simple chart with a clear message. I have to use charts that obfuscate the message, but they grab your attention and that’s all that matters. Let me take my shirt off too.”

I’m a business analyst I usually try to create charts that can support the decision-making process. I am not a graphic designer, trying to illustrate a story and get reader’s attention.

When you are in a corporate environment you can enjoy the attention of your audience (the organization is paying for it…). Also, information is shared among people with similar professional profile that at least know what the basic concepts are.

On the contrary, in a magazine, your readers don’t know or may not care about your subject. How do you grab their attention? Your best option is to add a photo of a naked male/female. Can you justifiably use it to illustrate the story? Do it. You don’t? OK, try other attention-grabber devices, like a nice, glossy pie chart (not as satisfying, though).

These are different needs, but we, the so called “visualization experts” often fail to aknowledge that.

Eye-Catching Charts vs. Decision-Support Charts

Eye-catching charts are used to get the reader’s attention by providing some sort for light entertainment. Their primary focus is on the format. They use many colours and and large textured surfaces. Because of that, their data density is low and context is almost absent. A 3D pie chart is the typical eye-catching chart.

Decision-support charts  focus on the data and should be “invisible” (the audience sees the patterns, not the chart). There are no textured surfaces and colors are used to highlight specific details. The display real estate can be filed with context data, maximizing data density. The typical decision-support chart is, obviously, the scatterplot.

Charts for Analysis and Charts for Communication – Not Anymore?

This is the traditional split. After the analysis stage, the analyst should prepare his/her findings for the communication stage. But vendors like Microsoft and Business Objects have been short-circuiting this process, selling the idea that all you need is form, not content, and whatever stage you are in, you must have a nicely textured 3D chart.

These charts are sold as “professional-looking” and let’s accept that for a moment. They are professional-looking from a graphic designer perspective, but they are completely useless in a corporate environement where you have masssive amounts of data to deal with. I’m sorry to say, but the more textured charts you have the dumber you look.

Pin-Up Charts Don’t Belong Here

I don’t really care about pin-up charts (charts that the media pin up on their pages…).  Sometimes they are amusing (not sexy, unfortunately) but they just don’t belong in a corporate environment. If you need attention, make better use of your data to find its inner beauty or use a photo of a proper pin-up.

{ 6 comments }

← Previous Entries