(You’ll see in a moment why there is a vertical line in 2005.) Things get a little more interesting if you split population by age groups. You can see that population is getting old at the top (more elderly people) and at the bottom (fewer children):

In population studies, there is a very useful dependency ratio, a ratio between Young and Adult population (Young Dependency) and between Old and Adult population (Old Dependency). Now, think about Jon, a baby-boomer. He was born in 1961. You can see him in the orange series. And you can see him again 65 years later:

 A strange scenario

It is clear that, while the young dependency ratio remains flat, the old dependency ratio sharply increases as soon as the first baby boomers enter old age, by 2011. But what happens if you plot a ratio against the other?

Here is our L-shaped trend, where the red dot marks 2005. There are several things at play here. The first one is, obviously, the impact of baby boomers. But you must see it in the context of the long-term demographic transition (from high birth and death rates to low birth and death rates). And there is a model of the projected fertility rate (in this case, the so called “medium-fertility variant”).

There are basically three scenarios. The line always goes up. If you assume that fertility rate remains stable then you get a vertical line. If you you assume that fertility rate keeps declining, the line bends to the left, and if  you are an incorrigible optimist and you believe that couples will start having more than two children again, then the line bends to the right:

I don’t like L-shaped trends, so I wanted to see how well the model is predicting the actual ratios. Here is a zoomed-in chart with estimates from the US Census and actual data from the latest census (2010).

Apparently, things are getting worse than predicted by the model: the old dependency ratio is going up (you can’t do much to change that) but the young dependency ratio is not frozen at the 2005 level (Americans are making fewer babies than expected). The US is not alone. Here are a few countries with a similar pattern:

And the largest ones:

So, unless a country is planning to kill or export its older citizens, its old dependency ratio is likely to keep increasing over the next years. Assuming that you don’t have much control over the number of elderly people, your only option is to play with fertility rates. Given current economic and financial uncertainty, I don’t know how fertility rate can increase.

The incredibly shrinking country

If the model is right, many countries will have to deal with an over-sized elderly population followed by a dramatic decline of total population. Here is the UN data for Portugal (in 1990 the 85+ age group is split):

______________________

Post from: Excel Charts Blog.

Strange L-shaped trends

The original post is titled Strange L-shaped trends , and it came from The Excel Charts Blog .

If you want to make a choropleth/thematic map in Excel without programming perhaps conditional formatting is all you need. Here is how to do it:

1. Select a few hundred columns and rows;
2. Set width and height to 3 (more or less, depending on the resolutions you want);
3. Set font size to 1;
4. Place a map file over those cells and make it transparent (if possible, use a WMF file) so that you can see the cells and the map borders;
5. In a new sheet, add a table with all the regions and some random data;
6. Get back to the first sheet and for each cell within a region enter a lookup function and change cell background (the lookup must include the region ID);
7. Save your work and make a copy of this sheet;
8. Select a rectangle around your map and add a border;
9. Choose a conditional formatting rule for the entire rectangle;
10. Clear the existing cell background color;
11. You should see a colored map using the conditional formatting rule.
12. Inside the rectangle, merge a few cells and create a legend (in this case, I used percentiles to automatically update it);
13. You can delete the overlapping file, but the maps will look nicer with the borders;
14. Use the camera tool to copy the map to a new sheet.

Here is the sheet for the example above:

Make Excel choropleth/thematic map without programming.

Enjoy!

______________________

Post from: Excel Charts Blog.

Making Excel maps without VBA

The original post is titled Making Excel maps without VBA , and it came from The Excel Charts Blog .

what you really want to do is to compare each slice to the whole, like this:

because, if you want to compare them you must do this:

I hope that you find this pretty obvious. If you don’t, let’s add one more series.

When you do this:

what you really want is to compare categories for each year, so you use a bar chart:

or, even better, a slope chart:

You never ever compare proportions, only a single proportion to the whole. The moment you change your question and want to compare data points you have to use a different chart. That’s why the optimal number of categories in a pie chart is one.

Capisci?

______________________

Post from: Excel Charts Blog.

Finally revealed: the optimal number of categories in a pie chart

The original post is titled Finally revealed: the optimal number of categories in a pie chart , and it came from The Excel Charts Blog .

(click to enlarge)

I have a single and very simple resolution for 2013: make more charts. Simple charts, just to play with the data. Here is the first one. I like scatter plots with a time dimension, even though data points often look like drunken sperm.

Ah, by the way, I decided to use a dotted line to denote a break in a series. We often forget these things.

So, how would you improve it?

______________________

Post from: Excel Charts Blog.

Chart: Education vs. GDP in Europe

The original post is titled Chart: Education vs. GDP in Europe , and it came from The Excel Charts Blog .

Fortunately, life is not that simple. Many people dislike charts and a good table with the latest data is more than enough to get the insights they need. This kind of puzzles us. Haven’t they seen the light yet?

Is context overrated?

Let me put it this way: external visual context may be overrated.

Suppose you’re a programmer. You want to solve a very specific problem, like making a routine run faster. You search your favorite tips & tricks site and find what you were looking for. End of story. Your knowledge and experience provide all the context you need. Let’s call it internally supplied context.

If you are a beginner things are a bit different. You probably don’t even know that your routine can run faster. If you do, you don’t exactly know what to look for. And if you do, you don’t know how that new piece of code works and you need help. You need explicit externally supplied context.

Journalists and graphic designers love external context (hence they love charts) because they are not subject-matter experts. They work with other people’s data. If I say “an unemployment rate of 12.1%” a subject-matter expert can easily provide internal context, while the journalist (and her audience) need a more explicit external context.

Are patterns overrated?

I’d like to know more about table-based decision-making process, but I suspect that people are less aware of data patterns and more interested in some kind of fluctuation bands, and they compare data points against them. This is an interesting alternative data reduction technique.

The dangers of a consolidated knowledge

Internally supplied context and fluctuation bands are two by-products of a mature and consolidated knowledge. They can be very effective in decision-making and help seeing beyond short-term trends. On the other hand, in a rapidly changing environment they can (dis)miss relevant but unexpected changes.

So, what do we do?

We may not like it, but we must accept the fact that some people are less visual that others and that they can get the information they need from a table or a written report. They already know what they need to know and they can provide the necessary context to deal with a few data points. A chart is useless and redundant.

How can we convince them that a chart is a good thing? Well, try this:

• Don’t tell them quantitatively what they already know qualitatively.
• Show them more complex relationships.
• Try to find unexpected patterns.
• Try to find their pain points and solve them (all processes can be improved).

So, what do you think? How can you convince a table person to become a chart person?

______________________

Post from: Excel Charts Blog.

Context comes in many ways and shapes

The original post is titled Context comes in many ways and shapes , and it came from The Excel Charts Blog .

You make charts all the time, but you don’t really know if this new “data visualization” is the same thing with a pompous name or is something bigger and more complex. Perhaps you could start with a quick image search in Google for “data visualization“. Here is what you’ll get in the first page:

Wow! Fascinating! But… is it relevant for you, for your work? Let’s check the prerequisites:

• Data types that you don’t have in your work;
• Artistic talent you are not born with;
• programming skills you are not interested in;
• tools your company will never buy.

What’s left? Aesthetic enjoyment as a consumer and little inspiration for a chart maker.

Is data visualization for you?

Well, don’t feel excluded from the fascinating world of data visualization just because these results don’t help you in your work. There is a vast silent majority equally frustrated!

Before you return to your simple and boring charts, let me ask you this: do you think that a poetry soiree is a good sample of how people use their mother tongue? Of course not! These search results are like a poetry soiree (in the sense of a very specialized use of language). If you just need a functional use of language, poetry may not be your best role model (don’t get me wrong: we need poetry too).

Is there information visualization without aesthetics?

One of the data visualization founding fathers, Jacques Bertin, believed that data visualization is an objective set of signs and rules, with no place for aesthetics. I actually disagree, but back then the world was in black and white and it was easier to accept it. You can’t avoid aesthetics. It is embedded in every single one of your choices when making a chart.

You can approach data visualization from multiple perspectives, depending on your background, skills or goals. Given the same data set,  a designer, a product manager or a statistician will come up with very different visualizations. The problem, today, is that we are witnessing some kind of “over-aesthetization“. The more visible perspective of what data visualization is about is defined by the results returned by search engines. They obviously emphasize design (beauty tends to be linked and shared more often), not visualizations made by engineers or statisticians. And it all started with Edward Tufte. He must be proud (I always wanted to write this). This is bad, because many people may start to equate data visualization and pretty charts, nice to look at but not terribly useful for their job and their skill sets.

Occam’s razor, or How can we avoid an aesthetic disaster?

To tell you the truth, I’m glad most corporate charts are not listed in the search engine results. Many are ugly, useless and childish (yes, childish). But if you can’t even choose the right tie for your shirt how can you avoid a disaster when making a chart?

Glad you ask. There are no easy answers, but let me put it this way: if your goal is to facilitate chart reading, remove everything that cannot be justified and justify all the other options. That’s probably the safest way to avoid major disasters.

It’s easier to explain with an example. Compare the first chart, using the embarrassing defaults in Excel 2003 (the smiling cow adds the childish touch) with the second chart, also made using Excel 2003.

Let me justify my options:

• Why is the title not centered? Because left aligned text is easier to read;
• Why is the vertical axis line not displayed? Because most of the time is unnecessary, specially when you have axis labels and grid lines;
• Why don’t you use a legend? Because direct labeling minimizes eye movement and working memory usage;
• Why are grid lines light gray? Because they are less relevant than the data itself;
• Why are grid lines dashed? To differentiate them from the horizontal axis line;
• Why is the line Chicken wider than the others? Because it is the series that interests me the most and I want to draw your attention to it;
• Why there seems to be three colors with varying tones? For you to realize that the series in each group have something in common (red meat, poultry, fish) ;
• Why was the happy cow removed? Because its childish, distracts and adds no benefit whatsoever.

I suggest you to continue the exercise with some more questions and answers (why don’t you label Lamb and Veal? Why is Year on the right?). At first, not all your answers will be coherent and consistent with your goal but you’ll get there.

Color is one of the major components of aesthetics and very hard to manage. The colors above were not chosen at random. The reddish tones are associated with red meat, orange with poultry and blue with fish. Is this the best possible color combination (assuming it exists)? Far from it. This is just a choice that I can justify, and that in general will be better than a different one base on personal taste alone.

This chart will not win a beauty contest in the near future but, because your goal is to facilitate reading and you are forced to justify all your options, the risk of an arbitrary solution based on a not-to-be-trusted aesthetic sensibility is greatly minimized. Gradually your answers will become more consistent, the process more natural and you’ll find your own style.

There are multiple aesthetics

Newton’s Binomial is as beautiful as Venus de Milo. What happens is that few people notice it.

Perhaps we could follow Fernando Pessoa and write:

Minard’s map of Napoleon’s March is as beautiful as the Sistine Chapel.

Beauty in a chart lies not in its ability to provide a moment of aesthetic enjoyment but in helping us understand the world, elegantly. So, start making elegant charts and aesthetics be damned!

P.S.: If you’re interested to find the right balance between aesthetics and function in the context of data visualization, you cannot avoid Alberto Cairo’s The Functional Art, probably the best data visualization book published in 2012.

______________________

Post from: Excel Charts Blog.

Aesthetics be damned

The original post is titled Aesthetics be damned , and it came from The Excel Charts Blog .

Becoming a Data Visualization Anarchist

I think things can be changed from the inside, improving the way people use Excel. I write for Excel users because I’m one of them. That’s not going to change soon. But I love data visualization, not the tools that make it happen. I specially like interaction, multiple charts and making them available on the web. And I need to manage more data (not big data, just more data). Some things can’t be done in Excel or require too much effort.

The Portuguese poet Fernando Pessoa has a funny short story called The Anarchist Banker. The idea is that only a banker can be a true anarchist, because only a banker can be free from “social fictions”, specially money… In data visualization, this means getting the tools out of the way, by learning them or avoiding them.

I chose the learning path and I’m learning R now. I always wanted to make those scatterplot matrices. And I want to play with the ggplot2 package. A programming language is covered but R is not going to pay the bills.

Enter Tableau and Qlikview

I will not complicate matters  by discussing how I chose Tableau and Qlikview and not Spotfire, for example. I just want to choose one. Qlikview vs Tableau. Comparison articles like this and this are very helpful, but a man is a man with his circumstances. Each starting point is different from everyone else. Let me tell you what I think I know about these tools in this early stage.

Tableau

I like Tableau, I have to admit it. I like the fact that you don’t have to fight stupid defaults in design and formatting, because I share the same data visualization principles.

I like its enthusiastic and knowledgeable community. Let me give you two simple examples. I spent a lot of time making this horizon chart in Excel, and Joe Mako quickly came up with a better version in Tableau.

Then I tried to be creative with the bamboo charts and Joe Mako strikes again, with a better implementation. I’m starting to get nervous… (kidding)

I know and respect many Tableau users, not because of Tableau but because I share the same views regarding data visualization.

Apparently, maps in Tableau are good enough, so that’s a good point.

And as a blogger, I want to make my work available to the web, and Tableau Public is a nice option (my population pyramid).

The Guardian often publishes Tableau visualizations. I’d like to try that with the local newspapers here.

On the other hand, we know how stubborn some datavis experts are. Is Tableau that stubborn? Can clear vision and the right principles become a straitjacket? I really hate straitjackets (“the idea of”, never actually tried one…)

In my country, Tableau is virtually unknown and I am not sure if I want to sell shoes in Africa.

Qlikview

I know even less about Qlikview. The first chart I see in its video is the  pie chart above. Not exactly a shiny example of good data visualization principles.

Apparently there is a very active Qlikview community on Linkedin but not so much on Twitter. Probably this is meaningful.

I keep reading that Qlikview is better than Tableau when it comes to making dashboards, while you should explore the data with Tableau. It’s a good point in favor of Qlikview (that’s what I need now). Extensions and the market seem to be interesting too.

Qlikview has several business partners here. Actually, I was invited to work in some Qlikview projects in 2013 (obviously I have to learn the basics until then). They can pay a lot of bills.

Its not always about features

Not everything is black and white, not everything can be decided based on feature-by-feature comparison. Not everything is heart, not everything is reason. If I choose Tableau, my data visualization skills will improve a lot. Qlikview is harder to predict. I’m sure there are many users that dislike the pie above. If not, Qlikview can be more, hummm, challenging.

What I’m going to do

I mentioned those Qlikview projects, but I’ll try to remove them from the equation, at least for now.

I have a simple dashboard in Excel and I’d like to create Tableau and Qlikview versions. That’s probably one best ways to evaluate a tool, using my own work.

So, can you help me?

I’d love to learn from you. Can you answer questions like:

• How do they compare regarding maps? Is it simple to add your own maps?
• Is it true that it’s easier to make a a centrally designed dashboard in Qlikview, while Tableau has a more exploratory nature?
• How can I share a Qlikview chart in my blog?

______________________

Post from: Excel Charts Blog.

Qlikview vs Tableau? I have to choose and I’m not sure

The original post is titled Qlikview vs Tableau? I have to choose and I’m not sure , and it came from The Excel Charts Blog .

You already know that a bar chart helps you to compare data points, while a line chart is better at displaying trends, right? But you keep staring at them, not knowing which one to choose.  Suddenly, your little guardian angel whispers: “the bar chart is wrong”.

The bar chart? What’s wrong with the bar chart? And why is the line chart OK?

Then you realize that the vertical scale starts at 30, and apparently it should start at zero, so you change it in both charts:

That leaves a lot of white space under the line. The experts say you don’t have to start scales at zero if you are using line charts. So you revert the changes in the line chart:

Ah, yes! Now you can choose one of them. Your guardian angel agrees that both charts are correct. So, do it, pick one!

What’s the problem? Is there something bothering you?

Humm, I see. They don’t look that similar. You believe that people may draw different conclusions depending on the chart you choose.

OK, let’s discuss this a bit. Let’s talk about resolution.

What is chart resolution, anyway?

Higher resolution is usually a good thing. It means that you can see more clearly the difference between data points. To improve resolution in a chart, you zoom in, using the numeric scales:

How much you can zoom in? Well, the lower limit should be the first nice round number below the minimum value in your data set, and the upper limit should be (you’ve guess it) the first nice round number above the maximum value.

You have to make a little change to the rule when it comes to bar charts. In a bar chart, people compare heights, so if the bars are not proportional to the data they encode, you are misleading your audience. So, the charts above are both correct, but the one below is not:

That’s why the lower limit in a bar chart should always be the value that maintains the right proportions (usually zero). So, the take-away message is, improve resolution byt changing the scale, but in a bar chart you must keep proportions aligned with the data.

What about slopes?

Line charts are more subtle. Both charts below are correct:

The only difference between them is that the one on the left has a higher resolution, and in general is a better option considering Cleveland’s suggestions for banking to 45º (slopes should average around 45º). You can do it by changing the numeric scales and/or the chart aspect ratio. This is a suggestion (an excellent suggestion), but it also tells you that there is no strict rule to obey.

To be completely honest, I don’t care much about scales or aspect ratio in line charts, as long as they do not go overboard. What really matters is to have something to compare with. In the post Weltanschauung, Lies and Charts, I use these politically biased charts…

… and argue that only when you have more than one series you can learn anything from a line chart, like this:

So it doesn’t matter much what you do with scales or  aspect ratio, as long as you have two or more series and your goal is to compare them.

You can learn more about scales in Naomi Robbin’s Creating More Effective Graphs and she often writes about it, like in here and here.

Perhaps you could use this as a rule of thumb: use a bar chart when you have a single series and a line chart when you have two or more series. It will not always work, but it’s a good starting point, don’t you think?

By the way: “méfiez-vous des morceaux choisis” is a lovely French expression that roughly translates to ”beware of selected pieces”.

______________________

Post from: Excel Charts Blog.

Bars and lines: méfiez-vous des morceaux choisis

The original post is titled Bars and lines: méfiez-vous des morceaux choisis , and it came from The Excel Charts Blog .

One little thing bothered me, though. Explicitly or not, many stories ended with the words “and now you know why”.

And now you know why.

We, the data visualization community, love to talk about visualization as a kind of storytelling but can we honestly say that our visualizations can deliver a good story? Can our audience say, “now we know why”?

We can argue that this “visual storytelling” is nothing more than new-agey mumbo jumbo. After all, we just need to process data, get insights and act. We don’t have to mimic a folktale.

If we do agree that storytelling can help us creating better visualizations, I have a humble suggestion: from now on, each datavis conference should be required to invite a renowned storyteller and let him/her show us what storytelling is really about. I liked Ben Haggarty and Jan Blake by the way.

If you want to improve your charts/infographics/dashboards, here is a simple trick: write, below them, the words “and now you know why”. But first make sure they deserve it.

______________________

Post from: Excel Charts Blog.

And now you know why

The original post is titled And now you know why , and it came from The Excel Charts Blog .

We are going to resist the urge to identify them and use our preconceptions against them. Let’s dig deeper instead. Some groups are more exposed to the risk of poverty. Let’s see what happens when we split the data by sex:

How do you read this? For each sex, at the top you have the national average and at the bottom you have the group value. If the group has a lower risk, the line points to the left; otherwise, it points to the right. So, in this case the data shows very consistently that females have a higher risk of poverty or social exclusion than males.

Perhaps there are other factors. with higher variability.  Let’s try age:

Lots of interesting stuff here: leaving your parents (16-24) is risky, while if you are 65 or over it can be heaven or it can be hell, depending on the country you live in. Let’s now add education:

As expected, there is a strong correlation between education level and risk of poverty. What about income and household type?

As you can see in the income, poor people run an extremely high risk of poverty (Monsieur de Lapalisse dixit). Explore the household type. Very interesting stuff too.

If you want to focus on the leaves only:

Here is the complete chart (click to enlarge):

Here is a comparison between two countries, Norway and Romania.

The bamboo chart:  the bastard son of Mr. Slopegraph and Madame Parallel Coordinates.  Do you like it? Do you find it useful? Confusing, perhaps? Do you know of a similar chart (link, please)?

How would you represent the same data? Here is the dataset (xls) for you to play.

______________________

Post from: Excel Charts Blog.

Bamboo charts: People at risk of poverty or social exclusion

The original post is titled Bamboo charts: People at risk of poverty or social exclusion , and it came from The Excel Charts Blog .