DSCI 100 Week 4 : Effective data visualization

Sky (Kehan) Sheng

Housekeeping

• Release of lecture recordings of Section 008?
• Fill out the Qualtrics survey here: https://ubc.ca1.qualtrics.com/jfe/form/SV_bIRJA5JgOjp5Gei

Canvas annoucement sent out

🔁 Week 3 Recap: Cleaning and wrangling

iClicker: Which function makes this tidy?

country	1999	2000
Afghanistan	745	2666
Brazil	37737	80488
China	212258	213766

• (A) pivot_wider()
• (B) pivot_longer()
• (C) separate()
• (D) select()

Answer: B, pivot_longer().

• year is a variable, but it is hiding in the column names 1999 and 2000.
• We want fewer columns and more rows, so the table gets longer.

🔬 Demo: pivot_longer()

• cols: which columns to collapse into longer rows
• names_to: the new column that holds the collapsed column name (i.e., year now records either 1999 or 2000)
• values_to: the new column that holds the values in each cell in the old columns (i.e., cases now records the number of cases in each country in each year)

Q: why does the column names in names_to and values_to take quotes?

• 3 rows × 2 year columns = 6 rows after.
• names_to and values_to take quotes, because these columns do not exist yet: we are naming them.
• Point out year is now a character column. Optional: mutate(year = as.numeric(year)).

🤔 And this one?

country	year	key	value
Afghanistan	1999	cases	745
Brazil	1999	cases	37737
Brazil	1999	population	172006362
China	1999	cases	212258
China	1999	population	1272915272

Q: “which function?” Q: “Why no quotes for names_from and values_from?”

• names_from: which column holds the new column names. values_from: which column holds the values to fill in.
• No quotes here, because key and value already exist as columns.
• Memory hook: longer = fold columns down into rows, wider = spread rows out into columns.

iClicker: Pick the right penguins

Keep only Adelie and Gentoo penguins that do not live on Torgersen island. Which code does that?

• (A) filter(penguins, species == c("Adelie", "Gentoo"), island != "Torgersen")
• (B) filter(penguins, species %in% c("Adelie", "Gentoo"), island != "Torgersen")
• (C) filter(penguins, species %in% c("Adelie", "Gentoo") | island != "Torgersen")
• (D) filter(penguins, species == "Adelie" & species == "Gentoo", island != "Torgersen")

Q: “what is %in%?” Q: “what is c()?” Q: “what is !=?” Answer: B.

• %in% asks “is this value one of the values in this list?”. c() builds that list. == with a list compares row 1 to “Adelie”, row 2 to “Gentoo”, row 3 to “Adelie” and so on, so it silently drops about half of the rows you wanted. Run A and B and compare the row counts.

Week 4: Effective data visualization

I love visualization and beautiful plots! I’m a visual person and hate reading tables. but plots are not always legitimate.

🚨 Visualization Crime Zone: What’s wrong with this bar plot?

The bar heights don’t match the numbers.

Is global warming real?

• Q: “So global warming is not real?”
• Let them sit with the flat line for a moment before the next slide.
• Hint if nobody spots it: read the y axis. It runs from -10 to 110 °F, so a few degrees of change is a hair’s width.

Same data, honest axis

• Same temperatures, y axis 55 to 60 instead of -10 to 110.
• Punchline for today: the same numbers can tell opposite stories, depending on how we draw them. Today is about drawing them honestly.
• Bridge: “So let’s look at some climate data ourselves.”

🤔 Is the air changing?

CO₂ traps heat in the atmosphere, so more of it means a warmer planet. Scientists on Mauna Loa, Hawaii, measured the CO₂ in the air every month from 1959 to 1997.

CO₂ (ppm)	month
315.42	1959-01
316.31	1959-02
316.50	1959-03
317.56	1959-04
318.13	1959-05
318.00	1959-06
316.39	1959-07
314.65	1959-08

… and 460 more rows

Look at the numbers.

Is CO₂ going up, going down, or staying the same?

Could you tell from 468 numbers? 😵‍💫

Let them try. The first 8 rows go 315, 316, 316, 317, 318, 318, 316, 314. Up? Down? Nobody can tell from a table.

🤔 Same numbers, as a picture

co2_info = transpose(co2_range)[0]
viewof co2_until = Inputs.range([co2_info.first_year + 1, co2_info.last_year],
  {step: 1, value: co2_info.first_year + 1, label: "draw up to year"})

{
  // Draw only: every point and the axis range were computed in R.
  const pts = transpose(co2_points).filter(d => d.year < co2_until + 1);
  return Plot.plot({
    width: 880, height: 360, marginLeft: 60, marginBottom: 46,
    style: {fontSize: "16px", background: "transparent"},
    x: {label: "year →", domain: [co2_info.first_year, co2_info.last_year + 1],
        tickFormat: d => String(d)},
    y: {label: "↑ CO₂ (ppm)", domain: [co2_info.y_min, co2_info.y_max], grid: true},
    marks: [Plot.line(pts, {x: "year", y: "concentration", stroke: "#2a78d6", strokeWidth: 2})]
  });
}

A good picture answers the question in one second.

• Drag the slider slowly. First few years: wiggles. Keep going: it clearly climbs.
• Two stories jump out that the table hid: the long upward trend, and the yearly wiggle (plants breathe in CO₂ in the northern summer).
• This is the same co2_df data we will plot ourselves in R later today.

🔄 Where are we in the workflow?

Week 3 was tidy and transform. Today is visualize.

✏️ Designing a visualization

• Ask a question, then answer it.
• The purpose of a visualization is to answer a question about a data set.
• A good visualization answers the question clearly. A great visualization also hints at the question itself.

Callback: the CO₂ chart. The question was “is the air changing?”. Anyone looking at the chart could guess the question just from the picture.

✏️ What questions can a plot answer?

Visualizations alone help us answer two types of questions:

• descriptive: What are the 12 largest landmasses on Earth?
• exploratory: Is there a relationship between penguin body mass and bill length?
• inferential, predictive, causal, mechanistic: we need more tools than a plot for these

• Descriptive: summarise characteristics of a data set, report a fact, no interpretation.
• Exploratory: look for patterns, trends or relationships within one data set. Often used to come up with hypotheses to test later.
• Inferential: does a pattern we saw hold in the wider population? Predictive: predict a value for a new observation. Causal: does changing one thing change another? Mechanistic: how does it happen?
• We meet inference and prediction later in the course.

🧱 Plots in R with ggplot2

ggplot2 comes with tidyverse. A plot has three key pieces:

1. aesthetic mappings aes(): which column goes to which visual property (x, y, colour, …)
2. geometric objects geom_*(): how to draw it (points, lines, bars, …)
3. scales: transform variables, set limits

Add the pieces one at a time with +.

Tip

Visualization is an iterative procedure. Try things, make mistakes, and refine!

🧱 Types of variables

A variable is a characteristic we measured. It comes in two kinds:

1. Categorical: falls into groups (categories). e.g. marital status, continent
2. Quantitative: a number on a scale, usually with units. e.g. height, horsepower

The kind of variable, and the question you ask, decide which plot to use.

⚠️ A number is not always quantitative: a car has 4, 6 or 8 cylinders. Those are three groups.

📊 Scatter plots

One of the best ways for us to battle climate change is to drive cars less frequently or drive cars with better fuel efficiency! Better fuel economy means less CO₂ generated per mile.

Is there a relationship between horsepower and fuel economy of an engine (relationship between two quantitative variables)? Does the number of cylinders change that?

Build up one piece at a time:

# Base 
ggplot(mtcars, aes(x = hp, y = mpg)) +
    geom_point()

# It's difficult to see the text so let's make it bigger 
ggplot(mtcars, aes(x = hp, y = mpg)) +
  geom_point() +
  theme(text = element_text(size = 26))

As horsepower goes up, miles per gallon tends to go down (negative relationship). Is that true for every car? Can we group the cars? Colour by cylinders:

ggplot(mtcars, aes(x = hp, y = mpg, color = cyl)) +
  geom_point()

cyl is a number, so R draws a colour gradient. Tell R it is a category:

ggplot(mtcars, aes(x = hp, y = mpg, color = factor(cyl))) +
  geom_point()

Cars with more cylinders tend to have more horsepower and lower fuel efficiency. Add labels:

ggplot(mtcars, aes(x = hp, y = mpg, color = factor(cyl))) +
  geom_point() +
  theme(text = element_text(size = 26)) +
  labs(x = "Horsepower", y = "Miles per Gallon", color = "Cylinders")

🧱 What each piece added

viewof build_step = Inputs.range([1, transpose(builder_steps).length],
  {step: 1, value: 1, label: "step"})
build_row = transpose(builder_steps).find(d => d.step === build_step)

html`<pre style="font-size: 15px; line-height: 1.4; white-space: pre-wrap;"><code>${build_row.code}</code></pre>
<div class="ds-caption" style="text-align: left; font-size: 18px;">${build_row.what}</div>`

{
  // Draw only: the cars, labels and axis range were prepared in R.
  const s = build_row;
  const r = transpose(builder_range)[0];
  const color = s.color_mode === "gradient"
    ? {type: "linear", range: ["#c7dbf4", "#123f75"], legend: true, label: s.color_label}
    : s.color_mode === "group"
      ? {type: "categorical", domain: ["4", "6", "8"],
         range: ["#2a78d6", "#eb6834", "#52514e"], legend: true, label: s.color_label}
      : undefined;
  const fill = s.color_mode === "gradient" ? "cyl"
    : s.color_mode === "group" ? "cyl_group" : "#52514e";
  return Plot.plot({
    width: 500, height: 360, marginLeft: 56, marginBottom: 46,
    style: {fontSize: "16px", background: "transparent"},
    x: {domain: [r.x_min, r.x_max], label: s.x_label, labelAnchor: "center",
        axis: s.show_axes ? "bottom" : null, grid: s.show_axes},
    y: {domain: [r.y_min, r.y_max], label: s.y_label, labelAnchor: "center",
        axis: s.show_axes ? "left" : null, grid: s.show_axes},
    color,
    marks: [
      Plot.frame({stroke: "#b8b6b0"}),
      s.show_points
        ? Plot.dot(transpose(builder_cars), {x: "hp", y: "mpg", fill, r: 5})
        : null
    ]
  });
}

📊 Line plots

Show a trend over an ordered quantity, usually time.

How has CO₂ in the air changed over the last 40 years?

Start with geom_point(), as we just learned:

ggplot(co2_df, aes(x = date, y = concentration)) +
  geom_point()

Clear upward trend. Q: what is not quite right here? The dots pile on top of each other (overplotting) and hide the order.

Switch to a line:

ggplot(co2_df, aes(x = date, y = concentration)) +
  geom_line()

Extra conclusion: the concentration also goes up and down every year. Optionally add labels:

  labs(x = "Date", y = "CO2 Concentration")

📊 Why a line and not points?

viewof co2_geom = Inputs.radio(["geom_point()", "geom_line()"],
  {value: "geom_point()", label: "draw with"})
viewof co2_zoomed = Inputs.toggle({label: "zoom into 1990 to 1992", value: false})

{
  // Draw only: points, the zoom flag and both axis ranges were computed in R.
  const all = transpose(co2_points);
  const pts = co2_zoomed ? all.filter(d => d.in_zoom) : all;
  const x = co2_zoomed ? [co2_info.zoom_from, co2_info.zoom_to]
                       : [co2_info.first_year, co2_info.last_year + 1];
  const y = co2_zoomed ? [co2_info.zoom_y_min, co2_info.zoom_y_max]
                       : [co2_info.y_min, co2_info.y_max];
  const mark = co2_geom === "geom_line()"
    ? Plot.line(pts, {x: "year", y: "concentration", stroke: "#2a78d6", strokeWidth: 2})
    : Plot.dot(pts, {x: "year", y: "concentration", fill: "#2a78d6", r: co2_zoomed ? 4 : 2.5});
  return Plot.plot({
    width: 880, height: 360, marginLeft: 60, marginBottom: 46,
    style: {fontSize: "16px", background: "transparent"},
    x: {label: "year →", domain: x, tickFormat: d => String(d)},
    y: {label: "↑ CO₂ (ppm)", domain: y, grid: true},
    marks: [mark]
  });
}

The line connects the measurements in order, so the up and down of every year shows.

📊 Bar plots

Compare amounts across categories.

Which are the 12 largest landmasses on Earth? Are they all continents, or are some islands just as large?

stat controls how geom_bar() turns rows into bar height. By default it counts rows. stat = "identity" means: skip counting, use the y value as is.

Simplest approach first:

ggplot(islands_df, aes(x = landmass)) +
  geom_bar()

What’s wrong with this plot?

To make geom_bar() use the y values as they are, set stat = "identity" (don’t count anything):

ggplot(islands_df, aes(x = landmass, y = size)) +
  geom_bar(stat = "identity")

The x axis labels overlap. Rotating them makes them hard to read. Put the labels on the y axis instead:

ggplot(islands_df, aes(x = size, y = landmass)) +
  geom_bar(stat = "identity")

Reorder the bars by size so the ranking is instant. Usually the largest bar goes closest to the axis, but that is not a hard rule:

ggplot(islands_df, aes(x = size, y = reorder(landmass, -size))) +
  geom_bar(stat = "identity")

Mention the labels should be real names. Optionally colour by is_continent as the last step:

ggplot(islands_df, aes(x = size, y = reorder(landmass, -size), fill = is_continent)) +
  geom_bar(stat = "identity") +
  labs(x = "Size (1000 square miles)", y = "Landmass", fill = "Type")

📊 Why did geom_bar() complain?

viewof bar_mode = Inputs.radio(transpose(bar_modes).map(d => d.mode),
  {value: "count the rows", label: "geom_bar() should"})

{
  // Draw only: counts, sizes, code and caption were computed in R.
  const m = transpose(bar_modes).find(d => d.mode === bar_mode);
  const color = {domain: ["Continent", "Other"], range: ["#2a78d6", "#eb6834"], legend: true};
  const chart = bar_mode === "count the rows"
    ? Plot.plot({
        width: 880, height: 330, marginLeft: 60, marginBottom: 46,
        style: {fontSize: "16px", background: "transparent"}, color,
        x: {label: "is_continent"}, y: {label: "↑ count (rows)", grid: true},
        marks: [Plot.barY(transpose(bar_counts), {x: "is_continent", y: "n", fill: "is_continent"}),
                Plot.text(transpose(bar_counts), {x: "is_continent", y: "n", text: "n", dy: -12}),
                Plot.ruleY([0])]
      })
    : Plot.plot({
        width: 880, height: 330, marginLeft: 120, marginBottom: 46,
        style: {fontSize: "16px", background: "transparent"}, color,
        x: {label: "size (1000 square miles) →", grid: true},
        y: {label: null, domain: transpose(bar_sizes).map(d => d.landmass)},
        marks: [Plot.barX(transpose(bar_sizes), {x: "size", y: "landmass", fill: "is_continent"}),
                Plot.ruleX([0])]
      });
  return html`<pre style="font-size: 15px;"><code>${m.code}</code></pre>${chart}
    <div class="ds-caption">${m.what}</div>`;
}

📊 Histograms

Show the distribution of one quantitative variable: how the values are spread out.

viewof bin_width = Inputs.radio(transpose(hist_summary).map(d => d.binwidth),
  {value: 5, label: "bin width (years)"})

{
  // Draw only: dots, bins and the axis range were computed in R.
  const r = transpose(hist_range)[0];
  const bins = transpose(hist_bins).filter(d => d.binwidth === bin_width);
  const x = {domain: [r.x_min, r.x_max]};
  const strip = Plot.plot({
    width: 880, height: 80, marginLeft: 60,
    style: {fontSize: "16px", background: "transparent"},
    x: {...x, axis: null}, y: {axis: null, domain: [-0.5, 0.5]},
    marks: [Plot.ruleX(bins, {x: "bin_start", stroke: "#b8b6b0"}),
            Plot.dot(transpose(hist_dots), {x: "life_expectancy", y: "jitter",
                                            fill: "#52514e", r: 3, fillOpacity: 0.6})]
  });
  const hist = Plot.plot({
    width: 880, height: 270, marginLeft: 60, marginBottom: 46,
    style: {fontSize: "16px", background: "transparent"},
    x: {...x, label: "life expectancy in 2016 (years) →"},
    y: {label: "↑ countries", grid: true},
    marks: [Plot.rectY(bins, {x1: "bin_start", x2: "bin_end", y: "n",
                              fill: "#2a78d6", inset: 1}),
            Plot.ruleY([0])]
  });
  return html`${strip}${hist}`;
}

{
  // Print only: every number was computed in R (hist_summary, hist_range).
  const s = transpose(hist_summary).find(d => d.binwidth === bin_width);
  const r = transpose(hist_range)[0];
  return html`<div class="ojs-readout">
    <span class="grey">${r.n_countries} countries</span> cut into
    <b class="blue">${s.n_bins}</b> bins. The tallest bin, ${s.max_from} to ${s.max_to} years,
    holds <b class="orange">${s.max_n}</b> countries.</div>`;
}

• Each grey dot is one country. The grey lines cut the axis into bins, and each bar counts the dots in its bin.
• Too narrow (1 year): spiky, every bin has a handful. Too wide (10 years): only a few bars, the shape is lost.
• geom_histogram() picks 30 bins by default and warns you to choose a better binwidth.
• Histogram vs bar plot: a histogram’s x axis is a number cut into bins, a bar plot’s x axis is categories.

🔬 Demo: Life expectancy by continent

Was there a difference in life expectancy across continents in 2016?

All countries first (this is the chart from the last slide):

ggplot(gapminder_2016, aes(x = life_expectancy)) +
  geom_histogram()

Colour by continent. fill, not color: color is only the outline of each bar.

ggplot(gapminder_2016, aes(x = life_expectancy, fill = continent)) +
  geom_histogram()

The bars are stacked on top of each other, hard to read. Overlay them instead, so they share the same baseline, and add transparency:

ggplot(gapminder_2016, aes(x = life_expectancy, fill = continent)) +
  geom_histogram(position = "identity", alpha = 0.6)

Still not effective. One small histogram per continent (facets):

ggplot(gapminder_2016, aes(x = life_expectancy, fill = continent)) +
  geom_histogram() +
  facet_grid(rows = vars(continent))

Much easier to compare. Mention the labels should be human readable:

  labs(x = "Life Expectancy (years)", y = "Count", fill = "Continent")

🗺️ Which plot for which question?

Question	Variables	Plot	geom
How are two numbers related?	2 quantitative	scatter	geom_point()
How does it change over time?	quantitative over time	line	geom_line()
How do amounts compare across groups?	categorical + amount	bar	geom_bar(stat = "identity")
How are the values spread out?	1 quantitative	histogram	geom_histogram()

More in the textbook: Chapter 4, Effective data visualization

📏 Rule of thumb 1: No tables or pie charts

Which one is easier to read?

• The colours in the pie mean nothing on their own, so the reader keeps looking back at the legend.
• It is hard to see the size of one slice compared to the others. Our eyes compare lengths from a common baseline far better than angles, which is why the bars win.

📏 Rule of thumb 2: No 3D

• The third dimension does not help us read the data.
• Perspective distorts: bars closer to the reader look bigger than the ones at the back.
• 3D is discouraged for charts in general, and only for very specific jobs.

📏 (Bonus) Sky’s exception: when 3D earns its place

• This is a plot from my own paper, so I know exactly why I reached for 3D here.
• Each panel crosses two categorical variables, prompt style (‘basic’, ‘typical’, ‘reality’) and whether the prompt was revised, against one quantitative measure: the percentage of AI generated farm images showing a practice. That is three variables in one chart.
• A 2D version would need six separate small multiples or a crowded grouped legend. The 3D grid lets a reader see all six bars in one glance.
• The point of this chart is a pattern, not a precise reading. Watch how the ‘revise’ bars nearly vanish next to ‘no revise’. That drop is the finding: an AI model quietly erasing outdoor access once the prompt gets revised.
• Still the exception, not the rule. Reach for 3D only when you can name a reason like this one, otherwise default back to rule of thumb 2.

📏 Rule of thumb 3: Simple, colourblind-friendly colours

Tip

Check your own plots: Coblis, Color Blindness Simulator

• Too many colours overwhelm. Less is more.
• Pick palettes people with colour blindness can read, e.g. from the RColorBrewer package, and check with an online colour blindness simulator.
• Redundancy helps: use shapes as well as colours for groups.
• Upload any plot to the Coblis simulator to see it the way someone with red green or blue colour blindness would.

Colourblind-friendly palettes in R

R has lots of colourblind-friendly palettes, such as the viridis package:

📏 Rule of thumb 4: Legible labels and legends

A great visualization tells its own story without you standing next to it.

• Column names like life_expectancy are for R, not for people. Use labs() to write real labels with units.
• Use theme(text = element_text(size = ...)) to make the text readable.

📏 Rule of thumb 5: Avoid overplotting

Usually the fix is a different geometric object. 53,940 diamonds on one plot:

• Add alpha = 0.2 to geom_point()
• alpha sets transparency, between 0 (invisible) and 1 (solid)

The plot draws on its own when the page loads (give it a few seconds, it is 53,940 points). The cell now shows the fixed version with alpha = 0.2.

Before: one solid blob. After: you can see where most diamonds are (small and cheap), and the stripes at 1, 1.5 and 2 carats (people like round numbers). Callback: for CO₂ we fixed overplotting by switching to geom_line().

📅 Upcoming Due Dates

Important

• worksheet_viz due next Monday, Oct 5, 11:59pm
• tutorial_viz due next Monday, Oct 5, 11:59pm

Tip

Before next class: please register for a free GitHub account, it will help you follow along: https://github.com/signup

Need a data viz refresh? Check out this optional video.

❓ Questions?

If not, let’s work on this week’s worksheet together.

• Want to save these slides? Right click → Print → Save to PDF.
• Your code and notes only save if you save as a PDF!