Lesson 7 — Exploring Data (EDA) in R

1Objectives¶

By the end of this lesson you will be able to:

1. Compute a full set of summary statistics for a numerical variable with favstats().

2. Build frequency tables for a categorical variable with tally(), including proportions.

3. Make a histogram, boxplot, and bar chart with the gf_ (“ggformula”) family of plotting functions, using the same y ~ x formula grammar from Lesson 6.

4. Read a plot’s shape, center, and spread out loud, in plain language.

Everything in this lesson runs the same way whether you’re on your own laptop or on the CSUB JupyterHub (https://csub.jupyter.cal-icor.org/) — same functions, same output.

2Setup¶

As of Lesson 5, every session starts the same way:

library(mosaic)
library(BSDA)

library(mosaic) also turns on its plotting layer, ggformula, automatically — you never need a separate library(ggformula) line. This lesson reuses KidsFeet (the built-in foot-measurement dataset from Lesson 6) and coffee, the coffee_wait_sim.csv file you imported there:

data(KidsFeet)
coffee <- read.csv("data/coffee_wait_sim.csv")

31. favstats(): one variable, then by group¶

Chapter 2 of the coursebook is where this section matches up. You met favstats() briefly in Lessons 5–6 on foot length; here’s the full picture, and now on foot width:

favstats(~ width, data = KidsFeet)

 min   Q1 median   Q3 max     mean        sd  n missing
 7.9 8.65      9 9.35 9.8 8.992308 0.5095843 39       0

One line, nine numbers: the smallest width (min), the 25th percentile (Q1), the middle value (median), the 75th percentile (Q3), the largest (max), the average (mean), the standard deviation (sd), the sample size (n), and how many values were blank (missing). Read it out loud: “39 kids’ feet were measured for width, averaging about 9.0 cm, with a typical spread (SD) of about half a centimeter, ranging from 7.9 to 9.8 cm.”

Put a grouping variable on the left of ~ — the exact same y ~ x grammar from Lesson 6 — to get the same nine numbers per group:

favstats(width ~ sex, data = KidsFeet)

  sex min    Q1 median    Q3 max     mean        sd  n missing
1   B 8.4 8.875   9.15 9.625 9.8 9.190000 0.4517801 20       0
2   G 7.9 8.550   8.80 9.150 9.5 8.784211 0.4935846 19       0

Boys (B, n = 20) average about 9.19 cm; girls (G, n = 19) average about 8.78 cm — a difference of roughly 0.4 cm in this sample. Whether that difference is bigger than you’d expect from chance alone is exactly the kind of question Lesson 9 teaches you to answer with a hypothesis test.

42. tally(): counting a categorical variable¶

favstats() is for numbers; tally() is for categories — counts of how many observations fall into each group. Same ~ grammar, no left-hand side needed for one variable:

tally(~ biggerfoot, data = KidsFeet)

biggerfoot
 L  R 
22 17 

22 kids have a bigger left foot, 17 a bigger right foot. Add format = "proportion" to see those same counts turned into fractions of the whole, instead of raw counts:

tally(~ biggerfoot, data = KidsFeet, format = "proportion")

biggerfoot
        L         R 
0.5641026 0.4358974 

About 56.4% of kids have a bigger left foot, 43.6% a bigger right foot — the same information as the counts, just rescaled to add up to 1.

4.1Two-way tables¶

Put a second categorical variable on the left of ~ and tally() cross-tabulates the two — the same “y broken down by x” idea from Lesson 6, applied to counting instead of averaging. Here’s a real question you can ask of this data: is a kid’s bigger foot related to which hand is dominant?

tally(biggerfoot ~ domhand, data = KidsFeet)

          domhand
biggerfoot  L  R
         L  2 20
         R  6 11

Read this like a grid: of the 8 left-handed kids (domhand = L), 2 have a bigger left foot and 6 have a bigger right foot; of the 31 right-handed kids, 20 have a bigger left foot and 11 have a bigger right foot. Adding format = "proportion" rescales each column to add up to 1, so you can compare the two hand groups on equal footing even though they have different sample sizes (8 vs. 31):

tally(biggerfoot ~ domhand, data = KidsFeet, format = "proportion")

          domhand
biggerfoot         L         R
         L 0.2500000 0.6451613
         R 0.7500000 0.3548387

Now it reads directly: 75.0% of left-handed kids have a bigger right foot, versus only 35.5% of right-handed kids. tally() scales the same way to your own imported data — here it is again on coffee from Lesson 6:

tally(~ day_type, data = coffee)

day_type
Weekday Weekend 
     12       8 

53. Plotting with gf_: the same formula, now as a picture¶

Every gf_ function (“ggformula”) uses the identical y ~ x grammar you just used for favstats() and tally(). A histogram summarizes one numerical variable’s shape:

gf_histogram(~ width, data = KidsFeet, bins = 10, fill = "#0072B2", color = "white",
             xlab = "Foot width (cm)", ylab = "Count",
             title = "Foot width of 39 kids")

What you’ll see: a roughly mound-shaped histogram of the 39 widths, with a tall bar near 8.9 cm, a smaller dip around 9.1–9.3 cm, and bars tapering off toward the low (7.9 cm) and high (9.8 cm) ends — consistent with the mean of about 9.0 and SD of about 0.5 you already saw in favstats().

fill = "#0072B2" is one color from the Okabe–Ito palette, a set of colors chosen to stay distinguishable for colorblind readers — every figure in this course uses it (or the closely related viridis palette) instead of R’s default colors.

A boxplot compares a numerical variable across groups in one compact picture — put the grouping variable on the right of ~, exactly like the grouped favstats() above:

gf_boxplot(width ~ sex, data = KidsFeet, fill = ~ sex) %>%
  gf_refine(scale_fill_manual(values = c("#0072B2", "#E69F00"))) %>%
  gf_labs(x = "Sex (B = boy, G = girl)", y = "Foot width (cm)",
          title = "Foot width by sex")

What you’ll see: two boxes side by side. The boys’ box sits noticeably higher (median line near 9.15 cm) than the girls’ box (median line near 8.80 cm), matching the group means from favstats() above; both boxes have roughly similar height (spread), and neither shows an extreme outlier point.

A bar chart counts a categorical variable — the picture version of tally():

gf_bar(~ domhand, data = KidsFeet, fill = ~ domhand) %>%
  gf_refine(scale_fill_manual(values = c("#0072B2", "#E69F00"))) %>%
  gf_labs(x = "Dominant hand", y = "Count", title = "Dominant hand of 39 kids")

What you’ll see: two bars, “L” and “R.” The “R” bar is much taller (31 kids) than the “L” bar (8 kids) — right-handed kids clearly outnumber left-handed kids in this sample, which you can also confirm with tally(~ domhand, data = KidsFeet).

Finally, a scatterplot shows the relationship between two numerical variables — one point per observation:

gf_point(width ~ length, data = KidsFeet, color = ~ sex, shape = ~ sex) %>%
  gf_refine(scale_color_manual(values = c("#0072B2", "#E69F00"))) %>%
  gf_labs(x = "Foot length (cm)", y = "Foot width (cm)",
          title = "Foot width vs. length, by sex")

What you’ll see: a cloud of 39 points trending up and to the right — as foot length increases, width tends to increase too (a positive association) — with circles marking boys and triangles marking girls, colored differently as well. Notice the shapes are also different, not just the colors — this course’s plots never rely on color alone to separate groups, so the picture still reads clearly in grayscale or for a colorblind viewer.

You can back up that “trending up” impression with a single descriptive number, cor() (the correlation coefficient — a bonus tool for this course, since MATH 1209 doesn’t test formal inference on relationships between two numerical variables):

cor(width ~ length, data = KidsFeet)

[1] 0.6410961

A correlation of about 0.64 is a moderately strong positive relationship — bigger feet tend to be both longer and wider, which is exactly what the upward-trending scatterplot showed you.

6Summary¶

• favstats(~x, data=) gives nine summary numbers for one numerical variable; favstats(y ~ group, data=) gives the same nine numbers per group.

• tally(~x, data=) counts a categorical variable; format = "proportion" rescales counts to fractions; tally(y ~ x, data=) cross-tabulates two categorical variables, and format = "proportion" there divides within each column.

• gf_histogram, gf_boxplot, gf_bar, and gf_point all reuse the y ~ x formula grammar, and load automatically with library(mosaic).

• Always describe a plot’s shape, center, and spread in a full sentence, not just a glance.

• Every plot in this course uses the Okabe–Ito colorblind-safe palette, and distinguishes groups by shape or fill (not color alone).

7Check your understanding¶

1. Using KidsFeet, write the R for the favstats() of length, broken down by domhand. (You have every piece of syntax you need from this lesson.)

2. tally(~ biggerfoot, data = KidsFeet, format = "proportion") returned 0.5641 for L. In your own words, what does that number mean, and what should the two proportions in that table add up to?

3. You want a boxplot of wait_minutes by day_type using the coffee data from this lesson. Write the gf_boxplot() call from memory, then check it against the pattern used for width ~ sex above.

4. Look back at the gf_point() scatterplot of width vs. length. In one sentence, describe its shape, and say whether the correlation you computed with cor() is consistent with what you see.