Session 4: Data Structures — Vectors, Lists, Subsetting, and Reshaping

Vectors and types, vectorized calculations, base-R subsetting, lists, tidy data, and pivoting

NoteIn this session you will learn to
  1. Explain that every column of a data frame is a vector with one type
  2. Use vectorized calculations and comparisons (and explain why mutate() needs no loop)
  3. Subset vectors and data frames with base R ([ ], $, [[ ]]) and compare it with filter()/select()
  4. Explain what a list is, and pull results out of one (including the result of a statistical test)
  5. Describe tidy data, and reshape between wide and long with pivot_longer() and pivot_wider()

Before you start: open the workshop folder in Positron (File → Open Folder…) and open student_scripts/Session 4 - Data Structures_STUDENT.R. This session starts from the finished data of Session 3: processed_data/combined_linelist.rds and processed_data/measles_final.rds. Missed Session 3? Ready-made copies of both files are already in processed_data/.

Where this session fits. Today covers Reshape (highlighted): changing a table between wide and long. First, you look under the hood at how R stores data (vectors, types, and lists), which explains why the tidyverse functions from Sessions 1–3 work the way they do.

  1. Import
  2. Inspect
  3. Clean
  4. Combine
  5. Reshape
  6. Summarize
  7. Visualize
  8. Save and share
  9. Automate

Sessions 1–3 used tidyverse functions that hide how R stores data. Today you look under the hood: vectors and vectorization explain why mutate() works on a whole column at once, base-R subsetting is everywhere in help pages and other people’s code (and you will need it to write functions in Session 8), lists are how R stores test and model results, and reshaping is needed for many summaries and plots.

Part 1: Set up

library(tidyverse)
library(here)
library(rio)

linelist <- import(here("processed_data", "combined_linelist.rds"), trust = TRUE)
Note🔍 What this code does
  • library() loads today’s three packages: tidyverse (which includes pivot_longer() and pivot_wider()), here, and rio.
  • import() reads combined_linelist.rds, the finished linelist you saved in Session 3. .rds files can contain R code, so rio asks you to confirm that you trust the file with trust = TRUE.
  • linelist has 6451 rows (patients) and 19 columns.
Important✏️ TRY IT 1 (2 min): import the measles data

Import processed_data/measles_final.rds (with trust = TRUE) and call it measles.

✅ Check: in the Variables pane, measles shows 1163 rows and 14 columns

measles <- import(here("processed_data", "measles_final.rds"), trust = TRUE)
dim(measles)
[1] 1163   14

The cleaned measles data from Session 3 has 1163 rows (cases) and 14 columns. You can see this in the Variables pane too.

Part 2: Vectors and types

A vector is a set of values of the same type. Every column of a data frame is a vector. $ pulls one column out:

ages <- linelist$age_years
head(ages)
[1]  2  3 56 18  3 16
length(ages)
[1] 6451

length() tells you how many values a vector has: 6451, one per row of linelist.

Note🔍 What this code does
  • linelist$age_years pulls the age_years column out of the data frame as a vector, and ages <- saves it as an object of its own.
  • head(ages) shows the first 6 values: 2, 3, 56, 18, 3, 16.
  • length(ages) counts the values: 6451, the same as the number of rows. A column always has one value per row, even if some of them are NA.

The main types

Type Example class() says
Number 38.5, 2 "numeric"
Text "male", "Recover" "character"
TRUE/FALSE TRUE, FALSE "logical"
Date as.Date("2014-05-13") "Date"
Factor categories with an order "factor"
class(linelist$age_years)
[1] "numeric"
class(linelist$gender)
[1] "character"
class(linelist$onset_date)
[1] "Date"
Note🔍 What this code does
  • class() tells you what type of vector something is.
  • age_years is "numeric" (numbers you can calculate with), gender is "character" (text), and onset_date is "Date". In the raw data, ages and some dates were text; the cleaning in Session 2 gave them the right types.

One type per vector

If you mix types, R converts everything to the most flexible type — usually text. This is why a few “unknown” values turned the measles Age (years) column into text in Session 1:

c(1, 2, "unknown")
[1] "1"       "2"       "unknown"
c(1, TRUE, FALSE)    # TRUE becomes 1, FALSE becomes 0
[1] 1 1 0
Note🔍 What this code does
  • c(1, 2, "unknown") mixes numbers and text, so R turns the numbers into text: "1" "2" "unknown". The quotes in the output show they’re text now, and you can no longer calculate with them.
  • c(1, TRUE, FALSE) mixes numbers and TRUE/FALSE. Numbers are the more flexible type, so TRUE becomes 1 and FALSE becomes 0: 1 1 0.
  • From least to most flexible: logical (TRUE/FALSE), then numeric, then character. R always picks the most flexible type in the vector.

Missing values spread

Any calculation with NA gives NA, unless you tell R to skip them:

mean(c(38, 39, NA))
[1] NA
mean(c(38, 39, NA), na.rm = TRUE)
[1] 38.5
Note🔍 What this code does
  • mean(c(38, 39, NA)) gives NA. One value is unknown, so the mean is unknown too: R tells you instead of guessing.
  • na.rm = TRUE (“NA remove”) tells mean() to skip the missing values: the mean of 38 and 39 is 38.5.
  • The same argument works in sum(), min(), max(), median(), and sd().
Important✏️ TRY IT 2 (3 min): types in the measles data

Use class() on measles$age_years, measles$vaccinated, and measles$report_date, and length() on measles$temp_c.

✅ Check: “numeric”, “character”, “Date”, and 1163

class(measles$age_years)
[1] "numeric"
class(measles$vaccinated)
[1] "character"
class(measles$report_date)
[1] "Date"
length(measles$temp_c)
[1] 1163

Ages are numbers, vaccination status is text, and the report date is a real date. The length is 1163 — one value per row, because a column is just a vector.

Note📖 Good to know
  • NA and “Unknown” are different. NA means the value is missing, and R treats it specially: calculations with it give NA. “Unknown” is ordinary text, which R counts like any other value. In this data, unknown symptoms are written “Unknown”, so fever has no NAs.
  • str() describes any object: its type, its size, and its first values. For a data frame it works like glimpse().

Part 3: Vectorization

R does most calculations on every element of a vector at once. No loop needed:

temps_c <- c(36.8, 38.5, 40.1)
temps_c * 9 / 5 + 32
[1]  98.24 101.30 104.18
Note🔍 What this code does
  • temps_c * 9 / 5 + 32 applies the Celsius-to-Fahrenheit formula to all three values at once: 98.24, 101.30, and 104.18.
  • temps_c itself doesn’t change; the result only prints. Save it with <- to keep it.

That is exactly what happens inside mutate(): the calculation runs on the whole column.

# Weight in pounds for every patient, in one line
head(linelist$wt_kg * 2.2)
[1]  59.4  55.0 200.2  90.2  79.2 123.2
Note🔍 What this code does
  • linelist$wt_kg * 2.2 multiplies every one of the 6451 weights by 2.2 (kilograms to pounds) in one step. head() shows the first 6: 59.4, 55.0, 200.2, and so on.
  • mutate(wt_lb = wt_kg * 2.2) does exactly the same inside the data frame and saves it as a new column.

Comparisons are vectorized too

A comparison gives a logical vector, one TRUE/FALSE per element:

temps_c > 38
[1] FALSE  TRUE  TRUE

TRUE counts as 1 and FALSE as 0, so:

  • sum() of a comparison counts the TRUEs
  • mean() of a comparison gives the proportion that are TRUE
sum(linelist$temp > 38, na.rm = TRUE)    # how many patients had temp above 38
[1] 4943
mean(linelist$temp > 38, na.rm = TRUE)   # what proportion
[1] 0.7846032
mean(linelist$fever == "yes")            # proportion with fever
[1] 0.7870098
Note🔍 What this code does
  • temps_c > 38 compares each value with 38 and gives one TRUE or FALSE per value: FALSE TRUE TRUE.
  • sum(linelist$temp > 38, na.rm = TRUE) counts the TRUEs: 4943 patients had a temperature above 38 °C. na.rm = TRUE skips the 151 patients with no temperature.
  • mean() of the same comparison gives the proportion: 0.785, so about 78% of the patients with a recorded temperature had one above 38 °C.
  • mean(linelist$fever == "yes") is the proportion of all patients with fever: 0.787. fever has no missing values (unknowns are written “Unknown”), so no na.rm is needed.

You will use this trick again in Session 5: mean(fever == "yes") * 100 inside summarise() gives the percentage with fever in each group.

Important✏️ TRY IT 3 (4 min): vectorized calculations
  1. Convert measles$temp_c to Fahrenheit and look at the first values with head()
  2. Count the cases with a temperature of 39 or more: sum(... >= 39)
  3. Find the proportion of cases who were vaccinated: mean(... == "Yes")

✅ Check: a) starts 100.94, 101.12 b) 565 c) 0.238

head(measles$temp_c * 9 / 5 + 32)    # a)
[1] 100.94 101.12 102.74 100.40 102.02 101.30
sum(measles$temp_c >= 39)            # b)
[1] 565
mean(measles$vaccinated == "Yes")    # c)
[1] 0.2381771

565 cases had a temperature of 39 °C or more, and 23.8% of cases were vaccinated. No na.rm = TRUE is needed here because temp_c has no missing values after cleaning.

Note📖 Good to know
  • Why no loop? In many programming languages you’d write a loop to go through the values one at a time. In R, vectorized code is shorter and much faster, because the loop runs inside R’s own built-in code. Loops still have their uses (Session 8).
  • The functions you used in Session 2, such as as.Date(), if_else(), case_when(), and str_detect(), are vectorized too. That’s why they worked on whole columns.

Part 4: Subsetting with base R

Subsetting means pulling out part of a vector or data frame. The tidyverse does this with filter() and select(); base R uses square brackets. You will see base R in help pages, older code, and when writing functions (Session 8).

Vectors: x[ ]

temps_c[1]            # first element
[1] 36.8
temps_c[c(1, 3)]      # first and third
[1] 36.8 40.1
temps_c[-1]           # everything EXCEPT the first
[1] 38.5 40.1
temps_c[temps_c > 38] # elements that meet a condition
[1] 38.5 40.1
Note🔍 What this code does
  • temps_c[1] gives the first value, 36.8. Positions in R start at 1.
  • temps_c[c(1, 3)] gives the first and third values: 36.8 and 40.1.
  • temps_c[-1] drops the first value: 38.5 and 40.1.
  • In temps_c[temps_c > 38], the comparison inside the brackets gives FALSE TRUE TRUE, and the brackets keep the values where it’s TRUE: 38.5 and 40.1.

Data frames: df[rows, columns]

Inside the brackets, rows come first, then columns, separated by a comma. Leave one side empty to keep everything:

linelist[1:3, c("case_id", "age_years", "outcome")]   # rows 1–3, three columns
  case_id age_years outcome
1  5fe599         2 Unknown
2  8689b7         3 Recover
3  11f8ea        56 Recover
linelist[1:3, ]                                       # rows 1–3, all columns
linelist[, "age_years"]                               # all rows, one column
Note🔍 What this code does
  • In linelist[1:3, c("case_id", "age_years", "outcome")], the part before the comma chooses rows (1:3 means rows 1, 2, and 3) and the part after it chooses columns, by name.
  • 1:3 is short for c(1, 2, 3).
  • In the second block, an empty side keeps everything: linelist[1:3, ] is rows 1–3 with all 19 columns, and linelist[, "age_years"] is every row of one column.

One column: $ and [[ ]]

linelist$hospital_name[1:3]        # $ with a name you type
[1] "Other/Unknown"                 "Other/Unknown"                
[3] "St. Mark's Maternity Hospital"
linelist[["hospital_name"]][1:3]   # [[ ]] with a name in quotes — useful inside functions
[1] "Other/Unknown"                 "Other/Unknown"                
[3] "St. Mark's Maternity Hospital"
Note🔍 What this code does
  • linelist$hospital_name and linelist[["hospital_name"]] both pull out the whole column as a vector, and [1:3] then keeps its first three values.
  • With $ you type the name itself. [[ ]] takes the name as text, so the name can also come from an object: col <- "hospital_name" and then linelist[[col]]. That’s why [[ ]] is the one to use inside your own functions (Session 8).

Conditions

# Mean temperature of patients who died
mean(linelist$temp[linelist$outcome == "Death"], na.rm = TRUE)
[1] 38.59154
Note🔍 What this code does
  • linelist$outcome == "Death" gives TRUE for every patient who died.
  • linelist$temp[...] keeps the temperatures where that is TRUE: the temperatures of the patients who died.
  • mean(..., na.rm = TRUE) averages them, skipping missing temperatures: 38.6 °C. In Session 5 you’ll do the same with filter() and summarise().
WarningBase R and missing values
nrow(linelist[linelist$temp > 40, ])   # 244 rows?!
[1] 244
nrow(filter(linelist, temp > 40))     # 93 rows
[1] 93

Base R returns an extra row of NAs for every missing temperature (151 of them). filter() drops rows where the condition is NA. With base R, add & !is.na(temp), or just use filter():

nrow(linelist[linelist$temp > 40 & !is.na(linelist$temp), ])   # 93: fixed
[1] 93
Note🔍 What this code does
  • linelist$temp > 40 is NA for the 151 patients with no temperature. Inside [ ], each NA becomes a row filled with NAs, so base R returns 244 rows (93 + 151) instead of 93.
  • filter() keeps only the rows where the condition is TRUE, so it drops the NAs and returns 93.
  • & !is.na(linelist$temp) adds a second condition, “and the temperature isn’t missing”, which turns those NAs into FALSE.

Base R vs tidyverse

Task Base R Tidyverse
Rows that meet a condition df[df$age > 40, ] filter(df, age > 40)
Some columns df[, c("a", "b")] select(df, a, b)
One column as a vector df$a or df[["a"]] pull(df, a)
First rows df[1:5, ] slice(df, 1:5) or head(df, 5)
WarningCommon mistakes
  • Forgetting the comma: df[1:3] picks columns 1–3, not rows!
  • Base R adds a row of NAs for every NA in the condition (see the box above). Add & !is.na(...), or use filter().
Important✏️ TRY IT 4 (5 min): subsetting the measles data

With base R,

  1. Show rows 1–5 of the patient_id and temp_c columns of measles
  2. Find the mean temperature of children who died (outcome == "Died")
  3. Count the cases aged 15 or older with nrow(measles[...]), then with filter(). Why are the answers different? Fix the base R version.

✅ Check: a) the first ID is MS-0931 b) 38.75 c) 225 vs 196 (29 missing ages)

# a)
measles[1:5, c("patient_id", "temp_c")]
  patient_id temp_c
1    MS-0931   38.3
2    MS-0113   38.4
3    MS-0975   39.3
4    MS-0297   38.0
5    MS-0159   38.9
# b)
mean(measles$temp_c[measles$outcome == "Died"])
[1] 38.754
# c)
nrow(measles[measles$age_years >= 15, ])
[1] 225
nrow(filter(measles, age_years >= 15))
[1] 196
nrow(measles[measles$age_years >= 15 & !is.na(measles$age_years), ])   # fixed
[1] 196

Base R gives 225 rows because it adds a row of NAs for each of the 29 cases with a missing age. Adding & !is.na(measles$age_years) gives 196, the same as filter().

Note📖 Good to know
  • Where you’ll meet base R: help-page examples (?mean), answers on Stack Overflow, older scripts, and many Bioconductor packages. Being able to read it is the main skill; you can keep writing tidyverse code.
  • Out-of-range positions don’t give an error: temps_c[10] gives NA, because there is no tenth value. If you see unexpected NAs after subsetting, check your positions.

Part 5: Lists

A vector holds values of one type. A list can hold anything: numbers, text, dates, whole data frames, even other lists, and each element can have a different length.

patient <- list(
  id       = "5fe599",
  age      = 2,
  symptoms = c("fever", "cough"),
  visits   = as.Date(c("2014-05-13", "2014-05-15"))
)
patient
$id
[1] "5fe599"

$age
[1] 2

$symptoms
[1] "fever" "cough"

$visits
[1] "2014-05-13" "2014-05-15"

Pulling things out

patient$symptoms     # $ with a name you type
[1] "fever" "cough"
patient[["age"]]     # [[ ]] with a name in quotes (or a position)
[1] 2
patient["age"]       # [ ] gives back a smaller LIST, not the value
$age
[1] 2
length(patient)      # 4 elements
[1] 4
str(patient)         # the structure: what is in each element
List of 4
 $ id      : chr "5fe599"
 $ age     : num 2
 $ symptoms: chr [1:2] "fever" "cough"
 $ visits  : Date[1:2], format: "2014-05-13" "2014-05-15"
Note🔍 What this code does
  • list() stores four different things together, each with a name: a piece of text, a number, two symptoms, and two dates.
  • patient$symptoms and patient[["age"]] take one element out: the vector itself.
  • patient["age"] gives back a list with one element in it. The output shows the difference: it starts with $age. You can’t calculate with it: patient["age"] + 1 gives an error.
  • length(patient) counts the elements: 4. str() shows one line per element with its type and size: chr, num, chr [1:2], and Date[1:2].

Think of a list as a box of items: [[ ]] takes an item out of the box; [ ] gives you a smaller box.

A data frame is a list

A data frame is a list of columns that all have the same length:

is.list(linelist)
[1] TRUE
length(linelist)     # the number of columns
[1] 19
Note🔍 What this code does
  • is.list(linelist) is TRUE: a data frame is a list whose elements are its columns.
  • So length(linelist) counts the columns, 19, not the rows. Use nrow() for the number of rows.

That is why $ and [[ ]] work on data frames too, and why map() in Session 8 runs once for each column.

Results are lists

Statistical tests and models return lists. You don’t need to know the statistics yet; look at the shape of the result:

age_test <- t.test(age_years ~ gender, data = filter(linelist, gender != "Unknown"))
age_test

    Welch Two Sample t-test

data:  age_years by gender
t = -21.98, df = 5390.7, p-value < 2.2e-16
alternative hypothesis: true difference in means between group female and group male is not equal to 0
95 percent confidence interval:
 -7.441575 -6.222858
sample estimates:
mean in group female   mean in group male 
            12.68343             19.51564 

The printout is a friendly view for people. Underneath, it is a list your code can use:

names(age_test)
 [1] "statistic"   "parameter"   "p.value"     "conf.int"    "estimate"   
 [6] "null.value"  "stderr"      "alternative" "method"      "data.name"  
age_test$p.value
[1] 1.271186e-102
age_test$estimate
mean in group female   mean in group male 
            12.68343             19.51564 
Note🔍 What this code does
  • t.test(age_years ~ gender, ...) compares the mean age of the two genders. Read ~ as “by”: age by gender.
  • data = filter(linelist, gender != "Unknown") gives t.test() only the rows with a known gender, because it needs exactly two groups.
  • age_test is a list. names(age_test) lists its 10 elements, such as p.value, estimate, and conf.int.
  • age_test$p.value takes out the p-value as a plain number: 1.271186e-102, R’s way of writing 1.27 × 10-102 (extremely small). age_test$estimate gives the two means: 12.7 years (female) and 19.5 years (male).

Every test and model in R works this way (chisq.test(), lm(), glm(), survival models, Bioconductor objects), so this is how you get one number out of an analysis and into a table or a report.

WarningCommon mistakes
  • patient["age"] + 1 gives an error: [ ] returns a list. Use patient[["age"]] or patient$age.
  • t.test() compares exactly two groups. “grouping factor must have exactly 2 levels” means there is a third group (like “Unknown”): filter() it out first.
Important✏️ TRY IT 5 (5 min): a test result is a list
  1. Run t.test() on temp_c by vaccinated in measles, leaving out the “Unknown” vaccination status. Save it as temp_test.
  2. Pull out the p-value and the two group means with $.

✅ Check: p-value 0.565; means 38.80 (No) and 38.84 (Yes)

temp_test <- t.test(temp_c ~ vaccinated, data = filter(measles, vaccinated != "Unknown"))   # a)

temp_test$p.value    # b)
[1] 0.5645313
temp_test$estimate
 mean in group No mean in group Yes 
         38.80480          38.83682 

Vaccinated and unvaccinated children had almost the same temperature (38.80 vs 38.84 °C), and a p-value of 0.565 means the small difference is easily explained by chance.

Note📖 Good to know
  • When you get an object you don’t recognize (from a test, a model, or someone else’s code), try str() and names() first. They show what’s inside and what you can pull out with $.
  • The broom package (installed with the tidyverse) turns test and model results into data frames: broom::tidy(age_test) gives a one-row table with the means and the p-value, ready for a report.

Part 6: Tidy data and pivot_longer()

Tidy data

Data is tidy when:

  1. Each variable is a column
  2. Each observation is a row
  3. Each value is a cell

The tidyverse works best on tidy data. But “observation” depends on your question. Our symptoms are in wide format — one column per symptom:

linelist %>% select(case_id, fever:vomit) %>% head(3)
  case_id   fever  chills   cough   aches   vomit
1  5fe599      no      no     yes      no     yes
2  8689b7 Unknown Unknown Unknown Unknown Unknown
3  11f8ea Unknown Unknown Unknown Unknown Unknown
Note🔍 What this code does
  • select(case_id, fever:vomit) keeps the ID and the five symptom columns: fever, chills, cough, aches, and vomit. head(3) shows the first 3 patients.
  • Each symptom has its own column, with “yes”, “no”, or “Unknown” in each cell. This is wide format.

To count or plot all symptoms together, we want long format: one row per patient per symptom.

pivot_longer(): wide → long

symptoms_long <- linelist %>%
  select(case_id, fever:vomit) %>%
  pivot_longer(
    cols      = fever:vomit,   # the columns to stack
    names_to  = "symptom",     # new column for the old column NAMES
    values_to = "present"      # new column for the VALUES
  )

head(symptoms_long)
# A tibble: 6 × 3
  case_id symptom present
  <chr>   <chr>   <chr>  
1 5fe599  fever   no     
2 5fe599  chills  no     
3 5fe599  cough   yes    
4 5fe599  aches   no     
5 5fe599  vomit   yes    
6 8689b7  fever   Unknown
nrow(symptoms_long)   # 6451 patients × 5 symptoms
[1] 32255

fever:vomit means “every column from fever to vomit”.

Tip🧩 Reading a function call

Here is the key call of this session, taken apart:

pivot_longer(cols = fever:vomit, names_to = "symptom", values_to = "present")

Piece What it is What it does
pivot_longer(...) a function from the tidyr package (part of the tidyverse) reshapes wide data into long data
the data arrives through the pipe %>% as the first argument, so you don’t type it the table to reshape
cols = fever:vomit an argument: which columns to stack the five columns from fever to vomit
names_to = "symptom" an argument: the name of a new column, in quotes will hold the old column names (fever, chills, …)
values_to = "present" an argument: the name of another new column will hold the values from the old cells (yes, no, Unknown)

Because every argument is named, you can read the call like a sentence: “pivot longer the columns fever to vomit; put their names into symptom and their values into present.” The new column names are in quotes because those columns don’t exist yet. fever:vomit isn’t, because those columns already exist.

Note🔍 What this code does
  • select() keeps the ID and the five symptoms, as before.
  • pivot_longer() stacks the five symptom columns, so each patient now has five rows, one per symptom.
  • The old column names (fever, chills, …) go into the new symptom column, and the yes/no values go into the new present column.
  • case_id isn’t in cols, so it’s repeated on each of the patient’s five rows.
  • nrow() confirms the size: 6451 patients × 5 symptoms = 32255 rows.

Now one count() answers “how many patients had each symptom?”:

symptom_counts <- symptoms_long %>%
  filter(present == "yes") %>%
  count(symptom)

symptom_counts
# A tibble: 5 × 2
  symptom     n
  <chr>   <int>
1 aches     604
2 chills   1209
3 cough    5347
4 fever    5077
5 vomit    3095

And it’s ready to plot (a preview of Session 6):

ggplot(symptom_counts, aes(x = n, y = fct_reorder(symptom, n))) +
  geom_col(fill = "#2a78d6") +
  labs(title = "Patients reporting each symptom", x = "Number of patients", y = NULL) +
  theme_minimal()

Horizontal bar chart of the number of patients reporting each symptom, longest at the top: cough (about 5,300), fever (about 5,100), vomiting (about 3,100), chills (about 1,200), and aches (about 600).

Note🔍 What this code does
  • filter(present == "yes") keeps only the rows where the patient had that symptom.
  • count(symptom) counts the rows left for each symptom: cough 5347, fever 5077, vomit 3095, chills 1209, and aches 604.
  • Because the data is long, one count() does what would take five separate counts in the wide data.
  • ggplot() draws one bar per symptom: geom_col() uses n as the bar length, and fct_reorder(symptom, n) orders the bars by length, so the longest is at the top. You’ll learn plotting in Session 6.
Important✏️ TRY IT 6 (5 min): measles symptoms long

Select patient_id, fever, and rash from measles, then pivot_longer() the two symptom columns. count() symptom and present.

Pattern: pivot_longer(cols = c(COLUMN_1, COLUMN_2), names_to = "symptom", values_to = "present")

✅ Check: 2326 rows (1163 × 2); fever yes 1064, rash yes 976

measles_symptoms <- measles %>%
  select(patient_id, fever, rash) %>%
  pivot_longer(
    cols      = c(fever, rash),
    names_to  = "symptom",
    values_to = "present"
  )

nrow(measles_symptoms)
[1] 2326
measles_symptoms %>% count(symptom, present)
# A tibble: 4 × 3
  symptom present     n
  <chr>   <chr>   <int>
1 fever   no         99
2 fever   yes      1064
3 rash    no        187
4 rash    yes       976

Each of the 1163 cases now has two rows, one for fever and one for rash, so there are 2326 rows. One count() gives the yes/no totals for both symptoms.

Note📖 Good to know
  • How to tell wide from long: if some column names are really values of a variable (fever, chills, cough, or years like 2019, 2020, 2021), the data is wide. Spreadsheets are usually wide; count(), ggplot(), and most tidyverse functions work best on long data.

Part 7: pivot_wider()

The opposite: long → wide. Most useful for tables people read. count() gives long output:

linelist %>% count(hospital_name, outcome)
                   hospital_name outcome    n
1               Central Hospital   Death  193
2               Central Hospital Recover  165
3               Central Hospital Unknown   96
4                     HMO Health   Death  124
5                     HMO Health Recover   94
6                     HMO Health Unknown   62
7              Military Hospital   Death  399
8              Military Hospital Recover  309
9              Military Hospital Unknown  188
10                 Other/Unknown   Death 1006
11                 Other/Unknown Recover  804
12                 Other/Unknown Unknown  544
13               Premiere Health   Death  129
14               Premiere Health Recover   81
15               Premiere Health Unknown   64
16       Regional Medical center   Death  790
17       Regional Medical center Recover  579
18       Regional Medical center Unknown  402
19 St. Mark's Maternity Hospital   Death  199
20 St. Mark's Maternity Hospital Recover  126
21 St. Mark's Maternity Hospital Unknown   97

pivot_wider() spreads it into a cross-table:

linelist %>%
  count(hospital_name, outcome) %>%
  pivot_wider(
    names_from  = outcome,   # values that become the new column NAMES
    values_from = n          # values that fill the cells
  )
# A tibble: 7 × 4
  hospital_name                 Death Recover Unknown
  <chr>                         <int>   <int>   <int>
1 Central Hospital                193     165      96
2 HMO Health                      124      94      62
3 Military Hospital               399     309     188
4 Other/Unknown                  1006     804     544
5 Premiere Health                 129      81      64
6 Regional Medical center         790     579     402
7 St. Mark's Maternity Hospital   199     126      97
Note🔍 What this code does
  • count(hospital_name, outcome) gives one row per combination of hospital and outcome: 7 hospitals × 3 outcomes = 21 rows. This is long format.
  • pivot_wider() turns it around: the values of outcome (Death, Recover, Unknown) become column names, and the counts in n fill the cells.
  • The result has one row per hospital (7 rows), which is much easier to read as a table.

values_fill = 0 puts 0 instead of NA where a combination doesn’t exist.

NoteLong for R, wide for people

Keep data long for analysis and plotting; make it wide at the end for a table someone will read.

Important✏️ TRY IT 7 (4 min): a district × vaccination table

count() district and vaccinated in measles, then pivot_wider() so each vaccination status is a column.

✅ Check: 6 rows (5 districts + Unknown); North has No 205, Unknown 49, Yes 88

measles %>%
  count(district, vaccinated) %>%
  pivot_wider(
    names_from  = vaccinated,
    values_from = n
  )
# A tibble: 6 × 4
  district    No Unknown   Yes
  <chr>    <int>   <int> <int>
1 Central     70      17    23
2 East       134      40    49
3 North      205      49    88
4 South      176      36    72
5 Unknown     21       4     9
6 West       103      31    36

The values of vaccinated (No, Unknown, Yes) become the column names, and the counts fill the cells: one row per district.

Tip⭐ Bonus
  1. Turn the district × vaccination table into percentages: before pivot_wider(), group_by(district) and add mutate(pct = round(n / sum(n) * 100, 1)), then use values_from = pct.
  2. pivot_longer() the fever and rash columns, keep the “yes” rows, and plot the count of each symptom by vaccinated with geom_bar(position = "dodge").
  3. Brought your own data? Is it wide or long? Reshape it the other way.

The code for 1 and 2 is in the solutions file.

Key points

  • Every column of a data frame is a vector, and every vector has one type — check it with class(). Mixing types turns everything into text.
  • Any calculation with NA gives NA unless you add na.rm = TRUE.
  • Calculations and comparisons are vectorized: they run on every element at once, which is why mutate() needs no loop.
  • sum() of a comparison counts the TRUEs; mean() gives the proportion.
  • Base R subsets with x[ ], df[rows, columns], df$col, and df[["col"]] — but watch out for the extra NA rows that filter() would drop.
  • A list holds anything. [[ ]] and $ take one element out; [ ] returns a smaller list. A data frame is a list of columns, and test and model results are lists: result$p.value.
  • Tidy data has one variable per column and one observation per row. pivot_longer() makes data long (for analysis and plots); pivot_wider() makes it wide (for tables people read).

Next: Session 5 — Summarizing. We summarize data with count(), group_by() + summarise(), and tabyl(), and build a publication-ready Table 1.