Instructor notes: Session 2

Cleaning your data

→ Open the Session 2 lesson

Timing

Part Topic Time (incl. TRY IT)
1 Set up and recall the problem list 8 min
2 The pipe %>% 8 min
3 Column names: clean_names() and rename() 10 min
4 Fixing types with mutate() 11 min
5 Fixing values with if_else() (+ 🔁 ifelse() demo) 15 min
6 Recoding categories with case_when() (2 TRY ITs) 22 min
7 Creating new variables (+ 🔁 cut() demo) 15 min
8 One pipeline and saving your work 10 min
Wrap-up and preview 5 min
Total 104 min

Short on time? This session is now the longest of the series. Do TRY IT 7 (district) together together as a live-coded demo instead of solo work, or assign it as homework. Do not cut TRY IT 9 (saving measles_clean.rds) — Session 3 starts from that file. Learners who fall behind can use the ready-made copy in processed_data/. The two 🔁 base R demos (ifelse(), cut()) can also be skipped and left for learners to read.

Before you start

  • Learners need the workshop folder open in Positron (File → Open Folder…), student_scripts/Session 2 - Cleaning_STUDENT.R open, and their measles problem list from the end of Session 1 — today’s TRY ITs work through it.
  • Today starts from the raw files (raw_data/linelist_raw.xlsx, raw_data/measles_raw.csv), so learners who missed Session 1 can still follow.
  • Sticky notes: green = done with the TRY IT, red = need help. Start the answer when most notes are green.

Teaching notes by part

Overall rhythm. Learners build two cleaned datasets side by side: linelist_clean with you (Follow along), and measles_clean in the TRY ITs. Both are built one step at a time: each part takes the current object, adds one fix, and saves it back with <-. Part 8 then shows the same linelist steps as one pipeline — the version you would keep in a real project. If a learner’s object gets into a bad state, they can re-run from Part 3 (linelist_clean <- linelist_raw %>% clean_names()). The filled-in script is solutions/Session 2 - Cleaning_SOLUTIONS.R.

Part 1 — Set up. Briefly put up the Session 1 problem list and the “Fixed in” column; emphasize never edit the raw data file.

Part 2 — The pipe. New to learners today (Session 1 deliberately used one function inside another). Start from the Session 1 code head(arrange(linelist_raw, wt_kg), 3) and read it aloud from the inside out, then type the pipe version and read it as “and then”. Point out that the data no longer goes inside each function: count(linelist_raw, hospital) becomes linelist_raw %>% count(hospital). Show the keyboard shortcut. select() is introduced here too (it was left out of Session 1). The two common slips: %>% at the start of a line, and a trailing %>% on the last line (the Console shows + and waits — press Esc).

Part 3 — Column names. Show names() before and after clean_names(). Point out that backticks are no longer needed. Stress that in rename() the new name goes on the left.

Part 4 — Types. Point out the “NAs introduced by coercion” warning when learners convert measles age_years: it is expected (the 27 “unknown” values) — a warning is not an error. The TRY IT’s summary() showing a max of 999 sets up Part 5. A common slip is forgetting the <-, so nothing is saved.

Part 5 — if_else(). Ask the class (a “Think about it” on the lesson page):

  • What would have happened in later analyses if we had skipped the months → years step? — A 6-month-old baby would be counted as 6 years old (37 ages are in months).

Explain why we replace impossible weights with NA rather than deleting the row: we keep the patient’s age, outcome, and dates.

🔁 ifelse() demo (3 min). Run both lines and let the class react to the numbers ifelse() returns. The message: base R ifelse() is everywhere in other people’s code, but it strips the Date type; if_else() is strict about types, which is why we use it. Learners only need to recognize ifelse(), not switch to it.

Part 6 — case_when(). Point out that rows matching no condition become NA (gender), and that .default catches everything else (hospital). Show count(hospital, hospital_name) as the way to check every recode. Common errors: = instead of ~, a missing or extra comma. TRY IT 7 (district) is the hardest of the session — expect red sticky notes; a hint is to run str_to_title() first so each str_detect() only needs to catch misspellings.

Part 7 — New variables. Ask the class (a “Think about it” on the lesson page):

  • A BMI of 1,250? — The heights and weights don’t fit together. Set BMIs outside 10–60 to NA.

About 1,400 BMIs (1,382) end up NA because the height and weight values in this teaching dataset are unrealistic. The old version of this workshop filtered out those rows, which removed about 20% of the cases from every later analysis. Setting only the bad value to NA keeps everything else about the patient.

For age_cat, point out that case_when() stops at the first TRUE line, and warn against ending with TRUE ~ "80+" / .default = "80+" — it puts missing ages into the oldest group. count() shows the groups in alphabetical order; that is fixed with factors in Session 5.

🔁 cut() demo (3 min). cut() is very common in epidemiology and clinical code for age groups. Show that the counts match case_when() and that the groups come out in order (a factor, a preview of Session 5). Point out right = FALSE: without it, an 18-year-old goes into “Under 18”.

Part 8 — One pipeline and saving. Point out that inside one mutate(), later lines can use columns made on earlier lines (bmi is created, then fixed). Make sure everyone saves measles_clean.rds — they need it in Session 3. Early finishers: the ⭐ Bonus asks them to rewrite all their measles steps as one pipeline.

Wrap-up. Today: clean_names(), rename(), mutate(), as.Date(), as.numeric(), if_else(), case_when(), str_detect(), export(). Preview Session 3 — Missing data, joins, and export: rows with no Case ID, duplicates, replacing missing values, joining case_info, and saving the final datasets.

TRY IT answers at a glance

The numbers match the TRY IT numbers in the lesson, the student script, and the solutions file. Follow-along prompts are numbered by part (Follow along 5.3 = Part 5, third prompt).

TRY IT Answer
1. Import the practice data measles_raw <- import(here("raw_data", "measles_raw.csv")) — 1215 rows, 11 columns
2. The pipe measles_raw %>% select(...) %>% arrange(desc(...)) %>% head(10) — two columns, top temperature 104.7
3. Clean the measles column names measles_raw %>% clean_names() — temp_c and vaccinated
4. Fix the measles types as.Date(report_date), as.numeric(age_years); 27 “unknown” become NA; max age 999
5. 999 and Fahrenheit if_else(age_years == 999, NA, age_years); if_else(temp_c > 45, round((temp_c - 32) * 5 / 9, 1), temp_c) — max age 52 (31 NA), max temp 40.9
6. Sex and vaccinated case_when() + %in% — sex: female 533, male 642, NA 40; vaccinated: No 741, Unknown 183, Yes 291
7. District str_to_title() then case_when() + str_detect() — Central 113, East 233, North 356, South 297, West 182, NA 34
8. Outcome and age groups str_to_title(outcome) — Died 52, Recovered 1095, NA 68; age_cat — Under 5 379, 5-9 368, 10-14 232, 15+ 205, NA 31
9. Save the measles data export(measles_clean, here("processed_data", "measles_clean.rds"))