my_age <- 35
my_age[1] 35
my_age + 5[1] 40
Objects, functions, packages, and project folders, then your first dataset
here() makes file paths work on any computerimport() and here()Before you start: complete the Setup and open student_scripts/Session 1 - R Basics, Importing, and Inspecting_STUDENT.R.
Where this session fits. Most data projects go through the same steps, whatever the topic. Today covers the first two, Import and Inspect (highlighted): getting the data into R, and looking at it carefully before changing anything. You could do these steps in Excel or SPSS too. What R adds is that every step is written down in a script, so you (or a colleague) can check it, rerun it on next month’s data, and share it.
Before we open any data, a few building blocks you’ll use in every line of R. Type these in your script and run each line with Cmd + Return (Mac) or Ctrl + Enter (Windows); the result appears in the Console.
<-<- stores a value in an object so you can use it later. Read it as “gets”. The object appears in the Variables pane.
👀 Follow along 1.1 — Store a number in an object called my_age, then add 5 to it.
Pattern: OBJECT_NAME <- VALUE
my_age <- 35 creates an object called my_age and stores the number 35 in it. Nothing prints: saving is silent. my_age now appears in the Variables pane.my_age on its own prints what is stored: [1] 35. The [1] means “this is the first value”. It helps you keep your place when the output runs over several lines.my_age + 5 uses the object in a calculation and prints 40. my_age itself is still 35: to change it, you would save again with <-.c() combines values into a vector. Every value in a vector has the same type.
👀 Follow along 1.2 — Make a vector of ages with c(), then find the mean().
c(2, 18, 56, 61) combines four numbers into one vector, and ages <- saves it.mean(ages) gives the whole vector to mean(), which adds the values and divides by how many there are: (2 + 18 + 56 + 61) / 4 = 34.25.👀 Follow along 1.3 — Make a vector of hospital names (text needs quotes!) and find its length().
Central and give an error.length(hospitals) counts the values in the vector: 3. It counts values, not letters.A function does something to its inputs (arguments): function_name(argument1, argument2, ...). Arguments can be named.
👀 Follow along 1.4 — Use round(3.14159) with and without digits = 2.
round(3.14159) has one argument, the number. The digits argument wasn’t given, so round() uses its default, 0, and rounds to a whole number: 3.round(3.14159, digits = 2) adds a second, named argument and gives 3.14.?round) lists every argument and its default.Type ?round in the Console to open the help page for any function.
A data frame is a table: each column is a variable (a vector), each row is an observation. (Later you’ll also see tibbles: the tidyverse version of a data frame, which prints more neatly.)
👀 Follow along 1.5 — Make a small data frame with data.frame(), then pull out one column with $.
Pattern: data.frame(COLUMN_1 = c(...), COLUMN_2 = c(...))
name age
1 Ana 34
2 Ben 51
3 Cho 27
[1] 34 51 27
data.frame() builds a table from vectors. Each column_name = c(...) becomes one column: name holds text and age holds numbers.patients$age pulls out the age column as a vector: 34 51 27. You can give it to any function, for example mean(patients$age).Age and age are different"Central Hospital"); object names do not (my_age)( needs a ) — Positron highlights the matching pair# starts a comment; R ignores the rest of the line<- or =? Both can store a value, but this course always uses <-, so that = only ever means “name an argument” (as in digits = 2).c(5, 12, "unknown") is three pieces of text. You’ll see this in the practice data, where a few ages say “unknown”.A package is a collection of functions someone else wrote. There are two steps:
| Step | Function | How often | Analogy |
|---|---|---|---|
| Install | install.packages() |
Once per computer | Buying a book |
| Load | library() |
Every time you open R | Taking it off the shelf |
Install everything for the whole workshop series (skip this if you did the Setup):
Load only what you need today:
👀 Follow along 2.1 — Load today’s packages, one library() line each. ✅ You should see no line that starts with “Error”.
Pattern: library(PACKAGE_NAME)
library() line makes one package’s functions available until you close Positron or restart R. That’s why every session starts with these lines.library(tidyverse) loads a whole family of packages at once (dplyr, ggplot2, tidyr, readr, stringr, forcats, lubridate, tibble, and purrr) and lists them.filter(). From now on, filter() means the tidyverse version. This is normal, not an error.# is a comment: a note for people. R ignores it.install.packages() but not in library().install.packages() downloads from there.library(tidyverse) loads several of them.here()raw_data/, processed_data/, student_scripts/)cwml-r-workshop folder, and click Open. Positron uses that folder as the working directoryraw_data, student_scripts, …)Make a new folder for each project (with subfolders like raw_data/ and scripts/), and open it in Positron with File → Open Folder…. Add an empty file called .here at the top so here() always finds it.
"/Users/mwegener/Desktop/maybe_here/data.csv" — only works on one computerhere("raw_data", "data.csv") — works on any computer that has the project folderhere() finds the top of the project folder (it looks for a .here file, which the workshop folder has) and builds the path from there.
👀 Follow along 3.1 — Find the top of the project folder.
Pattern: here::here()
here::here() prints the full path to the top of the project folder: wherever you saved cwml-r-workshop on your computer. It finds the top by looking for the .here file.👀 Follow along 3.2 — Build the path to raw_data/linelist_raw.xlsx.
Pattern: here::here("FOLDER", "FILE_NAME.EXTENSION")
here::here("raw_data", "linelist_raw.xlsx") adds a folder and a file name to that path, joined with /. R uses / on every computer, Windows included.import(), next.here::here() means “the here() function from the here package”. The :: lets you use a function before loading its package.
setwd("C:/Users/..."). That path only exists on one computer, so the script breaks everywhere else. A project folder with here() avoids this.here() finds the top of the project by looking for a marker: a .here file (the workshop folder has one), an .Rproj file, or a .git folder.import() + here()rio::import() reads almost any file type (.csv, .xlsx, .txt, SPSS, Stata, SAS, …). It works out the type from the file extension, so you only need to learn one function. Save the result with <-, or it only prints to the Console and is gone.
👀 Follow along 4.1 — Import raw_data/linelist_raw.xlsx and call it linelist_raw.
Pattern: NEW_NAME <- import(here("FOLDER", "FILE_NAME.EXTENSION"))
Here is the line you just ran, taken apart:
linelist_raw <- import(here("raw_data", "linelist_raw.xlsx"))
| Piece | What it is | What it does |
|---|---|---|
linelist_raw |
the name of a new object | holds the result |
<- |
the assignment arrow | saves the result in linelist_raw |
import(...) |
a function from the rio package | reads a file into R |
here(...) |
a function from the here package, used as the argument of import() |
builds the full path to the file |
"raw_data", "linelist_raw.xlsx" |
two text arguments of here() |
the folder, then the file name |
R runs it from the inside out: first here() builds the path, then import() reads the file at that path, and finally <- saves the result. Reading code aloud in the same order helps: “linelist_raw gets: import the file at here, raw_data, linelist_raw.xlsx.”
import() saw the .xlsx ending and read the file as an Excel workbook (the first sheet, unless you ask for another).linelist_raw appears in the Variables pane with 6604 rows and 12 columns.patients in Part 1, only bigger. Each row is one patient (a case), and each column is one piece of information about them.The same function works for other file types:
👀 Follow along 4.2 — Import raw_data/case_info.csv (case_info) and raw_data/hosp_info.txt (hosp_info).
import() chooses how to read each file from its ending. .csv is comma-separated values; this .txt file is separated by tabs.case_info has 5888 rows and 7 columns of extra details about each case. We join it to the linelist in Session 3.hosp_info is a small look-up table (6 rows): the correct name and some facts about each hospital. We use it in Session 2 to fix the hospital names.read.csv() — older, slower, and needs more settingsread_csv() — the tidyverse version, fast and returns a tibbleread_sav(), read_dta(), read_sas() — SPSS, Stata, SAS filesimport(here("raw_data", "file.xlsx"), which = "Sheet2")import() passes the work to a specialist package based on the file ending: readxl for Excel, data.table for CSV and text files, and haven for SPSS, Stata, and SAS. You only need to remember one function.import() returns a plain data frame, which prints every row. That’s why this lesson wraps long output in head(). Tidyverse functions return a tibble, a data frame that prints only the first 10 rows.<dttm>. One column, date.onset, arrives as text: Excel files often hold surprises like this, which is why you inspect before you clean.Always look before you clean. Inspecting tells you whether the data imported correctly and what needs fixing.
👀 Follow along 5.1 — Size and structure of linelist_raw: view(), dim(), names(), head(), and glimpse() (the next three code blocks).
[1] 6604 12
[1] "Case ID" "infection date" "date.onset" "hosp date"
[5] "date of outcome" "OUTCOME" "Gender" "hospital"
[9] "age" "age_unit" "wt_kg" "ht_cm"
Spaces, dots, capitals, and a mix of styles (Case ID, date.onset, OUTCOME). They are hard to type and need special handling. We fix all of them in Session 2 with one function.
Case ID infection date date.onset hosp date date of outcome OUTCOME Gender
1 5fe599 2014-05-08 2014-05-13 2014-05-15 <NA> <NA> m
2 8689b7 <NA> 2014-05-13 2014-05-14 2014-05-18 Recover f
3 11f8ea <NA> 2014-05-16 2014-05-18 2014-05-30 Recover m
4 b8812a 2014-05-04 2014-05-18 2014-05-20 <NA> <NA> f
5 893f25 2014-05-18 2014-05-21 2014-05-22 2014-05-29 Recover m
6 be99c8 2014-05-03 2014-05-22 2014-05-23 2014-05-24 Recover f
hospital age age_unit wt_kg ht_cm
1 Other 2 years 27 48
2 <NA> 3 years 25 59
3 St. Mark's Maternity Hospital (SMMH) 56 years 91 238
4 Regional Medical Center 18 years 41 135
5 Military Hospital 3 years 36 71
6 Regional Medical Center 16 years 56 116
Rows: 6,604
Columns: 12
$ `Case ID` <chr> "5fe599", "8689b7", "11f8ea", "b8812a", "893f25", "b…
$ `infection date` <dttm> 2014-05-08, NA, NA, 2014-05-04, 2014-05-18, 2014-05…
$ date.onset <chr> "2014-05-13", "2014-05-13", "2014-05-16", "2014-05-1…
$ `hosp date` <dttm> 2014-05-15, 2014-05-14, 2014-05-18, 2014-05-20, 201…
$ `date of outcome` <dttm> NA, 2014-05-18, 2014-05-30, NA, 2014-05-29, 2014-05…
$ OUTCOME <chr> NA, "Recover", "Recover", NA, "Recover", "Recover", …
$ Gender <chr> "m", "f", "m", "f", "m", "f", "f", "f", "m", "f", "m…
$ hospital <chr> "Other", NA, "St. Mark's Maternity Hospital (SMMH)",…
$ age <chr> "2", "3", "56", "18", "3", "16", "16", "0", "61", "2…
$ age_unit <chr> "years", "years", "years", "years", "years", "years"…
$ wt_kg <dbl> 27, 25, 91, 41, 36, 56, 47, 0, 86, 69, 67, 84, 68, 4…
$ ht_cm <dbl> 48, 59, 238, 135, 71, 116, 87, 11, 226, 174, 112, 18…
In glimpse(), the type is shown in angle brackets: <chr> text, <dbl> number, <date> date, <dttm> date-time.
view() opens the data like a spreadsheet in the Data Explorer. You can sort and filter there to look around; that doesn’t change the data.dim() gives the size: rows first, then columns (6604 12).names() lists the 12 column names exactly as they’re spelled. Code has to use exactly this spelling.head() shows the first 6 rows. head(linelist_raw, 10) shows 10.glimpse() turns the table on its side: one line per column, with its type and the first few values. It’s the easiest way to see every column of a wide dataset.date.onset is <chr> (text) but the other dates are <dttm> (date-time)age is <chr> — it should be a numberage_unit column — are all ages in years?Use dim() and glimpse() on measles_raw.
<chr>)?✅ Check: dim() gives 1215 11
[1] 1215 11
Rows: 1,215
Columns: 11
$ `Patient ID` <chr> "MS-0931", "MS-0113", "MS-0975", "MS-0297", "MS-015…
$ `Report Date` <IDate> 2024-04-13, 2024-04-28, 2024-03-25, 2024-03-24, 2…
$ District <chr> "West", "North", "Central", "Nroth", "South", "West…
$ SEX <chr> "Female", "Male", NA, NA, "Female", "F", "Male", "M…
$ `Age (years)` <chr> "4", "5", "7", "1", "10", "7", "4", "2", "0", "3", …
$ `Vaccinated?` <chr> "No", "Yes", "No", "No", "No", "NO", "NO", "Unknown…
$ Fever <chr> "yes", "yes", "yes", "yes", "yes", "yes", "yes", "y…
$ Rash <chr> "yes", "no", "no", "no", "yes", "yes", "yes", "no",…
$ `Temp (C)` <dbl> 38.3, 38.4, 39.3, 38.0, 38.9, 38.5, 38.7, 38.5, 39.…
$ `Days in Hospital` <int> NA, NA, NA, 4, NA, 5, 6, 5, 9, NA, 3, 1, 4, NA, NA,…
$ Outcome <chr> "Died", "recovered", "Recovered", "Recovered", "Rec…
Age (years) is <chr> — it contains “unknown” and the code “999”`Patient ID`, `Report Date`, `Age (years)`, `Vaccinated?`, `Temp (C)`, `Days in Hospital`👀 Follow along 5.2 — skim(linelist_raw). Look at n_missing, and the min and max of wt_kg and ht_cm.
| Name | linelist_raw |
| Number of rows | 6604 |
| Number of columns | 12 |
| _______________________ | |
| Column type frequency: | |
| character | 7 |
| numeric | 2 |
| POSIXct | 3 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| Case ID | 130 | 0.98 | 6 | 6 | 0 | 5888 | 0 |
| date.onset | 286 | 0.96 | 10 | 10 | 0 | 580 | 0 |
| OUTCOME | 1493 | 0.77 | 5 | 7 | 0 | 2 | 0 |
| Gender | 317 | 0.95 | 1 | 1 | 0 | 2 | 0 |
| hospital | 1509 | 0.77 | 5 | 36 | 0 | 13 | 0 |
| age | 100 | 0.98 | 1 | 2 | 0 | 75 | 0 |
| age_unit | 0 | 1.00 | 5 | 6 | 0 | 2 | 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| wt_kg | 0 | 1 | 52.69 | 18.59 | -11 | 41 | 54 | 66 | 111 | ▁▃▇▅▁ |
| ht_cm | 0 | 1 | 125.25 | 49.57 | 4 | 91 | 130 | 159 | 295 | ▂▅▇▂▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| infection date | 2315 | 0.65 | 2012-04-09 | 2015-04-27 | 2014-10-04 | 538 |
| hosp date | 0 | 1.00 | 2012-04-20 | 2015-04-30 | 2014-10-15 | 570 |
| date of outcome | 1061 | 0.84 | 2012-05-14 | 2015-06-04 | 2014-10-26 | 575 |
Look at the n_missing column, the minimum of wt_kg (below zero!), and the maximum of ht_cm (almost 3 meters!).
skim() summarizes every column at once, grouped by type: text (character), numbers (numeric), and date-times (POSIXct).n_missing is the number of missing values in each column; complete_rate is the share that is filled in.p0 is the smallest value, p50 the median, and p100 the largest. Here the smallest weight is -11 kg and the tallest height is 295 cm: both impossible.n_unique is the number of different values. A high number in a column that should have only a few categories is a sign of misspellings.count()count() shows every distinct value in a column and how often it appears. It’s the fastest way to find misspellings.
The data goes first, then the column: count(DATA, COLUMN).
👀 Follow along 5.3 — count() the hospital, Gender, OUTCOME, and age_unit columns.
Pattern: count(DATA, COLUMN)
hospital n
1 Central Hopital 11
2 Central Hospital 457
3 HMO Health 288
4 Military Hopital 32
5 Military Hospital 800
6 Mitylira Hopital 1
7 Mitylira Hospital 79
8 Other 907
9 Premier Health 288
10 Regional MC 48
11 Regional Medical Center 1756
12 St. Mark's Maternity Hospital (SMMH) 417
13 St. Marks Maternity Hopital (SMMH) 11
14 <NA> 1509
Gender n
1 f 3140
2 m 3147
3 <NA> 317
OUTCOME n
1 Death 2898
2 Recover 2213
3 <NA> 1493
age_unit n
1 months 37
2 years 6567
count(linelist_raw, hospital) lists each different value in hospital and how many rows have it (n). Values are sorted alphabetically, with missing values (NA) last.count(linelist_raw, age_unit) shows that 37 ages are in months, not years.table()
You will often see base R’s table() in other people’s code. It counts too, but it hides missing values unless you ask for them:
🔁 Follow along 5.4 — The base R way: table() OUTCOME, with and without useNA = "ifany".
Death Recover
2898 2213
Death Recover <NA>
2898 2213 1493
table() takes one column, pulled out with $, instead of the data and a column name.useNA = "ifany", it shows only Death (2898) and Recover (2213). The 1493 missing outcomes are left out without a warning, so any percentages you worked out from it would be wrong.useNA = "ifany" adds an <NA> count whenever there are missing values.count() always shows NA. With table(), add useNA = "ifany". See Base R ↔︎ tidyverse for more translations.
count() District, SEX, and `Vaccinated?` in measles_raw. How many different ways is “North” written? How many ways is “No” written?table(measles_raw$District). What is missing compared with count()?✅ Check: a) count(District) shows 18 rows (including NA) b) table() has no NA: 34 cases disappear
District n
1 Central 95
2 Central District 6
3 Cetnral 9
4 East 215
5 East District 7
6 North 318
7 North District 9
8 Nroth 19
9 South 273
10 South District 11
11 West 167
12 West District 6
13 central 3
14 east 11
15 north 10
16 south 13
17 west 9
18 <NA> 34
SEX n
1 F 81
2 Female 415
3 M 86
4 Male 509
5 female 37
6 male 47
7 <NA> 40
Vaccinated? n
1 N 40
2 NO 44
3 No 608
4 Unknown 183
5 Y 32
6 Yes 231
7 no 49
8 yes 28
central Central Central District Cetnral
3 95 6 9
east East East District north
11 215 7 10
North North District Nroth south
318 9 19 13
South South District west West
273 11 9 167
West District
6
central Central Central District Cetnral
3 95 6 9
east East East District north
11 215 7 10
North North District Nroth south
318 9 19 13
South South District west West
273 11 9 167
West District <NA>
6 34
“North” is written 4 ways (“North”, “north”, “Nroth”, “North District”), and so is “No” (“No”, “no”, “N”, “NO”). table() silently drops the 34 cases with no district; useNA = "ifany" brings them back. It also sorts lower-case values first, so “central” and “Central” are listed apart.
filter() and arrange()filter() keeps rows that meet a conditionarrange() sorts rows (desc() for largest first)Each one takes the data first. Column names with spaces need backticks: `Case ID`. (The backtick key is at the top left of most keyboards.) Another reason to clean the names!
linelist_raw is a regular data frame, so R prints every row it gets — all 6,604 of them, with 12 columns each! To do two things, such as filtering and then showing only the first 10 rows, put one function inside another. R runs the inside first:
👀 Follow along 5.5 — Use filter(), arrange(), and head() to find the patients with ages in months, the smallest weights, and the tallest patients (the next two code blocks).
[1] 37
Case ID infection date date.onset hosp date date of outcome OUTCOME Gender
1 e1c081 2014-10-13 2014-10-14 2014-10-14 2014-10-27 Death m
2 9d8358 2014-10-01 <NA> 2014-10-18 <NA> Recover <NA>
3 df9f64 2014-10-06 2014-10-26 2014-10-27 2014-11-06 Death f
4 f76807 <NA> 2014-10-27 2014-10-28 2014-11-02 Death f
5 58b05a <NA> 2014-12-04 2014-12-05 <NA> Recover <NA>
6 5a2c39 2014-12-17 2014-12-25 2014-12-25 2015-01-16 Recover m
7 092cd0 2014-12-22 2015-01-02 2015-01-03 2015-01-17 Death f
8 04a78f <NA> 2015-01-15 2015-01-17 2015-01-26 Recover f
9 833a20 2014-09-20 2014-09-30 2014-10-02 2014-10-05 Recover m
10 5439ab 2014-09-20 2014-10-04 2014-10-05 <NA> <NA> f
hospital age age_unit wt_kg ht_cm
1 Regional Medical Center 24 months 68 138
2 <NA> <NA> months 22 32
3 Regional Medical Center 7 months 49 72
4 Regional Medical Center 7 months 52 120
5 Military Hospital 24 months 49 123
6 Other 24 months 48 74
7 Other 6 months 72 223
8 Military Hospital 36 months 43 65
9 Regional Medical Center 4 months 40 65
10 <NA> 4 months 58 125
Case ID infection date date.onset hosp date date of outcome OUTCOME Gender
1 6056ba <NA> 2014-10-20 2014-10-21 2014-11-05 Death f
2 6056ba <NA> 2012-10-20 2012-10-21 2012-11-05 Death f
3 7400b3 2014-10-07 2014-10-12 2014-10-13 2014-10-18 Death f
4 0fde59 2014-09-01 2014-09-04 2014-09-08 2014-09-14 Recover f
5 fca629 <NA> 2014-12-04 2014-12-07 2014-12-30 Recover f
6 eb4d8f 2014-10-02 2014-10-06 2014-10-07 2014-10-11 <NA> f
7 4fb46f 2014-08-12 2014-08-21 2014-08-25 2014-09-01 Recover f
8 db9404 2015-01-18 2015-01-21 2015-01-24 2015-01-27 Death f
9 78b0f7 2014-10-29 2014-11-05 2014-11-05 2014-11-19 Recover f
10 35a38e 2014-09-11 2014-09-16 2014-09-17 2014-09-26 Recover f
hospital age age_unit wt_kg ht_cm
1 <NA> 0 years -11 24
2 HMO Health 0 years -11 24
3 Regional Medical Center 0 years -10 13
4 <NA> 0 years -10 18
5 Central Hospital 0 years -9 24
6 Other 0 years -8 22
7 Regional Medical Center 0 years -7 13
8 Regional Medical Center 0 years -7 16
9 <NA> 0 years -6 19
10 Regional Medical Center 0 years -6 23
Case ID infection date date.onset hosp date date of outcome OUTCOME Gender
1 ebb14f 2014-08-19 2014-09-01 2014-09-07 2014-09-13 Recover m
2 610c15 <NA> 2015-02-04 2015-02-06 2015-02-20 <NA> m
3 0b32d8 2015-01-07 2015-01-13 2015-01-14 2015-01-15 Death m
4 b06109 2014-11-13 2014-11-28 2014-11-30 2014-12-05 Death m
5 b21b3e <NA> 2015-01-11 2015-01-12 2015-01-14 Death m
6 9c3a56 2015-01-04 2015-01-20 2015-01-21 2015-01-27 Death <NA>
7 389175 2015-03-13 2015-03-24 2015-03-25 2015-04-22 Recover f
8 <NA> 2015-03-13 2015-03-24 2015-03-25 2015-04-22 Recover f
9 ae0163 2014-08-26 2014-08-29 2014-09-06 2014-09-19 <NA> m
10 4b158a <NA> 2014-11-07 2014-11-09 2014-12-08 Recover <NA>
hospital age age_unit wt_kg ht_cm
1 St. Marks Maternity Hopital (SMMH) 84 years 111 295
2 Military Hospital 69 years 105 287
3 Military Hospital 72 years 94 272
4 Regional Medical Center 68 years 85 268
5 <NA> 49 years 80 266
6 <NA> 76 years 101 266
7 <NA> 6 months 95 266
8 <NA> 6 months 95 266
9 <NA> 60 years 94 263
10 <NA> 69 years 87 262
filter(linelist_raw, age_unit == "months") keeps only the rows where age_unit is exactly “months”. == asks “is it equal?” for every row, and filter() keeps the rows where the answer is TRUE.nrow() counts the rows that are left: 37. head(..., 10) shows only the first 10 of them.arrange(linelist_raw, wt_kg) sorts all rows by weight, smallest first, so the impossible weights (down to -11 kg) come to the top. desc(ht_cm) sorts largest first, so the 295 cm height comes to the top.linelist_raw. They show a changed copy and then forget it. To keep a result, save it with <-.head(arrange(linelist_raw, wt_kg), 10) reads: take linelist_raw, arrange it by weight, then take the head. Two steps are manageable, but with three or four the brackets pile up. In Session 2 you’ll learn the pipe %>%, which writes the same steps in order, from left to right.
filter()
== equal · != not equal · > < >= <= · %in% c(...) one of several values · is.na() missing · & and · | or
Use == (two equals signs) to compare. One = is for naming arguments.
Sort measles_raw by `Temp (C)`, largest first, and show only the top 10 rows. What is the highest temperature? What do you think happened?
Pattern: head(arrange(DATA, desc(COLUMN)), 10)
✅ Check: the top value is 104.7
Patient ID Report Date District SEX Age (years) Vaccinated? Fever Rash
1 MS-0663 2024-03-15 South Male 14 No yes yes
2 MS-0444 2024-02-26 East Male 8 Unknown yes yes
3 MS-0908 2024-03-03 Nroth Male 1 yes yes yes
4 MS-0523 2024-04-21 East Female 13 No yes yes
5 MS-0132 2024-03-18 <NA> Male 1 Yes yes yes
6 MS-0052 2024-02-01 South Male 5 Yes yes yes
7 MS-1027 2024-03-21 East Male 3 Unknown yes yes
8 MS-0167 2024-04-10 Central Female 6 No yes yes
9 MS-0110 2024-04-22 South Male 4 No yes yes
10 MS-0943 2024-04-11 South Female 23 Unknown yes yes
Temp (C) Days in Hospital Outcome
1 104.7 NA Recovered
2 104.0 NA Recovered
3 104.0 NA Recovered
4 104.0 NA Recovered
5 103.5 NA Recovered
6 103.3 NA Recovered
7 103.3 NA Recovered
8 103.3 NA Recovered
9 102.9 NA Recovered
10 102.6 NA Recovered
104.7 is about 40 °C written in Fahrenheit. A few temperatures were entered in °F instead of °C.
👀 Follow along 5.6 — Missing values with colSums(is.na()), rows with no Case ID, and duplicates with sum(duplicated()) and get_dupes().
Case ID infection date date.onset hosp date date of outcome
130 2315 286 0 1061
OUTCOME Gender hospital age age_unit
1493 317 1509 100 0
wt_kg ht_cm
0 0
[1] 130
[1] 25
is.na(linelist_raw) checks every cell and gives TRUE where a value is missing. colSums() adds up each column, counting each TRUE as 1. The result is the number of missing values per column, for example 1493 in OUTCOME.filter(linelist_raw, is.na(`Case ID`)) keeps the rows with no Case ID, and nrow() counts them: 130.duplicated() marks every row that is an exact copy of an earlier row (every column the same), and sum() counts them: 25. The first copy of each row isn’t counted.get_dupes() (janitor) shows every row that has a copy, side by side, with a dupe_count column. Here that’s 50 rows: each of the 25 duplicates and its original.In measles_raw:
`Patient ID`?✅ Check: a) 37 b) 15
[1] 37
[1] 15
Patient ID Report Date District SEX
37 29 34 40
Age (years) Vaccinated? Fever Rash
0 0 0 0
Temp (C) Days in Hospital Outcome
0 766 68
Days in Hospital has the most (766). That makes sense: patients who were never hospitalized have no hospital days. Missing is not always an error!👀 Follow along 6.1 — The problem list for linelist_raw, filled in together.
Use these questions on your own data, too. The answers become your problem list.
count())?skim(), arrange())?Here is everything we found in linelist_raw. This list is the to-do list for Session 2.
| # | Problem | How we found it |
|---|---|---|
| 1 | Column names have spaces, dots, and capitals | names() |
| 2 | Date columns are different types (date.onset is text) |
glimpse() |
| 3 | age is stored as text, and 37 ages are in months not years |
glimpse(), count() |
| 4 | Hospital names are misspelled (“Hopital”) or inconsistent (“Regional MC”), and many are missing | count() |
| 5 | Gender and outcome have missing values | count(), skim() |
| 6 | Impossible weights (negative, 0) and heights (4 cm, 295 cm) | skim(), arrange() |
| 7 | 130 rows have no Case ID | is.na() |
| 8 | 25 rows are duplicates | duplicated() |
Write a numbered problem list for measles_raw in your student script, using everything you found in the TRY ITs. Save it — you will fix these problems in Session 2!
✅ Check: you should find at least 8 problems
? and parentheses)Age (years) is stored as text because of “unknown” (27 rows) and “999” (4 rows)Vaccinated? is written 8 different ways (Yes / yes / Y, No / no / N / NO, Unknown)Report Date, District, and SEX have some missing valuesBrought your own dataset? Put the file in the raw_data/ folder, then: import() it; look at it with dim(), glimpse(), and skim(); count() its text columns to look for misspellings; check for missing values and duplicates; and write its problem list.
<- saves a value in an object; c() makes a vector; functions take arguments.install.packages(); load it every session with library().here() makes file paths work on any computer.import(here("folder", "file")) reads almost any file type.glimpse(), skim(), count(), filter(), arrange(), is.na(), and duplicated() find the problems.head(arrange(data, column), 10). Wrap long output in head().Next: Session 2 — Cleaning. We start with the pipe %>%, then fix the problem list with clean_names(), mutate(), if_else(), and case_when().