Data Wrangling — Part I: The Native Pipe and Choosing Rows and Columns

Before you begin, take a few minutes to check what you already know and how confident you feel. You will see the same questions again at the end of the module — this helps both you and us measure what you have learned. Click an option for every question and confidence rating, then click Next to continue.

In Module 1 you loaded a clinical dataset into R and confirmed it came in correctly. Now you start to reshape it. Real data almost never arrives in the shape your analysis needs: you may want one age group, or just a handful of columns. This part gives you the shared rule behind every reshaping tool, the pipe that strings those tools together, and the first two verbs you will reach for.

This part of the module covers three foundations you will use in almost every analysis:

• One grammar — every dplyr verb takes a data frame as its first argument and returns a new data frame, so verbs chain together.
• The native pipe — read a pipeline aloud as "take the data, THEN do this, THEN do that".
• filter() and select() — keep the rows you want and the columns you need.

By the end of this part you will be able to explain why any dplyr verb can feed straight into the next, read and write a |> pipeline aloud as a sequence of steps, and keep exactly the rows and columns you want from a clinical table.

Try every snippet in the R Scratchpad on the right — the dataset patients.csv is already loaded and waiting for you.

Everything in this part comes from one package, dplyr, which gives you a small set of verbs — functions named after the action they do, like filter, select, and mutate. dplyr is part of the tidyverse, the family of packages this course uses throughout.

The verbs share one rule, and that rule is the whole trick. Every dplyr verb takes a data frame as its first argument and returns a new data frame. The output of one verb is therefore a valid input to the next, so you can chain them into a pipeline that reads like a recipe.

A data frame is just a table: one row per patient, one column per variable. The tidyverse's tidy version of a table is a tibble, which is what read_csv() handed you in Module 1. Verb in, verb out — same shape of object every time.

First, switch the package on for the session with library(). You call library(dplyr) once at the top of your script, not before every line.

library(dplyr)
library(readr)
patients <- read_csv("patients.csv")
patients

Because every verb returns a data frame, you could nest them — but that reads inside-out and quickly becomes unreadable. Instead you use the native pipe, written |>, which takes the result on its left and feeds it as the first argument to the function on its right.

Read |> aloud as the word then. The line patients |> filter(age > 60) says "take patients, THEN filter to ages over 60". The data flows left to right, top to bottom, in the same order you would say the steps out loud.

Writing each step on its own line makes a pipeline read like a numbered procedure. The pipe at the end of a line tells R the pipeline continues onto the next line.

Because the pipe just supplies the data frame as the first argument, patients |> filter(age > 60) does exactly the same thing as writing the data frame inside the verb: filter(patients, age > 60). The pipeline form is easier to read once you chain several steps, so prefer it — but the two are interchangeable, and you will see both.

library(dplyr)
library(readr)
patients <- read_csv("patients.csv")
patients |>
    filter(age > 60)

The filter() and select() are two verbs you will use most often do opposite jobs. One trims the table down the page; the other trims it across. Keeping them straight is the single most useful habit in this part.

filter() keeps the rows where a condition is TRUE and drops the rest. You give it a test such as age > 60 or smoker == "yes". Note the double equals == for "is equal to" — a single = means assignment, not a comparison.

Combine conditions with & for "and" and | for "or". filter(age > 60 & smoker == "yes") keeps patients who are both over 60 and smokers.

library(dplyr)
library(readr)
patients <- read_csv("patients.csv")
patients |>
    filter(age > 60, smoker == "yes")

Watch the ==. Writing filter(smoker = "yes") with one equals is a classic trap — R reads it as trying to set an argument named smoker and throws an error. Use == whenever you mean "is equal to".

library(dplyr)
library(readr)
patients <- read_csv("patients.csv")
filter(patients, smoker = "yes")

select() keeps only the columns you list by name and drops the rest. select(patient_id, age, bmi) returns a narrower table with just those three columns, in the order you wrote them.

library(dplyr)
library(readr)
patients <- read_csv("patients.csv")
patients |>
    select(patient_id, age, bmi)

The classic mix-up is using filter() when you meant select(), or the reverse. A mnemonic fixes it: filteR ends in R for Rows; seleCt has a C for Columns.

Let’s try and practise by trimming the table down the page and across in one pipeline.

library(dplyr)
library(readr)
patients <- read_csv("patients.csv")
patients |>
    filter(age > 50) |>
    select(patient_id, diagnosis)

You have reached the end of the module. Try the same questions again — your answers here, paired with your pre-test answers, are how we measure what the module taught you. Answer every question and confidence rating, then click Submit and see results to view your score.

Here is how your post-test answers compare with your pre-test answers. The pre/post pairing is the most reliable way to see what this module actually taught you.