Getting Started in R — Part IV: Missing-Data Decisions

Before you begin, take a few minutes to check what you already know and how confident you feel. You will see the same questions again at the end of the module — this helps both you and us measure what you have learned. Click an option for every question and confidence rating, then click Next to continue.

In Part III you found and counted missing values with is.na() and sum(is.na(x)), and you fixed mean() with na.rm = TRUE. Now you decide what to do about those gaps — and that decision is where an honest analysis is won or lost.

Removing incomplete rows is easy to type and easy to get wrong. Do it carelessly and you can quietly delete your sickest patients and report a reassuring average that is simply not true. This part covers three ideas:

• Dropping rows as a counted decision — removing incomplete rows on purpose with drop_na(), counting the rows before and after so the number you removed goes into your write-up.
• Complete-case bias — why keeping only the complete rows can shift your answer in a predictable direction, not merely make it less precise.
• Detect, count, report — the safe habit for this course: look at who is missing, decide whether the gaps are random, and know when dropping rows is the wrong move.

By the end of this part you will be able to remove incomplete rows with drop_na() while reporting the count you dropped, explain why a complete-case analysis can bias a clinical result when the missingness is tied to the patients, and say when dropping rows is the wrong move.

Try every snippet in the R Scratchpad on the right — the dataset diabetes_clinic.csv is already loaded, and some of its HbA1c values are missing on purpose so you can watch a drop change the row count and see how the count you report keeps you honest.

Once you know how many values are missing, you might decide to drop the incomplete rows so the rest of your analysis runs cleanly. The key word is decide: removing rows is a choice you make on purpose and write down, not a reflex.

The tidy tool is drop_na() from the tidyr package. drop_na(clinic) returns a new tibble with every row that has any NA removed; drop_na(clinic, hba1c) removes only rows where the hba1c column is missing, leaving gaps in other columns untouched.

Frame it as a counted decision every time. Count first with sum(is.na(clinic$hba1c)), drop second, then check the new row count with nrow(). The difference between the two row counts is exactly how many patients you removed — and that number belongs in your write-up.

library(readr)
library(tidyr)
clinic <- read_csv("diabetes_clinic.csv")
clinic_complete <- drop_na(clinic, hba1c)
nrow(clinic)
nrow(clinic_complete)

Now the trap that even experienced analysts fall into: treating drop_na() as a free, neutral fix. It is not free. Dropping rows always costs you data, and sometimes it costs you more than precision — it can bend your answer. The next section explains why.

When you keep only the rows with no missing values, you are doing what statisticians call a complete-case analysis. It is tempting because it is simple. The danger is that it quietly assumes the patients with missing data are just like the patients without — and in clinical work that assumption is often false.

Here is the heart of it. If values are missing for reasons connected to the patients themselves, then the patients you drop are different from the ones you keep. Dropping them does not just lose you a little precision — it can shift your answer in a specific direction. That shift is called bias.

Make it concrete. Suppose the sickest diabetic patients are the ones who miss their follow-up HbA1c tests, because they are too unwell to come in. Drop every patient with a missing HbA1c, and you have thrown out your sickest cases. The average HbA1c of who remains looks reassuringly low — not because the clinic is doing well, but because you deleted the people doing badly. The result is biased, not merely less precise.

So the safe habit is: count what is missing, look at who is missing and ask whether the missingness is random, report the count, and only then decide. When missingness is large or clearly related to the patients, dropping rows is the wrong move.

What is the right move in that case? A technique called imputation — filling the gaps with statistically principled estimates instead of deleting rows. That is genuinely useful and genuinely easy to do badly, so we defer it to Course 2. For this course, your job is to detect, count, and report missingness honestly — not to impute.

This Parsons problem gives you the right lines in the wrong order, plus a few lines that do not belong. Drag the correct lines into order — count the missing values before you drop anything — and leave the wrong ones in the bank.

The worked example below fades the support as you go: first you study a full solution on the real clinic data, then you fill the gap, then you solve a fresh one on your own. Watch the punchline at the end — it is the whole point of this part.

library(readr)
library(tidyr)
clinic <- read_csv("diabetes_clinic.csv")
mean(clinic$hba1c)
sum(is.na(clinic$hba1c))
nrow(clinic)
mean(clinic$hba1c, na.rm = TRUE)

You have reached the end of the module. Try the same questions again — your answers here, paired with your pre-test answers, are how we measure what the module taught you. Answer every question and confidence rating, then click Submit and see results to view your score.

Here is how your post-test answers compare with your pre-test answers. The pre/post pairing is the most reliable way to see what this module actually taught you.