Section 1

1 Before you start

This is not graded, and it does not count towards anything. Answer every question even if you are only guessing — having a go at something before you have been taught it is one of the best ways to remember it afterwards. The lesson ends with another short check, and there you will be shown the correct answer for anything you missed. Choose an option for every question and confidence rating, then click Submit answers.

Pre-test

df.sort_values("age") by default returns what?

Pre-test

df_a and df_b have the same columns. What does pd.concat([df_a, df_b], ignore_index=True) return?

Pre-test

df.groupby("clinic")["age"].mean() returns which kind of object?

Pre-test

You write df.apply(lambda row: row["weight_kg"] / row["height_m"]**2, axis=1). What does the lambda receive each call?

Pre-test

df["diagnosis"].str.contains("diabetes") returns what?

Pre-test

df.drop_duplicates(subset="patient_id") returns a DataFrame that keeps which copy of each duplicated patient_id?

Pre-test

Several columns in df contain scattered NaN values. What does plain df.dropna() do by default?

Pre-test

A column of integers contains one missing value. After read_csv, what dtype does pandas give it?

Pre-confidence

I can apply a custom function to every row of a DataFrame with .apply and a lambda, and explain what the function sees each call.

Not at all confident
Fully confident
Pre-confidence

I can clean a text column with the .str accessor — for example, lowercase every entry or filter rows where the text contains a keyword.

Not at all confident
Fully confident
Pre-confidence

I can decide whether to drop or fill missing values for a given column, and write the code for either.

Not at all confident
Fully confident
Section 2

2 Introduction

In Part I, you loaded data into a DataFrame and picked the rows and columns that you need. Part II changes the table itself. Real data often has repeated rows, missing values and untidy text. This part turns a raw table into one that is ready to analyse.

This part covers sorting, stacking, grouping, new columns, your own functions, text cleaning, repeated rows and missing values. Part III covers reshaping, joining tables and chaining steps together.

A quick overview on turning a raw dataframe to something analysis-ready.
A quick overview on turning a raw dataframe to something analysis-ready.

Try every example in the Python Scratchpad on the right.

Section 3

3 Basic data manipulation

Once you can pick the rows and columns that you need, the next step is to change the table. This section shows how to sort rows, stack tables, summarise groups and add columns.

Section 3.1

3.1 Sorting

The method sort_values sorts the rows by the values in one column. You give the name of the column. The default order is ascending, with the smallest value first. Add ascending=False to put the largest value first.

import pandas as pd

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong"],
    "ward": ["5A", "7B", "5A", "7B"],
    "potassium": [4.6, 3.9, 5.8, 5.2],
})
low_first = labs.sort_values("potassium")
print(low_first)
# patient ward potassium
# 1 Lim 7B 3.9
# 0 Tan 5A 4.6
# 3 Wong 7B 5.2
# 2 Kumar 5A 5.8
print(labs)
# patient ward potassium
# 0 Tan 5A 4.6
# 1 Lim 7B 3.9
# 2 Kumar 5A 5.8
# 3 Wong 7B 5.2

The second table is the original labs. It is unchanged. This is because sort_values returns a new DataFrame.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
 "patient_id": ["P001", "P002", "P003", "P004"],
 "clinic": ["A", "B", "A", "B"],
 "age": [54, 61, 47, 72],
})
by_age = df.sort_values("age", ascending=False)
print(by_age)

To keep the sorted rows, assign the result to a name, such as labs = labs.sort_values("potassium"). Most pandas methods return a new object in this way.

Calling sort_values without assignment creates a new DataFrame and leaves the original unchanged.
Calling sort_values without assignment creates a new DataFrame and leaves the original unchanged.

To sort by more than one column, pass a list of names. Pandas sorts by the first column. When two rows have the same value there, pandas uses the next column. To choose a direction for each column, pass a list of True and False values to ascending.

import pandas as pd

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong"],
    "ward": ["5A", "7B", "5A", "7B"],
    "potassium": [4.6, 3.9, 5.8, 5.2],
})
print(labs.sort_values(["ward", "potassium"], ascending=[True, False]))
# patient ward potassium
# 2 Kumar 5A 5.8
# 0 Tan 5A 4.6
# 3 Wong 7B 5.2
# 1 Lim 7B 3.9

The wards come out in order, 5A then 7B. This is because the first value is True. Within each ward, the highest potassium comes first. This is because the second value is False.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
    "patient_id": ["P001", "P002", "P003", "P004", "P005"],
    "clinic": ["B", "A", "A", "B", "A"],
    "age": [54, 61, 47, 72, 68],
})
by_clinic_then_age = df.sort_values(["clinic", "age"], ascending=[True, False])
print(by_clinic_then_age)
Section 3.2

3.2 Stack with concat

When two DataFrames have the same columns, the function pd.concat can stack them into one longer table. You can use it to combine files from different months, or results from two wards.

import pandas as pd

monday = pd.DataFrame({"patient": ["Tan", "Lim"], "potassium": [4.6, 3.9]})
tuesday = pd.DataFrame({"patient": ["Kumar", "Wong"], "potassium": [5.8, 5.2]})
print(pd.concat([monday, tuesday]))
# patient potassium
# 0 Tan 4.6
# 1 Lim 3.9
# 0 Kumar 5.8
# 1 Wong 5.2
print(pd.concat([monday, tuesday], ignore_index=True))
# patient potassium
# 0 Tan 4.6
# 1 Lim 3.9
# 2 Kumar 5.8
# 3 Wong 5.2

In the first result, the row numbers 0 and 1 appear twice. This is because each table keeps its own row numbers. The argument ignore_index=True renumbers the rows from 0, as in the second result.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df_a = pd.DataFrame({"patient_id": ["P001", "P002"], "age": [54, 61]})
df_b = pd.DataFrame({"patient_id": ["P003", "P004"], "age": [47, 72]})
all_patients = pd.concat([df_a, df_b], ignore_index=True)
print(all_patients)
Two small DataFrames being concatenated row-wise into a single table, then renumbered with a clean index via ignore_index=True.
Two small DataFrames being concatenated row-wise into a single table, then renumbered with a clean index via ignore_index=True.
Section 3.3

3.3 Grouping

The method groupby splits a table into groups, one for each different value in a column. You then apply a summary, such as .mean(), to every group at once.

import pandas as pd

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong", "Ng"],
    "ward": ["5A", "7B", "5A", "7B", "5A"],
    "potassium": [4.6, 3.9, 5.8, 5.2, 6.1],
    "sodium": [139, 141, 136, 138, 142],
})
print(labs.groupby("ward")["potassium"].mean())
# ward
# 5A 5.50
# 7B 4.55
# Name: potassium, dtype: float64

Ward 5A has a mean potassium of 5.5 mmol/L. Ward 7B has a mean of 4.55 mmol/L.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
    "clinic": ["A", "A", "B", "B", "B"],
    "age": [54, 47, 61, 72, 38],
    "bmi": [27.3, 31.2, 24.1, 22.8, 29.5],
})
print(df.groupby("clinic")["age"].mean())

Read the chain from left to right. The method groupby splits the rows into one group for each value in the column that you name. The brackets after it select one column inside each group. Then .mean() turns each group into a single number. The result is a Series with one row for each group. The methods .median(), .sum(), .count() and .max() work in the same way.

The Split-Apply-Combine pattern behind pandas groupby, where rows are partitioned by a key column, a reduction is applied independently to each group, and the per-group results are stitched back together into a single indexed Series.
The Split-Apply-Combine pattern behind pandas groupby, where rows are partitioned by a key column, a reduction is applied independently to each group, and the per-group results are stitched back together into a single indexed Series.

As in Part I, a list of names in the brackets gives a DataFrame. So [["potassium", "sodium"]] summarises two columns at once. The result has one row for each ward and one column for each name.

import pandas as pd

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong", "Ng"],
    "ward": ["5A", "7B", "5A", "7B", "5A"],
    "potassium": [4.6, 3.9, 5.8, 5.2, 6.1],
    "sodium": [139, 141, 136, 138, 142],
})
print(labs.groupby("ward")[["potassium", "sodium"]].mean())
# potassium sodium
# ward
# 5A 5.50 139.0
# 7B 4.55 139.5
Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
    "clinic": ["A", "A", "B", "B", "B"],
    "age": [54, 47, 61, 72, 38],
    "bmi": [27.3, 31.2, 24.1, 22.8, 29.5],
    "glucose": [5.4, 6.1, 5.9, 7.2, 6.8],
})
print(df.groupby("clinic")[["age", "bmi", "glucose"]].mean())
Section 3.4

3.4 Multiple summaries with .agg

The method .agg gives each column its own summary. Chain it after groupby and pass a dictionary of column names and summaries.

import pandas as pd

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong", "Ng"],
    "ward": ["5A", "7B", "5A", "7B", "5A"],
    "potassium": [4.6, 3.9, 5.8, 5.2, 6.1],
    "sodium": [139, 141, 136, 138, 142],
})
print(labs.groupby("ward").agg({"potassium": "max", "sodium": "min"}))
# potassium sodium
# ward
# 5A 6.1 136
# 7B 5.2 138

The potassium column holds the highest result in each ward. The sodium column holds the lowest.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
 "clinic": ["A", "A", "B", "B"],
 "age": [50, 60, 40, 80],
 "bmi": [25.0, 27.0, 23.0, 29.0],
})
print(df.groupby("clinic").agg({"age": "mean", "bmi": "max"}))
Passing a dict to .agg() after groupby applies a chosen summary function to each named column, returning one summary row per group.
Passing a dict to .agg() after groupby applies a chosen summary function to each named column, returning one summary row per group.

The method .agg also accepts a list. A list gives every selected column the same set of summaries.

import pandas as pd

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong", "Ng"],
    "ward": ["5A", "7B", "5A", "7B", "5A"],
    "potassium": [4.6, 3.9, 5.8, 5.2, 6.1],
    "sodium": [139, 141, 136, 138, 142],
})
print(labs.groupby("ward")[["potassium", "sodium"]].agg(["min", "max", "count"]))
# potassium sodium
# min max count min max count
# ward
# 5A 4.6 6.1 3 136 142 3
# 7B 3.9 5.2 2 138 141 2

The result has two levels of column headings: the column name on top and the summary underneath.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
    "clinic": ["A", "A", "A", "B", "B"],
    "age": [54, 47, 61, 72, 38],
    "bmi": [27.3, 31.2, 24.8, 22.8, 29.5],
})
print(df.groupby("clinic")[["age", "bmi"]].agg(["mean", "median", "count"]))

Keep count in the list whenever you report a mean or a median this way. This is because a mean of three results is less reliable than a mean of three hundred results, even when the two numbers are the same.

Section 3.5

3.5 Adding new columns

To add a column, assign values to a new column name in square brackets. The values usually come from a calculation on existing columns. Pandas applies the calculation to every row at once.

import pandas as pd

bp = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar"],
    "systolic": [128, 142, 118],
    "diastolic": [82, 90, 76],
})
bp["pulse_pressure"] = bp["systolic"] - bp["diastolic"]
print(bp)
# patient systolic diastolic pulse_pressure
# 0 Tan 128 82 46
# 1 Lim 142 90 52
# 2 Kumar 118 76 42

The new column pulse_pressure appears at the right of the table. Each value is the systolic pressure minus the diastolic pressure, in mmHg.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
 "weight_kg": [72, 80, 65],
 "height_m": [1.75, 1.80, 1.60],
})
df["bmi"] = df["weight_kg"] / (df["height_m"] ** 2)
print(df)
Pandas creates a new column by applying one formula to every row at once
Pandas creates a new column by applying one formula to every row at once
Section 4

4 Row-wise transforms with apply and lambda

When no built-in method does the job, .apply runs a function that you write. The function can be a lambda or a function written with def.

On a Series, .apply calls the function once for each value. This function gives each potassium result a status of low, normal or high:

import pandas as pd

def potassium_status(level):
    if level > 5.0:
        return "high"
    elif level < 3.5:
        return "low"
    else:
        return "normal"

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar"],
    "potassium": [4.6, 3.1, 5.8],
})
labs["status"] = labs["potassium"].apply(potassium_status)
print(labs)
# patient potassium status
# 0 Tan 4.6 normal
# 1 Lim 3.1 low
# 2 Kumar 5.8 high

On a DataFrame, the argument axis=1 makes .apply call the function once for each row. The function receives the row as a Series. This lambda finds the difference between the largest and the smallest of three systolic readings, in mmHg:

import pandas as pd

bp = pd.DataFrame({
    "reading_1": [128, 142, 118],
    "reading_2": [124, 138, 120],
    "reading_3": [126, 145, 117],
})
bp["spread"] = bp.apply(lambda row: max(row) - min(row), axis=1)
print(bp)
# reading_1 reading_2 reading_3 spread
# 0 128 124 126 4
# 1 142 138 145 7
# 2 118 120 117 3

Here row is a Series that holds the three readings of one patient. A built-in method is shorter and faster when one exists. Look for one before you write your own function.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
 "weight_kg": [72, 80, 65],
 "height_m": [1.75, 1.82, 1.68],
})
df["bmi"] = df.apply(lambda row: row["weight_kg"] / row["height_m"] ** 2, axis=1)
print(df)

df["weight_lbs"] = df["weight_kg"].apply(lambda kg: kg * 2.205)
print(df)
pandas .apply runs a function on each value of a Series and on each row of a DataFrame with axis=1.
pandas .apply runs a function on each value of a Series and on each row of a DataFrame with axis=1.
Section 5

5 Cleaning text columns with the .str accessor

Pandas has string methods for text columns, such as patient IDs, names or diagnoses. You reach them with .str.

These methods act on every value of the column at once. So no loop is needed. Here are three of them:

  • .str.upper() and .str.lower() change the case of every value.
  • .str.split(" ") breaks each value into a list of words at every space.
  • .str.contains("text") gives True for each value that holds the text, and False for the others. You can use the result to filter rows.
import pandas as pd

visits = pd.DataFrame({
    "sample_id": ["s-101", "S-102", "s-103"],
    "note": ["fever and cough", "Fever", "no fever"],
})
print(visits["sample_id"].str.upper())
# 0 S-101
# 1 S-102
# 2 S-103
# Name: sample_id, dtype: object
print(visits["note"].str.split(" "))
# 0 [fever, and, cough]
# 1 [Fever]
# 2 [no, fever]
# Name: note, dtype: object

The first call gives every ID the same case. The second call turns each note into a list of words.

import pandas as pd

visits = pd.DataFrame({
    "sample_id": ["s-101", "S-102", "s-103"],
    "note": ["fever and cough", "Fever", "no fever"],
})
has_fever = visits["note"].str.contains("fever")
print(has_fever)
# 0 True
# 1 False
# 2 True
# Name: note, dtype: bool
print(visits[has_fever])
# sample_id note
# 0 s-101 fever and cough
# 2 s-103 no fever
print(visits["note"].str.lower().str.contains("fever"))
# 0 True
# 1 True
# 2 True
# Name: note, dtype: bool

The note Fever has a capital letter. So contains("fever") gives False for it. The note no fever still gives True. This is because contains compares characters and does not read the meaning. The last line changes the case first. So all three notes match.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
    "patient_id": ["p001", "p002", "p003"],
    "diagnosis": ["Type 2 diabetes", "Hypertension", "Type 1 diabetes"],
})
print(df["patient_id"].str.upper())
print(df[df["diagnosis"].str.contains("diabetes")])
pandas .str enables vectorised text operations (upper, split, contains) across a Series of patient data.
pandas .str enables vectorised text operations (upper, split, contains) across a Series of patient data.
Section 6

6 Finding duplicates with duplicated and drop_duplicates

A table can hold the same row twice, for example when a patient is entered into a system two times. Two methods find and remove these rows:

  • duplicated() gives False for the first copy of a row and True for each later copy.
  • drop_duplicates() keeps the first copy of each row and removes the later copies.
import pandas as pd

samples = pd.DataFrame({
    "sample_id": ["S01", "S02", "S01", "S03"],
    "potassium": [4.6, 3.9, 4.6, 5.2],
})
print(samples.duplicated())
# 0 False
# 1 False
# 2 True
# 3 False
# dtype: bool
print(samples.drop_duplicates())
# sample_id potassium
# 0 S01 4.6
# 1 S02 3.9
# 3 S03 5.2

Row 2 is a copy of row 0. The rows that stay keep their row numbers. This is why the number 2 is missing from the result.

By default, pandas compares every column. Pass subset to compare only the columns that you name.

import pandas as pd

samples = pd.DataFrame({
    "sample_id": ["S01", "S02", "S01", "S03"],
    "potassium": [4.6, 3.9, 4.7, 5.2],
})
print(samples.drop_duplicates())
# sample_id potassium
# 0 S01 4.6
# 1 S02 3.9
# 2 S01 4.7
# 3 S03 5.2
print(samples.drop_duplicates(subset="sample_id"))
# sample_id potassium
# 0 S01 4.6
# 1 S02 3.9
# 3 S03 5.2

The two entries for S01 differ by 0.1 mmol/L. So the first call keeps all four rows. With subset, pandas compares only sample_id. So it removes the second S01. Check which entry is correct before you remove either one.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd

df = pd.DataFrame({
    "patient_id": ["P001", "P002", "P001", "P003"],
    "age": [54, 61, 54, 47],
})
print(df.duplicated())
print(df.drop_duplicates(subset="patient_id"))
Panda’s df.duplicated() flags repeat rows row by row, and how df.drop_duplicates() then keeps only the first occurrence.
Panda’s df.duplicated() flags repeat rows row by row, and how df.drop_duplicates() then keeps only the first occurrence.
Section 7

7 Inspecting and handling missing values (isna, dropna)

Real data has gaps. A patient skipped a question on a form. An instrument failed for one well of a plate. A column that you now need did not exist when an old record was filed. Pandas shows each gap as NaN. This stands for "not a number". Pandas has tools to find the gaps and to deal with them.

Section 7.1

7.1 Counting number of missing data

A good first step with a new dataset is to count its missing values. The method isna() gives a table of True and False values. True means the value is missing.

import pandas as pd
import numpy as np

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong", "Ng"],
    "potassium": [4.6, np.nan, 5.8, np.nan, 5.2],
    "sodium": [139, 141, np.nan, 138, 140],
})
print(labs.isna())
# patient potassium sodium
# 0 False False False
# 1 False True False
# 2 False False True
# 3 False True False
# 4 False False False
print(labs.isna().sum())
# patient 0
# potassium 2
# sodium 1
# dtype: int64
print(labs["potassium"].isna().sum())  # 2

Adding .sum() counts the True values in each column. A single column works the same way, as in the last line.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd
import numpy as np

df = pd.DataFrame({
 "patient_id": ["P001", "P002", "P003", "P004"],
 "age": [54, np.nan, 47, 72],
 "bmi": [27.3, 24.1, np.nan, np.nan],
})
print(df.isna().sum())

If a column is missing most of its values, you might rethink your data collection or analysis.

df.isna().sum() works first across an entire DataFrame and then on a single column
df.isna().sum() works first across an entire DataFrame and then on a single column
Section 7.2

7.2 Handling missing data

Once you know what is missing, you can drop rows with dropna or fill the gaps with fillna.

Plain dropna() drops every row that has a missing value in any column. This can remove more than you want. The subset argument makes dropna look only at the columns you list.

import pandas as pd
import numpy as np

labs = pd.DataFrame({
    "patient": ["Tan", "Lim", "Kumar", "Wong", "Ng"],
    "potassium": [4.6, np.nan, 5.8, np.nan, 5.2],
    "sodium": [139, 141, np.nan, 138, 140],
})
print(labs.dropna())
# patient potassium sodium
# 0 Tan 4.6 139.0
# 4 Ng 5.2 140.0
print(labs.dropna(subset=["potassium"]))
# patient potassium sodium
# 0 Tan 4.6 139.0
# 2 Kumar 5.8 NaN
# 4 Ng 5.2 140.0

The first call keeps only complete rows. The second call looks only at potassium. So it keeps Kumar, whose sodium result is missing. The axis=1 argument makes dropna drop columns instead of rows.

By default pandas df.dropna() drops every row that has any NaN (usually too aggressive), so pass subset=['col'] to only check specific columns, or use axis=1 to drop columns instead of rows.
By default pandas df.dropna() drops every row that has any NaN (usually too aggressive), so pass subset=['col'] to only check specific columns, or use axis=1 to drop columns instead of rows.
Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd
import numpy as np

df = pd.DataFrame({
 "patient_id": ["P001", "P002", "P003"],
 "age": [54, np.nan, 47],
 "bmi": [27.3, 24.1, np.nan],
})
print(df.dropna())
print(df.dropna(subset=["age"]))

Filling is the alternative when you cannot afford to lose rows. The method fillna replaces each missing value with a value you choose. A fill value of 0 is risky for a measurement like sodium.

import pandas as pd
import numpy as np

labs = pd.DataFrame({"sodium": [139, 141, np.nan, 138, 140]})
print(labs["sodium"].mean())  # 139.5
print(labs["sodium"].fillna(0).mean())  # 111.6

The mean of the four known results is 139.5 mmol/L. After the gap is filled with 0, the mean falls to 111.6 mmol/L. A sodium result of 0 mmol/L is not possible. The zero makes the mean wrong.

import pandas as pd
import numpy as np

labs = pd.DataFrame({"sodium": [139, np.nan, 141, 138]})
labs["sodium"] = labs["sodium"].fillna(labs["sodium"].median())
print(labs)
# sodium
# 0 139.0
# 1 139.0
# 2 141.0
# 3 138.0

A better fill uses a typical value, such as the median. The median of the three known results is 139. So the gap is filled with 139. You can also give fillna a dictionary with one fill value per column.

Try it out

Try this snippet in the Python Scratchpad on the right.

Try this snippet
import pandas as pd
import numpy as np

df = pd.DataFrame({"age": [54, np.nan, 47, 72]})
df["age"] = df["age"].fillna(df["age"].median())
print(df)

Filling with the median is a simple choice. It is not always the right one. Other methods exist, such as the mean within each group. Whichever method you use, write it down. This is because filled values change the results of your analysis.

Imputing a missing value in a pandas Series by replacing NaN with the median of the remaining values.
Imputing a missing value in a pandas Series by replacing NaN with the median of the remaining values.
Section 8

8 Check your understanding

You have reached the end of the module. Try the same questions again — your answers here, paired with your pre-test answers, are how we measure what the module taught you. Answer every question and confidence rating, then click Submit and see results to view your score.

Post-test

df.sort_values("age") by default returns what?

Post-test

df_a and df_b have the same columns. What does pd.concat([df_a, df_b], ignore_index=True) return?

Post-test

df.groupby("clinic")["age"].mean() returns which kind of object?

Post-test

You write df.apply(lambda row: row["weight_kg"] / row["height_m"]**2, axis=1). What does the lambda receive each call?

Post-test

df["diagnosis"].str.contains("diabetes") returns what?

Post-test

df.drop_duplicates(subset="patient_id") returns a DataFrame that keeps which copy of each duplicated patient_id?

Post-test

Several columns in df contain scattered NaN values. What does plain df.dropna() do by default?

Post-test

A column of integers contains one missing value. After read_csv, what dtype does pandas give it?

Post-confidence

I can apply a custom function to every row of a DataFrame with .apply and a lambda, and explain what the function sees each call.

Not at all confident
Fully confident
Post-confidence

I can clean a text column with the .str accessor — for example, lowercase every entry or filter rows where the text contains a keyword.

Not at all confident
Fully confident
Post-confidence

I can decide whether to drop or fill missing values for a given column, and write the code for either.

Not at all confident
Fully confident
Section 9

9 Your results

Here is how your post-test answers compare with your pre-test answers. The pre/post pairing is the most reliable way to see what this module actually taught you.

Your score

Submit the post-test to see your results.

Muddiest point

What is the one thing from this module that is still unclear to you?

Rate this module

Overall, how would you rate this module?

How likely are you to recommend this module to a peer? (0 = not at all, 10 = extremely likely)