Section 1

1 Introduction

A doctor gives a child paracetamol. The usual dose is 15 mg for each kilogram of body weight. That sentence is a function: a rule that takes one number in (the weight) and gives exactly one number back (the dose).

A box labelled f of x equals 15 x. Weights of 10, 20 and 32 kilograms go in one at a time and doses of 150, 300 and 480 milligrams come out.

We write the rule as

$$f(x) = 15x$$Read it as "f of x equals 15 times x".
  • \(x\) is the input: here, the weight in kg.
  • \(f(x)\) is the output: the dose in mg that the rule gives for that weight.
  • \(f(20) = 15 \times 20 = 300\) means "for a 20 kg child, the rule gives 300 mg".

A function does not have to be one neat formula. Real dosing rules have a maximum. For paracetamol, the dose for one time is not usually more than 1,000 mg, however heavy the patient. The rule is still a function, because each weight still gives exactly one dose.

Try it

A dosing rule is a function

Move the weight. The dot shows the dose the rule gives. Watch what happens above about 67 kg.

Most models in statistics and machine learning are built from a small set of functions. This lesson covers the ones you will meet again and again:

  • straight lines, for steady change;
  • exponentials, for growth and decay, such as a drug leaving the blood;
  • logarithms, for values that range over many tenfolds, such as viral load or gene expression;
  • odds and the sigmoid, for turning a risk score into a probability.
The same thing in Python
def dose(weight_kg):
    return min(15 * weight_kg, 1000)   # 15 mg/kg, at most 1,000 mg

print(dose(20))   # 300
print(dose(80))   # 1000
Section 2

2 Straight lines

On average, systolic blood pressure rises steadily with age in adults. A steady change is drawn as a straight line. Every straight line has the same form:

$$y = c + m\,x$$m is the slope and c is the intercept.
A straight line of blood pressure against age, y equals 100 plus 0.5 x. A triangle shows that 20 years to the right the line rises 10 mmHg, so the slope is 10 over 20, which is 0.5. The triangle slides along the line to show the slope is the same everywhere.
  • The slope \(m\) is how much \(y\) changes when \(x\) goes up by 1. It is "rise over run". In the animation, \(m = 0.5\): blood pressure goes up 0.5 mmHg for each extra year of age.
  • The intercept \(c\) is the value of \(y\) when \(x = 0\). It is where the line crosses the vertical axis.
  • A positive slope goes up to the right. A negative slope goes down. A slope of 0 is flat: \(x\) makes no difference to \(y\).
Try it

Move a line through the patients

Each dot is a patient. Change the intercept and the slope until the line runs through the middle of the cloud.

Why you will care later

Linear regression is this line with the slope and intercept chosen by the computer. With several inputs it becomes \(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots\), and each \(\beta\) is a slope: the change in the outcome for one more unit of that input. Unit 5 shows how the best line is found.
The same thing in Python
import numpy as np
age = np.array([30, 45, 60, 75])
sbp = 100 + 0.5 * age      # the line, applied to every patient at once
print(sbp)                 # [115.  122.5 130.  137.5]
Quick check

A model says: eGFR (kidney function, mL/min) \( = 120 - 0.8 \times \text{age}\). What does the number \(-0.8\) tell you?

Section 2.1

2.1 When a line is not enough

Some relationships bend. The risk of death is higher for people who are very underweight and for people who are very overweight. A straight line can only go up or down, so it cannot show this U shape. Adding a squared term lets the line bend:

$$y = c + b\,x + a\,x^2$$A quadratic (degree 2). Each extra power of x lets the curve bend once more.

A function made of powers of \(x\) like this is a polynomial. Its degree is the highest power: 1 for a line, 2 for a single bend, 3 for an S-like curve with two bends.

Try it

Can the curve follow the data?

The dots show an illustrative U-shaped link between BMI and the relative risk of death. Choose the degree of the polynomial and watch how well it follows the dots.

More bends is not always better

A polynomial with a very high degree can bend through every single point, including the random noise. It then predicts badly for new patients. This is called overfitting, and Unit 6 is about how to avoid it.
Section 3

3 Exponential growth and decay

After a dose, many drugs leave the blood in a special way: in each hour, the body removes the same fraction of what is left, not the same amount. When a fixed fraction goes each time, the result is exponential decay.

A drug concentration starts at 100 mg per litre and halves every 6 hours: 50, 25, 12.5, 6.25 and 3.125. The curve falls fast at first and then flattens, never reaching zero.

The time it takes for the amount to halve is the half-life, \(T\). After one half-life 50% is left, after two 25%, after three 12.5%, and so on:

$$C(t) = C_0 \times \left(\tfrac{1}{2}\right)^{t/T}$$C₀ is the starting concentration, t is the time and T is the half-life.

You will often see the same curve written with the number \(e \approx 2.718\), which is the natural base for growth and decay:

$$C(t) = C_0 \, e^{-k t}$$k is the elimination rate constant. The two forms are the same curve when k = ln 2 / T ≈ 0.693 / T.

Growth works the same way, with a plus sign. Bacteria that divide every 20 minutes double in number every 20 minutes. A tumour can have a doubling time of a few months. Growth that doubles at a steady rate is exponential growth.

Try it

Change the half-life

Move the half-life slider and watch how fast the drug is cleared. Then tick the log scale box: the curve becomes a straight line.

Straight on a log scale

On a log scale, exponential decay is a straight line. That is why pharmacologists plot drug concentrations on a log axis: the slope of the line gives the elimination rate. The next section explains log scales.
The same thing in Python
import numpy as np
t = np.array([0, 6, 12, 24])         # hours
k = np.log(2) / 6                    # half-life of 6 hours
print(np.round(100 * np.exp(-k * t), 2))   # [100.    50.    25.     6.25]
Quick check

An antibiotic has a half-life of 2 hours. About what fraction of a dose is still in the blood after 8 hours?

Section 4

4 Logarithms and log scales

A logarithm answers one question: how many times do I multiply the base to get this number?

  • \(\log_{10}(1000) = 3\), because \(10 \times 10 \times 10 = 1000\).
  • \(\log_{2}(8) = 3\), because \(2 \times 2 \times 2 = 8\).
  • \(\ln(x)\) is the natural log, with base \(e \approx 2.718\). It is the default in most statistics software, including np.log.

So a logarithm counts multiplications. This makes it the right tool for numbers that range over many tenfolds. An HIV viral load can be 1,000,000 copies/mL before treatment and below 50 a few months later.

Five viral loads, from 1,000,000 down to 100 copies per mL, on an ordinary axis: four of them pile up near zero. On a log axis the same values are evenly spaced, one step for each tenfold fall.

The logarithm has one property that makes it very useful in statistics: it turns multiplying into adding.

$$\log(a \times b) = \log a + \log b$$For example: log₁₀(100 × 1000) = 2 + 3 = 5.

In genomics, a change in gene expression is reported as a log2 fold change. A fold change is a ratio: treated divided by control. Its log2 is +1 when expression doubles and −1 when it halves.

Try it

Why gene expression uses log2 fold change

Move the slider to change how much a gene goes up or down after treatment. Compare where the gene lands on the two scales.

On the ratio scale, every "down" change is squeezed between 0 and 1, while "up" changes can stretch out to any size. A gene that falls 4 times (ratio 0.25) looks much smaller than a gene that rises 4 times (ratio 4). On the log2 scale they are mirror images, \(-2\) and \(+2\), which is fair.

Why you will care later

Fitting a model often means multiplying the probabilities of thousands of patients together (the \(\prod\) from lesson 1.1). The answer is far too small for a computer to store. Taking logs turns the product into a sum, which is easy. This is the log-likelihood you will meet in Unit 5.
The same thing in Python
import numpy as np
print(np.log10([100, 1000, 1_000_000]))   # [2. 3. 6.]
print(np.log2(400 / 100))                 # 2.0   expression up 4 times
print(np.log2(25 / 100))                  # -2.0  expression down 4 times
print(np.log(np.e))                       # 1.0   np.log is the natural log
Quick check

A patient's viral load falls from 100,000 to 1,000 copies/mL. What is the change on a \(\log_{10}\) scale?

Section 5

5 Odds and log-odds

A probability \(p\) is a chance between 0 and 1. If 80% of patients respond to a drug, \(p = 0.8\). Medicine also uses another way to say the same thing, the odds:

$$\text{odds} = \frac{p}{1 - p}$$The chance it happens, divided by the chance it does not.

For \(p = 0.8\), the odds are \(0.8 / 0.2 = 4\): responders outnumber non-responders four to one. The log-odds (also called the logit) is the natural log of the odds.

Three number lines: probability from 0 to 1, odds from 0 upwards, and log-odds from minus 3 to plus 3. Probabilities 0.5, 0.8 and 0.2 are mapped to odds 1, 4 and 0.25, and to log-odds 0, plus 1.39 and minus 1.39.
ScaleRange"As likely as not"
Probability \(p\)0 to 10.5
Odds \(p/(1-p)\)0 to infinity1
Log-odds \(\ln\frac{p}{1-p}\)minus infinity to plus infinity0
Try it

Turn a probability into odds and log-odds

Move the probability. Notice that the log-odds is 0 at p = 0.5, and is symmetric: p = 0.2 and p = 0.8 give the same number with opposite signs.

Why you will care later

Logistic regression models the log-odds as a straight line, because log-odds can take any value, just like a line. Its results are reported as odds ratios: the odds in one group divided by the odds in another.
The same thing in Python
import numpy as np
p = np.array([0.2, 0.5, 0.8])
odds = p / (1 - p)
print(odds)                   # [0.25 1.   4.  ]
print(np.round(np.log(odds), 2))   # [-1.39  0.    1.39]
Quick check

In a trial, 25% of patients have a side effect. What are the odds of the side effect?

Section 6

6 The sigmoid: from score to probability

Many models first add up a score from a patient's measurements, for example \(z = -9 + 1.5 \times \text{glucose}\). The score can be any number, but a risk must lie between 0 and 1. The sigmoid (or logistic) function does the conversion:

$$p = \sigma(z) = \frac{1}{1 + e^{-z}}$$It is the reverse of the log-odds: if z is the log-odds, the sigmoid gives back p.
An S-shaped curve. Scores of minus 4, minus 2, 0, 2 and 4 rise to the curve and give probabilities of 0.02, 0.12, 0.50, 0.88 and 0.98. A score of 0 gives exactly 0.5.
  • A score of 0 gives \(p = 0.5\).
  • Large positive scores give probabilities close to 1. Large negative scores give probabilities close to 0.
  • The curve is steepest in the middle. There, a small change in the score changes the risk a lot. Far out on either side, the risk hardly moves.
Try it

A risk model for one biomarker

Each dot is a patient: at the top if they have the disease, at the bottom if not. Change the two numbers in the score and fit the S-curve to the patients.

This is the whole idea of logistic regression, in one picture: a straight-line score, passed through the sigmoid, gives a probability. The intercept \(\beta_0\) moves the S-curve left and right. The slope \(\beta_1\) makes it steeper or flatter.

Key idea

The log-odds and the sigmoid undo each other. The log-odds turns a probability into a score on the whole number line. The sigmoid turns a score back into a probability.
The same thing in Python
import numpy as np
def sigmoid(z):
    return 1 / (1 + np.exp(-z))

z = np.array([-4, -2, 0, 2, 4])
print(np.round(sigmoid(z), 2))    # [0.02 0.12 0.5  0.88 0.98]

glucose = 7.0
print(round(sigmoid(-9 + 1.5 * glucose), 2))   # 0.82

Why you will care later

The sigmoid is also used inside neural networks, and as the last step of almost every model that predicts a yes/no outcome, such as "will this patient be readmitted within 30 days?".
Section 7

7 Check your understanding

Five short questions on this lesson. Nothing is graded or saved outside this browser. If you choose a wrong option you get a hint, and you can try again.

Check your understanding

A model says HbA1c (%) \( = 5.2 + 0.03 \times \text{weight (kg)}\). What does it predict for a patient who weighs 80 kg?

Check your understanding

A drug has a half-life of 12 hours. Starting from 80 mg/L, what is the concentration after 36 hours?

Check your understanding

A gene has a log2 fold change of \(-3\) after treatment. What happened to its expression?

Check your understanding

Why do models such as logistic regression put a straight line on the log-odds scale rather than on the probability scale?

Check your understanding

A risk score for a patient is \(z = 0\). What risk does the sigmoid give?