1 Before you start
This is not graded, and it does not count towards anything. Answer every question even if you are only guessing — having a go at something before you have been taught it is one of the best ways to remember it afterwards. The lesson ends with another short check, and there you will be shown the correct answer for anything you missed. Choose an option for every question and confidence rating, then click Submit answers.
You open a file with with open("data.tsv") as f: and want to walk it line by line. Which loop is correct?
Why is with open(...) as f: preferred over plain f = open(...)?
You loop over a file with for line in f: and call print(line). There is a blank line between every entry. Why?
You read a line from a tab-separated file and want each column as a separate value. What is the right call?
After parts = line.strip().split("\t"), you want to compare parts[3] to a number. What must you do first?
You open a file with open("output.csv", "w"). What happens if the file already exists?
Inside with open("out.txt", "w") as f:, which call writes text and adds a newline at the end for you?
Inside a script run as python analyse.py patients.tsv 0.05, what is sys.argv[2]?
I can read a tab-separated file line by line, split each line into columns, and process them one at a time.
I can write a script that takes a filename from the command line, reads it, and writes a result to another file.
2 Introduction
Real data is rarely typed into a program. It usually arrives in a file, such as a table of laboratory results or a list of gene names. So a program needs to read files, and often write results to new files.
This lesson shows how to read a file line by line, split a line into columns and write a new file. It also shows how a script receives values you type when you run it.
Try every example in the Python Scratchpad on the right.
3 Reading and writing files
The built-in function open() opens a file. The next example reads the file potassium.tsv, pre-loaded in the Python Scratchpad. It holds the potassium results, in mmol/L, of five patients. Each line is one row of a table, with the columns patient, ward and potassium. A tab character separates the columns. So this is a tab-separated values (TSV) file.
with open("potassium.tsv") as infile:
text = infile.read()
print(type(text)) # <class 'str'>The with line opens the file and gives it a name, here infile. The indented lines under it are the block. When the block ends, Python closes the file. Python closes the file even if an error stops the code part way through. It is important to close a file. This is because text you write to it may not be saved until it is closed. The method .read() gives back the whole file as one string.
The file samples.txt is also pre-loaded. Type the code below and run it.
Try this snippet in the Python Scratchpad on the right.
with open("samples.txt") as f:
contents = f.read()
print(contents)
Reading the whole file is fine for a small file, but not for a 50 GB file. This is because the whole file must fit in memory. The more common way is a for loop over the file, which gives one line at a time.
passes = 0
with open("potassium.tsv") as infile:
for row in infile:
passes = passes + 1
print(passes) # 6The file has six lines, so the loop runs six times. The next exercise uses .strip(), which is explained below.
Try this snippet in the Python Scratchpad on the right.
with open("samples.txt") as f:
for line in f:
print(line.strip())

A loop over a file also works for a 50 GB file. This is because Python does not keep a line after the loop moves on.
An open file is an iterator. An iterator is an object that gives its values one at a time. A function that uses the keyword yield can do the same. It is called a generator. The function next() asks a generator for its next value. The generator runs until yield, gives back that value, and pauses.
Try this snippet in the Python Scratchpad on the right.
def countdown(n):
while n > 0:
yield n
n -= 1
launch = countdown(3)
print(next(launch))
print(next(launch))
print(next(launch))
A data file often starts with a header line of column names. The function next() also works on a file. So next(infile) before the loop takes the header line. The loop then starts at the second line.
patients = 0
with open("potassium.tsv") as infile:
next(infile)
for row in infile:
patients = patients + 1
print(patients) # 5Try this snippet in the Python Scratchpad on the right.
with open("samples.txt") as f:
first_gene = next(f)
print("first line:", first_gene.strip())
for gene in f:
print("also here:", gene.strip())
Each line the loop gives still ends with an invisible newline character. The function print() adds a newline of its own. So printing each line leaves a blank line after it. The method .strip() removes spaces, tabs and newlines from both ends of a string.
with open("potassium.tsv") as infile:
next(infile)
row = next(infile)
print(len(row)) # 11
print(len(row.strip())) # 10Before .strip(), the first patient line has 11 characters. After .strip(), it has 10. The difference is the newline.
(a) Without .strip():
Try this snippet in the Python Scratchpad on the right.
with open("patients.tsv") as f:
for line in f:
print(line)
(b) With .strip():
Try this snippet in the Python Scratchpad on the right.
with open("patients.tsv") as f:
for line in f:
print(line.strip())
To use one column, split the line at its tabs. In Python, a tab is written "\t". The method .split("\t") cuts a string at every tab and gives back a list of strings. Call .strip() first, so the newline is removed before the line is split.
with open("potassium.tsv") as infile:
next(infile)
row = next(infile)
columns = row.strip().split("\t")
print(columns) # ['Tan', '5A', '4.6']
print(columns[2] == "4.6") # True
unstripped = row.split("\t")
print(unstripped[2] == "4.6") # FalseWithout .strip(), the last column keeps the newline. So the second test is False. Every item in the list is a string, even when it looks like a number. This is because Python reads everything in a text file as a string.
Convert a result with float() before you compare it with a number. The operator > cannot compare a string with a number. So the second comparison below raises a TypeError.
columns = ["Tan", "5A", "4.6"]
potassium = float(columns[2])
print(potassium > 5.5) # False
print(columns[2] > 5.5)
# TypeError: '>' not supported between instances of 'str' and 'float'To write to a file, pass a second argument to open(), called the mode. The mode "w" writes. If the file does not exist, Python creates it. If it already exists, Python first deletes everything in it. Without a mode, open() can only read.
The simplest way to write a line is print() with file= and the name of the open file. It writes into the file, not onto the screen. It also adds a newline.
with open("flagged.txt", "w") as outfile:
print("Kumar", file=outfile)
print("Ng", file=outfile)
with open("flagged.txt", "w") as outfile:
print("Wong", file=outfile)
with open("flagged.txt") as infile:
for name in infile:
print(name.strip()) # WongThe second open() uses "w", so it deleted Kumar and Ng first. Only Wong is left.
with open("flagged.txt", "w") as outfile:
print("Kumar", file=outfile)
print("Ng", file=outfile)
with open("flagged.txt", "a") as outfile:
print("Wong", file=outfile)
with open("flagged.txt") as infile:
for name in infile:
print(name.strip()) # Kumar, Ng, WongThe mode "a" appends. It adds new text at the end and keeps what is already in the file. So Wong is added after Kumar and Ng. The comment lists the three lines that the loop prints.
A file also has the method .write(). It writes exactly the string you give it. It does not add a newline.
with open("flagged.txt", "w") as outfile:
outfile.write("Kumar")
outfile.write("Ng")
with open("flagged.txt") as infile:
print(infile.read()) # KumarNgSo both names end up on the same line. To put each name on its own line, end each string with a newline, written \n, as in "Kumar\n".
Try this snippet in the Python Scratchpad on the right.
results = [("P001", 22.4), ("P002", 27.8), ("P003", 31.5)]
with open("bmi_results.tsv", "w") as f:
print("patient_id\tbmi", file=f)
for pid, bmi in results:
print(f"{pid}\t{bmi:.1f}", file=f)
4 Command-line arguments with sys.argv
Until now, the file name and the threshold 5.5 have been written inside the script. To use a new dataset, you must edit the .py file.
The standard fix is to give the script its values when you run it. In a terminal, type python, the script name and the values, separated by spaces. For example: python flag_high.py potassium.tsv 5.5.

The sys module is part of Python's standard library. There is nothing to install. You add import sys at the top of the script. Python stores what you typed after python in a list called sys.argv.
import sys
sys.argv = ["flag_high.py", "potassium.tsv", "5.5"]
print(sys.argv) # ['flag_high.py', 'potassium.tsv', '5.5']
print(sys.argv[1]) # potassium.tsv
print(sys.argv[2]) # 5.5The Python Scratchpad has no terminal, so the second line sets sys.argv by hand. A real script does not have that line.
The first item, sys.argv[0], is the script name. The values you typed come after it, in order.
Every item is a string, even if you typed a number. This is because the command line passes every value to the script as a string. To use a number, convert it with int() or float().
import sys
sys.argv = ["flag_high.py", "potassium.tsv", "5.5"]
print(type(sys.argv[2])) # <class 'str'>
threshold = float(sys.argv[2])
print(type(threshold)) # <class 'float'>Running a script in a terminal is beyond the scope of this lesson. You only need to recognise sys.argv and know what it holds.
5 Check your understanding
You have reached the end of the module. Try the same questions again — your answers here, paired with your pre-test answers, are how we measure what the module taught you. Answer every question and confidence rating, then click Submit and see results to view your score.
You open a file with with open("data.tsv") as f: and want to walk it line by line. Which loop is correct?
Why is with open(...) as f: preferred over plain f = open(...)?
You loop over a file with for line in f: and call print(line). There is a blank line between every entry. Why?
You read a line from a tab-separated file and want each column as a separate value. What is the right call?
After parts = line.strip().split("\t"), you want to compare parts[3] to a number. What must you do first?
You open a file with open("output.csv", "w"). What happens if the file already exists?
Inside with open("out.txt", "w") as f:, which call writes text and adds a newline at the end for you?
Inside a script run as python analyse.py patients.tsv 0.05, what is sys.argv[2]?
I can read a tab-separated file line by line, split each line into columns, and process them one at a time.
I can write a script that takes a filename from the command line, reads it, and writes a result to another file.
6 Your results
Here is how your post-test answers compare with your pre-test answers. The pre/post pairing is the most reliable way to see what this module actually taught you.
Submit the post-test to see your results.
What is the one thing from this module that is still unclear to you?