Learn/Python/14. Comprehensions and generators
Lesson 14 of 15
Sign in

Python, lesson 14

Comprehensions and generators

Python's signature style is to describe what collection you want rather than spelling out the loop that builds it. Comprehensions and generators are the tools for that, and you'll see them in almost every real Python codebase.

List comprehensions

This loop builds a list of squares:

A list comprehension says the same thing in one line:

Read it left to right: "n * n, for each n in the range". Add an if at the end to keep only some items:

You can use several for clauses; they nest in the same order as written loops would:

If a comprehension needs more than one if and two fors, a normal loop is usually easier to read. Comprehensions are for building collections, not for side effects like printing.

2 exercises to try what you just read.

Exercise 1

Lap times

Press Ctrl Enter or Check to test your code.

Exercise 2

Fibonacci stream

Press Ctrl Enter or Check to test your code.

Set and dict comprehensions

Swap the brackets and you get other collections:

Generator expressions

Use round brackets and you get a generator expression. It doesn't build a list at all; it produces values one at a time, only when asked. When it is the only argument of a function you can drop the extra brackets:

Generator functions

A function that contains yield is a generator function. Calling it doesn't run the body; it returns a generator. Each time you ask for the next value, the function runs until it reaches yield, hands that value out, and pauses with all its local variables intact:

Because values are made on demand, a generator can even be infinite. The loop that uses it decides when to stop.

itertools: tools for iterators

The itertools module has building blocks that work with any iterable, lazily:

  • islice(it, n) takes the first n items, so it is the safe way to sample an infinite generator.
  • count(start) counts upward forever.
  • accumulate(it) yields running totals.
  • chain(a, b) walks through a and then b.

A generator can be consumed only once. After it is exhausted, looping over it again produces nothing, so call the generator function again if you need a fresh one.

Further reading

  • The Python Tutorial: List comprehensions
  • The Python Tutorial: Generators
  • Functional programming HOWTO
  • itertools module

A runner's lap times (whole seconds) arrive on one line separated by spaces. Laps under 30 seconds are sensor glitches and must be ignored. Using comprehensions:

  1. Build a list of the valid laps.
  2. Build a dictionary mapping lap number (starting at 1, counting only valid laps) to the time.

Print the dictionary entries as lap N: T one per line, then fastest: T. There is always at least one valid lap.

Sample input:

Sample output:

Write an infinite generator fibonacci() that yields 0, 1, 1, 2, 3, 5, 8, … forever. Read n and print the first n numbers on one line separated by spaces, taking them with itertools.islice. If n is 0, print an empty line.

Sample input:

Sample output:

squares = []
for n in range(1, 6):
    squares.append(n * n)
squares = [n * n for n in range(1, 6)]     # [1, 4, 9, 16, 25]
evens = [n for n in range(10) if n % 2 == 0]
names = [name.title() for name in ["ana", "ben"] if name]
pairs = [(x, y) for x in "ab" for y in (1, 2)]   # [('a', 1), ('a', 2), ('b', 1), ('b', 2)]
first_letters = {word[0] for word in ["ant", "ape", "bat"]}    # {'a', 'b'}
lengths = {word: len(word) for word in ["ant", "bison"]}       # {'ant': 3, 'bison': 5}
total = sum(n * n for n in range(1_000_000))    # no million-item list in memory
has_negative = any(x < 0 for x in readings)
def countdown(n):
    while n > 0:
        yield n
        n -= 1

for x in countdown(3):
    print(x)          # 3, 2, 1
from itertools import accumulate, islice

def odd_numbers():
    n = 1
    while True:
        yield n
        n += 2

print(list(islice(odd_numbers(), 5)))   # [1, 3, 5, 7, 9]
print(list(accumulate([3, 1, 4])))      # [3, 4, 8]
62 3 58 71
lap 1: 62
lap 2: 58
lap 3: 71
fastest: 58
7
0 1 1 2 3 5 8