Python Generator Expressions

Introduction

Python generator expressions offer a succinct and efficient way of creating simple generators. Generator expression syntax is similar to that of list comprehensions. And much like those, generator expressions allow you to generate sequences by applying expressions to items in an iterable and potentially filtering them based on specified conditions.

Basic Python Generator Expression Syntax

The basic generator expression syntax has parentheses () that surround an expression followed by for loop-like syntax.

(expression for item in iterable)
  • expression: An expression that generates values.
  • item: A variable that represents each item in the iterable.
  • iterable: The iterable object that produces the values to compute the expression. This can be any object type that implements the iterable protocol.

Aside from using parentheses (), the generator expression syntax is identical to the list comprehension syntax, (which uses square brackets []).

Creating a Simple Generator With a Generator Expression

Let’s start with a straightforward example. Suppose you want to create a generator that yields the squares of numbers from 1 to 5.

Example:

squares_generator = (x**2 for x in range(1, 6))
for num in squares_generator:
    print(num)

Output:

2
4
9
16
25

In this example:

  • x**2: The expression that calculates the square of each number.
  • range(1, 6): A range, which is an iterable, generating numbers from 1 to 5.
  • for x in range(1, 6): Part of the generator expression that iterates through numbers from 1 to 5.
  • for num in squares_generator:: Defines the loop that prints all the generated numbers.

In the example above, the generator expression creates a generator that is assigned to the squares_generator variable (line 1). squares_generator is iterated over by the for loop (line 2), which prints its values (line 3).

Comparing Generator Expressions to Generator Functions

To make the comparison, the following example shows how the generator in the previous example can be created with a generator function.

Example:

def squares_generator_fn():
    for x in range(1, 6):
        yield x**2

for num in squares_generator_fn():
    print(num)

Output:

2
4
9
16
25

As you can see, the expression-based syntax is more compact for generating this type of generator.

Additionally, a generator expression creates a generator object, which can be iterated over directly. This is a little different from the behavior of a generator function, which has to be called to get a generator.

The next example demonstrates this difference by inspecting the object types of the generator expression squares_generator and generator function squares_generator_fn from above.

Example:

print(squares_generator)
print(squares_generator_fn)

Output:

<generator object <genexpr> at 0x7fdb5b1bf610>
<function squares_generator_fn at 0x7fdb5b272e60>

The example above prints squares_generator and squares_generator_fn. As the types in the output show, squares_generator, printed first, is a generator object, and squares_generator_fn, printed second, is a function.

Differences Between Generator Expressions and List Comprehensions

If you have read the list comprehensions lesson, you now also know all about generator expressions as well.

Generator expressions are different from list comprehensions in only these two ways:

  1. Generator expressions use parentheses instead of square brackets.
  2. Generator expressions create generators instead of lists.

Generator Expressions Should Be Based on Ranges and Other Generators

While generator expressions, like list comprehensions, can technically operate on any iterable, they should be based on ranges and other generators. This is because the main purpose of generators is to create sequences of values on-the-fly, without storing them all in memory. Creating generators out of collection objects that are stored in memory (such as lists or tuples) defeats this purpose.

Example:

numbers_lst = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
wrong_generator = (x**2 for x in numbers_lst) 

correct_generator = (x**2 for x in range(1, 11))

In the example above, wrong_generator generates squares on-the-fly, but it’s based on the list numbers_lst, which stores its values in memory all at once. Consequently, it takes the same amount of memory as a list of squares. On the other hand, correct_generator is based on a range object, and therefore its values are truly generated one by one, without using memory for all the values at once.

Using if Clauses in Generator Expressions

Generator expressions, like list comprehensions, can also include an optional if clause to filter elements from the original iterable based on a condition.

Here’s the syntax for using an if clause:

(expression for item in iterable if condition)
  • condition: A boolean expression that determines whether an item should be included in the generated sequence.

Let’s see an example where we use an if clause to generate only squares of even numbers

Example:

even_squares = (x**2 for x in range(1, 11) if x % 2 == 0)
for num in even_squares:
    print(num)

Output:

4
16
36
64
100

In the example above, the if x % 2 == 0 condition checks whether a number is even using the remainder upon division operator %. Only the squares of the even numbers within the range of 1 to 10 (range(1, 11)) are included in the generated sequence even_squares.

Other Generator Expressions Capabilities

Much like list comprehensions, in addition to if clauses, generator expressions also support ternary operators, and nesting. However, since these features also very closely resemble those of list comprehensions, I’ll refer you to the list comprehension lesson to learn about the latter two.

Summary & Reference for Python Generator Expressions

Generator expressions are a concise and efficient way to create generators by applying expressions to items in existing iterables.

The basic generator expression syntax has parentheses () that surround an expression followed by for loop-like syntax.

(expression for item in iterable)

Here is an example that creates a generator of the squares of numbers from 1 to 5:

squares_generator = (x**2 for x in range(1, 6))

It’s recommended to base generator expressions on ranges and other generators rather than collections stored in memory to fully utilize their memory-efficient behavior.

Generator expressions create generator objects directly. This is different from generator functions, which return generator objects only when called.

Generator expressions can include if clauses for filtering elements based on conditions.

even_squares = (x**2 for x in range(1, 11) if x % 2 == 0)

Much like list comprehensions, generator expressions also support ternary operators and nesting for more complex scenarios.

Tags

Related Posts