Python Generators and Iterators — yield, next, and Lazy Sequences
A list holds every value at once. A generator produces one value when you ask for it and pauses until you ask again. That one difference makes generators the right tool for long sequences, large files, and pipelines where building the whole list first would waste memory and time. This guide shows the working core: writing a generator with yield, driving it by hand with next, spotting the moment it runs out, choosing between a generator expression and a list comprehension, stopping an endless generator on purpose, the two methods every iterator needs, and handing work to a second generator with yield from.
Written November 12, 2026 · Tested with Python 3.12.3 · 8 min read
- The main program: a generator with yield
- Driving a generator by hand: next and StopIteration
- Generator expressions: lazy where a list is eager
- Endless generators and stopping on purpose
Written November 12, 2026
Tested with Python 3.12.3
What this guide is
An iterable is something a for loop can walk: a list, a string, a file. An iterator is the object that hands out the values one at a time and remembers where it stopped. A generator is the shortest way to write an iterator: you write a function that yields values instead of returning a finished list, and Python builds the iterator for you. The function body does not run when you call it. It runs a little at a time, pausing at each yield, until the values run out.
The main program: a generator with yield
A function that contains yield is a generator function. Calling it returns a generator object and runs none of the body. Each pass through a loop asks for the next value, the body runs up to the next yield, and that value comes out:
def squares(limit):
n = 0
while n < limit:
yield n * n
n += 1
print(list(squares(5)))
for value in squares(3):
print(value)
print(list(squares(0)))
The exact output:
[0, 1, 4, 9, 16]
0
1
4
[]
The list call pulls values until the generator runs out, so it collects all five squares. The for loop does the same thing one value at a time. A limit of zero makes the while condition false at once, so nothing is yielded and the list is empty. Each call to squares builds a fresh generator with its own position, which is why the same function can feed a list in one line and a loop in the next without the two interfering.
Driving a generator by hand: next and StopIteration
A for loop hides two steps that the next function makes visible. Calling next on an iterator returns its next value. When no values remain, next raises StopIteration. A generator also stays used up: once it runs out, asking again gives nothing more:
def countdown(n):
while n > 0:
yield n
n -= 1
g = countdown(3)
print(next(g))
print(next(g))
print(list(g))
print(list(g))
try:
next(g)
except StopIteration:
print("StopIteration: exhausted")
The exact output:
3
2
[1]
[]
StopIteration: exhausted
Two next calls take 3 and 2. The list call then collects what is left, which is just 1, and the generator is empty from that point on. A second list call returns an empty list instead of restarting, and a further next raises StopIteration. That one-way behavior is the fact to design around: if you need to walk the same values twice, either build a list once and walk the list, or call the generator function again to get a fresh generator.
Generator expressions: lazy where a list is eager
Square brackets around a comprehension build the full list at once. Round brackets make a generator expression, which computes each value only when something pulls it. Consuming a generator expression spends it, in the same way as any other generator:
squares_list = [n * n for n in range(5)]
print(squares_list)
gen = (n * n for n in range(5))
print(next(gen))
print(sum(gen))
print(sum(gen))
print(list(gen))
The exact output:
[0, 1, 4, 9, 16]
0
30
0
[]
The first next takes the 0, so the sum that follows adds only the four values that remain: 1, 4, 9, and 16 make 30. After that the generator is spent, so a second sum is 0 and a list call is empty. Pick the list form when you need the values more than once, need their length, or need to index into them. Pick the generator form when you walk the values once, as sum, max, and a for loop all do, and the list itself would be thrown away.
Endless generators and stopping on purpose
Because a generator runs only when pulled, it may describe a sequence that never ends. An endless source is safe as long as the side that pulls it decides when to stop. The break statement in a for loop, or a fixed count of next calls, supplies that stop:
def naturals():
n = 0
while True:
yield n
n += 1
taken = []
for value in naturals():
if len(taken) >= 5:
break
taken.append(value)
print(taken)
evens = (n for n in naturals() if n % 2 == 0)
print([next(evens) for _ in range(4)])
The exact output:
[0, 1, 2, 3, 4]
[0, 2, 4, 6]
The loop takes five values and breaks, so the endless generator simply stops being pulled and no more of it runs. The second half filters one endless generator through a generator expression and takes four values from that. Chaining small generators this way is a common shape for real work: one stage produces values, the next filters or changes them, and the final stage takes only as many as it needs, so the cost follows the amount actually used.
The iterator protocol: iter and next
A generator function writes the iterator for you, but the protocol underneath has two methods. The iter method returns the iterator itself. The next method returns the next value, or raises StopIteration when the values run out. A class with both can be used anywhere a for loop or list call works:
class Step:
def __init__(self, start, stop, step):
self.current = start
self.stop = stop
self.step = step
def __iter__(self):
return self
def __next__(self):
if self.current >= self.stop:
raise StopIteration
value = self.current
self.current += self.step
return value
s = Step(0, 10, 3)
print(list(s))
print(list(s))
values = [10, 20]
print(list(iter(values)))
print(list(iter(values)))
The exact output:
[0, 3, 6, 9]
[]
[10, 20]
[10, 20]
The Step object is its own iterator, so the first list call uses it up and the second returns nothing. A list behaves differently: a list is an iterable, not an iterator, and each iter call on it returns a new iterator positioned at the start, so both list calls collect the full contents. Most code should reach for a generator function first, since it states the same logic with less scaffolding. Writing the class is worth it when the iteration carries state or methods that a plain function cannot hold cleanly.
Delegating with yield from
One generator can hand its work to another with yield from. The values of the inner generator come out in order, as if the outer generator had yielded them itself. When the inner generator returns a value, that value becomes the result of the yield from expression in the outer generator:
def sub():
yield "a"
yield "b"
return "sub done"
def outer():
result = yield from sub()
print("sub returned:", result)
yield "c"
print(list(outer()))
The exact output:
sub returned: sub done
['a', 'b', 'c']
The list collects a, b, and then c. While the list is still being built, the finished inner generator hands its return value back, and the outer generator prints it before yielding c. A plain return in a normal function ends the function and gives a value to the caller. A return inside a generator ends that generator, and yield from is how a caller generator receives that final value instead of losing it.
Which form for which job
| Job | Form | Why |
|---|---|---|
| Produce a sequence one value at a time | A function with yield | The caller pulls values; nothing is built ahead |
| Walk values once and discard them | A generator expression | No throwaway list is held in memory |
| Walk values twice, or index them | Build a list | A generator is spent after one pass |
| Drive an iterator step by step | next, catching StopIteration | The two steps a for loop hides become explicit |
| Describe an endless source | A generator with a loop that never ends | Safe while the pulling side stops on purpose |
| Filter or change a stream | A generator expression over a generator | Stages chain; cost follows what is used |
| Hold iteration state in an object | A class with iter and next | The state travels with the object |
| Reuse a second generator inside a first | yield from | Inner values flow out; the return value comes back |
Checklist
- Before returning a list from a function, ask whether the caller walks it once; if so, yield the values instead.
- Before walking a generator a second time, call the generator function again or store the first pass in a list; a spent generator returns nothing.
- Before summing or looping over a comprehension, decide whether the list itself is needed later; if not, use the round-bracket form.
- Before writing an endless loop in a generator, name the stop on the pulling side: a break, a count, or a condition.
- Before writing an iterator class, try a generator function; keep the class for state a function cannot hold cleanly.
- Before ignoring the return value of an inner generator, check whether it carries a result; yield from is how you receive it.
Related guides on this site
- Python Loops and Conditionals — for, while, if, range, and enumerate — a for loop is the main way generators are consumed, and this guide shows what the loop does with each value.
- Python Functions — def, Default Arguments, and Functions as Values — a generator is a function whose return is spread over many yields instead of one return.
- Python Classes and Objects — init, Methods, Inheritance, and Dataclasses — the iter and next methods are class methods of the same kind that guide covers.
- Python Lists, Dicts, and Sets — The Three Collections You Use Every Day — a list is the eager counterpart of a generator, and this guide covers when holding every value at once is the right call.
Sources
- Python Tutorial, Classes: Iterators and Generators — the iterator protocol, generator functions, and generator expressions.
- Python Language Reference, Expressions — yield expressions, yield from, and generator expression semantics.
- Python Standard Library, Iterator Types — the iterator protocol contract and StopIteration.
Keep Exploring
The full Python hub \u2014 language basics, the standard library, and data tools.
Every guide on the site in one list, grouped by topic.
Every browser-only checker and verifier on the site.
The C++, Python, Sysadmin, Rust, Go, and TypeScript hubs.