Understanding Python Iterators and Generators for Efficient Collections Access
An iterator in Python provides a uniform way to traverse collections without exposing underlying structure. It captures the current location during traversal and proceeds strictly forward, never backward. Every iterator exposes two essential functions: iter() to obtain the object and next() to retrieve successive items. Strings, lists, and tuples can all produce iterators.
animals = ["eagle", "fox", "shark", "panda", "lynx"]
iterator_obj = iter(animals)
for item in iterator_obj:
print(item, end=" ")
Output:
eagle fox shark panda lynx
Alternatively, next() gives manual control over iteration, raising StopIteration once the collection is exhausted.
import sys
animals = ["eagle", "fox", "shark", "panda", "lynx"]
iterator_obj = iter(animals)
while True:
try:
print(next(iterator_obj))
except StopIteration:
sys.exit()
Output:
eagle
fox
shark
panda
lynx
Creating Custom Iterators
To turn a class into an iterator, implement __iter__() and __next__(). The constructor __init__() handles initial setup, while __iter__() returns the iterator object itself, which must provide the __next__() method. Iteration stops by raising StopIteration.
A counter satrting at 1 and incrementing by 1 can be built as follows:
class SimpleCounter:
def __iter__(self):
self.current = 1
return self
def __next__(self):
val = self.current
self.current += 1
return val
counter = SimpleCounter()
counter_iter = iter(counter)
print(next(counter_iter))
print(next(counter_iter))
print(next(counter_iter))
Output:
1
2
3
Using StopIteration to Limit Iteration
The StopIteration exception prevents infinite loops. Implement logic inside __next__() so that after a specified number of steps the exception gets triggered. The following iterator stops after five values:
class BoundedCounter:
def __iter__(self):
self.current = 1
return self
def __next__(self):
if self.current <= 5:
val = self.current
self.current += 1
return val
raise StopIteration
counter = BoundedCounter()
counter_iter = iter(counter)
for number in counter_iter:
print(number)
Output:
1
2
3
4
5
Building Generators with yield
A generator function uses the yield statement to produce a series of values on demand. Unlike a typical function that returns a single value, a generator returned by such a function is itself an iterator. Each time yield is reached, the current state is preserved and the value is returned. Subsequent next() calls resume just past the yield.
def count_up_to(limit):
n = 0
while n < limit:
yield n
n += 1
numbers = count_up_to(3)
print(next(numbers))
print(next(numbers))
print(next(numbers))
# Next call raises StopIteration
Output:
0
1
2