Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

Implementing Concurrency with Coroutines in Python

Tech Jun 28 2

The Global Interpreter Lock (GIL) in CPython prevents true parallel execution with threads. Coroutines offer a lightweight alternative for achieving concurrency within a single thread through controlled switching and state preservation.

Yield-Based Approach

Using yield allows state preservation similar to OS-level thread switching, but at the code level:

import time

def compute_sum():
    total = 0
    for i in range(10_000_000):
        total += i
        yield

def counter():
    gen = compute_sum()
    count = 0
    for _ in range(10_000_000):
        count += 1
        next(gen)

start = time.time()
counter()
print("Concurrent execution time:", time.time() - start)

# Sequential execution comparison
def sequential():
    sum_total = 0
    for i in range(10_000_000):
        sum_total += i
    
    cnt = 0
    for _ in range(10_000_000):
        cnt += 1

start = time.time()
sequential()
print("Sequential execution time:", time.time() - start)

For CPU-bound tasks, this approach often decreases performance due to switching overhead.

Greenlet Module

The greenlet module providse a cleaner interface for coroutine switching:

from greenlet import greenlet

def process_a(name):
    print(f"{name} executing process A")
    gr_b.switch(name)
    print("Resuming process A")
    gr_b.switch()

def process_b(name):
    print(f"{name} executing process B")
    gr_a.switch()
    print("Resuming process B")

gr_a = greenlet(process_a)
gr_b = greenlet(process_b)
gr_a.switch("test")

Gevent for IO-Bound Tasks

Gevent combines coroutines with automatic IO detection:

import gevent
from gevent import monkey
monkey.patch_all()
import time

def fetch_data():
    print("Fetching data...")
    time.sleep(2)
    print("Data received")

def process_logs():
    print("Processing logs...")
    time.sleep(1)
    print("Logs processed")

task1 = gevent.spawn(fetch_data)
task2 = gevent.spawn(process_logs)
gevent.joinall([task1, task2])

Key considerations:

  1. Always call join() on coroutine objects
  2. Monkey patching should occur before importing blocking modules
  3. Coroutines work best for IO-bound operations, not CPU-bound tasks

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.