Tutorials Python Python Context Managers
Ch 45 / 58
Python Intermediate 13 min read Chapter 45 of 58

Python Context Managers (with)

The with statement is Python's elegant solution to resource management — it guarantees that setup and teardown always happen, even if an exception occurs. You've already seen it with files (with open(...)), but context managers work for database connections, network sockets, thread locks, timers, temporary directories, and any resource that needs guaranteed cleanup. This chapter teaches you to build your own context managers using both the class protocol (__enter__/__exit__) and the elegant @contextmanager generator approach.

What you'll learn in this chapter
with statement lifecycle and protocol
Built-in context managers you already use
Custom class-based context managers
@contextmanager — generator-based approach
contextlib utilities — suppress, ExitStack
Real-world resource management patterns
Advertisement

The with Statement — Guaranteed Setup & Teardown

The with statement works through the context manager protocol — any object with __enter__ and __exit__ methods can be used with with. Python calls __enter__ at the start of the block and __exit__ at the end, always — even if an exception was raised.

with statement lifecycle — what Python does
1. cm.__enter__() called → setup resource
▼
return value bound to 'as' variable
2. as variable = __enter__ return value
▼
3. with block body executes
▼
always — even on exception
4. cm.__exit__(exc_type, exc_val, tb) called
▼
✅ Cleanup guaranteed — files closed, locks released
Python — with_statement_basics.py
# ── The classic file example ──────────────────────────
with open("data.txt", "w") as f:
    f.write("Hello\n")
# f.close() called automatically — guaranteed!

# ── Without with — risky ──────────────────────────────
f = open("data.txt")
try:
    content = f.read()
finally:
    f.close()              # must do this manually — with is better

# ── Multiple context managers in one with ─────────────
with (
    open("input.txt",  "r") as src,
    open("output.txt", "w") as dst
):
    dst.write(src.read().upper())

# ── The 'as' clause is optional ───────────────────────
import threading
lock = threading.Lock()

with lock:                  # no 'as' needed
    # lock acquired
    critical_section()
# lock released — even if exception occurred

# ── What Python does under the hood ──────────────────
# with EXPR as VAR:
#     BODY
# translates to:
mgr = open("data.txt")
f   = mgr.__enter__()
try:
    pass  # BODY
except:
    if not mgr.__exit__(*sys.exc_info()):
        raise
else:
    mgr.__exit__(None, None, None)

Built-in Context Managers You Already Use

Python's standard library provides many built-in context managers. Knowing them saves you from reinventing the wheel:

open()
Auto-closes files
threading.Lock()
Acquire/release locks
sqlite3.connect()
DB transaction commit/rollback
tempfile.TemporaryDirectory()
Auto-deletes temp dirs
socket.socket()
Closes network sockets
contextlib.suppress()
Silences specific exceptions
zipfile.ZipFile()
Auto-closes archives
decimal.localcontext()
Temp precision settings
Python — builtin_context_managers.py
import threading, sqlite3, tempfile, os

# ── Thread lock ───────────────────────────────────────
lock = threading.Lock()
with lock:
    # only one thread can run this at a time
    shared_list.append(42)

# ── SQLite — auto commit or rollback ──────────────────
with sqlite3.connect("app.db") as conn:
    conn.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER)")
    conn.execute("INSERT INTO users VALUES (1)")
# auto-committed if no exception; rolled back if one occurred

# ── Temporary directory ───────────────────────────────
with tempfile.TemporaryDirectory() as tmpdir:
    print(f"Temp dir: {tmpdir}")
    work_file = os.path.join(tmpdir, "data.txt")
    with open(work_file, "w") as f:
        f.write("temp data")
# tmpdir and all contents deleted here

# ── Decimal context — local precision ─────────────────
from decimal import Decimal, localcontext
with localcontext() as ctx:
    ctx.prec = 50                   # 50 decimal places locally
    result = Decimal(1) / Decimal(7)
    print(result)
# precision restored to default outside

Class-Based Context Managers — __enter__ & __exit__

Any class with __enter__ and __exit__ methods is a context manager. This is the most powerful approach — full OOP, can hold state, reusable, and supports complex lifecycle management.

Class-based — best for complex cases
Full class with attributes. Can implement __init__ for configuration. Best when the CM has state, needs reuse, or supports exception suppression.
@contextmanager generator — best for simple cases
Function with yield. Much less boilerplate. Setup before yield, cleanup after. Can't suppress exceptions (easily).
Python — class_context_manager.py
import time

# ── Timer context manager ─────────────────────────────
class Timer:
    """Measures elapsed time for a code block."""

    def __init__(self, label: str = ""):
        self.label   = label
        self.elapsed = 0.0

    def __enter__(self):
        self._start = time.perf_counter()
        return self              # 'as' variable = self

    def __exit__(self, exc_type, exc_val, tb):
        self.elapsed = time.perf_counter() - self._start
        if self.label:
            print(f"⏱ {self.label}: {self.elapsed:.4f}s")
        return False             # don't suppress exceptions

with Timer("List sort") as t:
    data = sorted(range(1000000, 0, -1))
# ⏱ List sort: 0.0831s
print(f"Elapsed: {t.elapsed:.4f}s")   # access after block

# ── Database transaction CM ───────────────────────────
class Transaction:
    """Wraps DB operations in a transaction."""

    def __init__(self, connection):
        self.conn = connection

    def __enter__(self):
        print("BEGIN transaction")
        return self.conn.cursor()

    def __exit__(self, exc_type, exc_val, tb):
        if exc_type is None:
            print("COMMIT")
            self.conn.commit()
        else:
            print(f"ROLLBACK due to {exc_type.__name__}")
            self.conn.rollback()
        return False             # let exception propagate

with Transaction(conn) as cursor:
    cursor.execute("INSERT INTO orders VALUES (?)", (1,))
Advertisement
K2infocom Python Masterclass & Backend Development
Master Python, Django & Backend APIs with AI
Go from absolute beginner to building production-ready REST APIs, web applications, and ML pipelines. Real projects, live mentoring in Vadodara, and placement support included.
2,400+
Students Enrolled
4.9 ⭐
Google Rating
6 Months
Full Program
100%
Placement Support
Python 3 Django & DRF FastAPI PostgreSQL Machine Learning Docker & AWS Real Projects AI Tools

@contextmanager — The Generator Approach

The @contextlib.contextmanager decorator turns a generator function with exactly one yield into a context manager. Code before yield = __enter__; the yielded value becomes the as variable; code after yield = __exit__.

Python — contextmanager_generator.py
from contextlib import contextmanager
import time, os

# ── Timer — generator approach ────────────────────────
@contextmanager
def timer(label: str):
    start = time.perf_counter()
    try:
        yield                         # code in with block runs here
    finally:
        elapsed = time.perf_counter() - start
        print(f"⏱ {label}: {elapsed:.4f}s")

with timer("Sorting"):
    data = sorted(range(100000, 0, -1))

# ── Change directory temporarily ─────────────────────
@contextmanager
def cd(path):
    """Temporarily change the working directory."""
    old = os.getcwd()
    os.chdir(path)
    try:
        yield path
    finally:
        os.chdir(old)               # always restore

with cd("/tmp") as p:
    print(f"In: {p}")
print(f"Back: {os.getcwd()}")

# ── Yield a value ─────────────────────────────────────
@contextmanager
def managed_connection(url):
    conn = create_connection(url)    # setup
    try:
        yield conn                    # conn becomes 'as' variable
    except Exception:
        conn.rollback()
        raise
    finally:
        conn.close()                  # always clean up

with managed_connection("db://localhost") as conn:
    conn.execute("SELECT 1")

# ── Handle exceptions inside @contextmanager ─────────
@contextmanager
def ignore_errors(*exc_types):
    try:
        yield
    except exc_types:               # suppress these exceptions
        pass

with ignore_errors(FileNotFoundError):
    os.remove("maybe_missing.txt")  # no crash if missing

__exit__ in Depth — Exception Handling

The __exit__ method receives exception information when an exception occurred inside the with block. Its return value determines whether the exception is suppressed or propagated.

Python — exit_details.py
# __exit__(self, exc_type, exc_val, traceback)
# exc_type  — exception class (or None if no exception)
# exc_val   — exception instance
# traceback — traceback object
# Return True  → suppress the exception (swallow it)
# Return False → let the exception propagate

class SuppressValueError:
    """Suppress ValueError only — let everything else propagate."""

    def __enter__(self): return self

    def __exit__(self, exc_type, exc_val, tb):
        if exc_type is ValueError:
            print(f"ValueError suppressed: {exc_val}")
            return True   # ← suppress ValueError
        return False       # ← propagate everything else

with SuppressValueError():
    raise ValueError("bad value")    # suppressed — no crash
print("Execution continues!")

# ── Check if exception occurred ───────────────────────
class LogErrors:
    def __enter__(self): return self

    def __exit__(self, exc_type, exc_val, tb):
        if exc_type is None:
            print("Success — no exception")
        else:
            print(f"Exception: {exc_type.__name__}: {exc_val}")
        return False           # don't suppress

with LogErrors():
    print("Working...")       # Success — no exception

try:
    with LogErrors():
        raise RuntimeError("oops")  # Exception: RuntimeError: oops
except RuntimeError:
    pass

contextlib Utilities

The contextlib module provides ready-made tools for common context manager patterns:

Python — contextlib_tools.py
from contextlib import (
    suppress, ExitStack, nullcontext,
    redirect_stdout, asynccontextmanager
)
import io, os

# ── suppress — silently ignore specific exceptions ────
with suppress(FileNotFoundError, PermissionError):
    os.remove("maybe_missing.txt")   # no error if missing

# ── nullcontext — no-op CM (useful for optional CMs) ──
def process(data, use_lock=True):
    import threading
    lock = threading.Lock() if use_lock else nullcontext()
    with lock:                          # works either way!
        return data

# ── redirect_stdout — capture print() output ─────────
output = io.StringIO()
with redirect_stdout(output):
    print("This goes to the buffer!")
print(f"Captured: {output.getvalue()!r}")

# ── ExitStack — manage dynamic number of CMs ──────────
with ExitStack() as stack:
    # Open variable number of files
    files = [
        stack.enter_context(open(f, "r"))
        for f in ["a.txt", "b.txt", "c.txt"]
    ]
    # Register a cleanup callback
    stack.callback(print, "All files processed")
# All files closed + callback run when block exits

# ── ExitStack for conditional CM ─────────────────────
with ExitStack() as stack:
    if use_profiling:
        stack.enter_context(profiler())
    if use_lock:
        stack.enter_context(lock)
    # run code — cleanup happens regardless
    run_code()

Real-World Context Manager Patterns

Context managers appear everywhere in production Python code. Here are the most important professional patterns:

Python — real_world_cm.py
from contextlib import contextmanager
import os, json, logging

# ── 1. Atomic file write — write to temp, rename ─────
@contextmanager
def atomic_write(filepath: str, encoding="utf-8"):
    """Write to temp file; rename on success, delete on failure."""
    tmp = filepath + ".tmp"
    try:
        with open(tmp, "w", encoding=encoding) as f:
            yield f
        os.replace(tmp, filepath)          # atomic rename
    except Exception:
        if os.path.exists(tmp):
            os.remove(tmp)
        raise

with atomic_write("config.json") as f:
    json.dump({"version": "2.0"}, f, indent=2)
# config.json is either fully written or unchanged

# ── 2. Logging context ────────────────────────────────
@contextmanager
def log_operation(name: str):
    """Log start, end and duration of an operation."""
    import time
    logging.info(f"START {name}")
    start = time.monotonic()
    try:
        yield
        logging.info(f"OK {name} ({time.monotonic()-start:.3f}s)")
    except Exception as e:
        logging.error(f"FAIL {name}: {e}")
        raise

with log_operation("data_import"):
    import_records()

# ── 3. Environment variable override ─────────────────
@contextmanager
def env_override(**overrides):
    """Temporarily override environment variables."""
    originals = {k: os.environ.get(k) for k in overrides}
    os.environ.update({k: str(v) for k, v in overrides.items()})
    try:
        yield
    finally:
        for k, v in originals.items():
            if v is None:
                os.environ.pop(k, None)
            else:
                os.environ[k] = v

with env_override(DEBUG="1", DB_URL="sqlite:///:memory:"):
    run_tests()    # env vars restored after block

Practice Python Online — Free Compilers

Don't want to install anything yet? Practice all context manager examples from this chapter directly in your browser with these free Python environments.

▶
Online Python 3 Compiler
Write, run and debug Python code in your browser. No installation required. Test every context manager example instantly.
Open Compiler
Replit
Full IDE online
Google Colab
Free GPU notebooks
JupyterLite
Notebooks in browser
Programiz
Beginner-friendly
Chapter Summary
  • The with statement calls __enter__ on entry (its return value becomes the as variable) and __exit__ on exit — always, even if an exception occurred.
  • Any object with __enter__ and __exit__ is a context manager. Built-ins: open(), threading.Lock(), sqlite3.connect(), tempfile.TemporaryDirectory().
  • Class-based CM: implement __enter__(self) (returns the resource) and __exit__(self, exc_type, exc_val, tb) (cleanup). Best for complex, stateful, reusable CMs.
  • @contextmanager generator: code before yield = setup; yielded value = as variable; code after yield = cleanup. Use try/finally to guarantee teardown.
  • __exit__ returning True suppresses the exception; False/None propagates it. Only suppress specific, intentional exception types.
  • contextlib tools: suppress(exc), nullcontext() (no-op), redirect_stdout(), ExitStack (dynamic multiple CMs).
  • ExitStack is essential when the number of context managers is determined at runtime — open N files, conditionally add profiling, register cleanup callbacks.
  • Real-world patterns: atomic file writes, DB transaction wrappers, operation logging, temp env var overrides, directory changers, connection pooling.
Free Resource
Download our Python Context Managers Cheat Sheet
with statement, __enter__/__exit__, @contextmanager, contextlib tools, ExitStack & real-world patterns — one printable page.
Get Sheet
Advertisement
Quick Check — Test Your Understanding
Question & Answer
✓ Answer

A class must implement the context manager protocol — two dunder methods:

  • __enter__(self) — called when entering the with block. Sets up and returns the resource (which becomes the as variable). Should return self or the resource.
  • __exit__(self, exc_type, exc_val, traceback) — called when leaving the block (always, even on exception). Returns True to suppress the exception, False/None to propagate it.
✓ Answer

@contextlib.contextmanager turns a generator function into a context manager. The function must yield exactly once:

  • Code before yield → runs as __enter__ (setup)
  • The yielded value → becomes the as variable
  • Code after yield → runs as __exit__ (cleanup)

Wrap the yield in try/finally to guarantee cleanup even if an exception occurs in the with block.

✓ Answer

__exit__ returning True suppresses the exception — execution continues after the with block as if no exception occurred. Only do this intentionally for specific exception types.

__exit__ returning False (or None) propagates the exception — it continues up the call stack. This is the correct default for almost all context managers.

def __exit__(self, exc_type, exc_val, tb):
    if exc_type is ValueError:
        return True   # ← suppress ValueError
    return False       # ← let everything else propagate
✓ Answer

ExitStack is a context manager that manages a dynamic number of other context managers. It's useful when you don't know at write-time how many CMs you need:

with ExitStack() as stack:
    files = [stack.enter_context(open(f))
             for f in file_list]  # N files
    stack.callback(cleanup_fn)    # register callback

All registered CMs are cleaned up in reverse order when the with block exits.

✓ Answer

nullcontext() is a no-op context manager — it does nothing on enter and exit. It's useful when a CM is optional and you need to keep the with syntax consistent:

import threading
from contextlib import nullcontext

lock = threading.Lock() if use_lock else nullcontext()
with lock:           # works in both cases!
    critical_code()

Without nullcontext(), you'd need an if/else to decide whether to use with or not.

MCQ 1 of 3
In with open("f.txt") as f:, what does the as f variable receive?
MCQ 2 of 3
In a @contextmanager generator, what does code in the finally block after yield guarantee?
MCQ 3 of 3
Which contextlib tool silently suppresses specific exceptions without needing try/except/pass?

Rate Us on Google

4.8 / 5
Enjoying this tutorial?
Your 5-star review helps other learners discover our free tutorials and encourages us to keep creating quality content.
80+ reviews
Write a Google Review
Join WhatsApp Channel