Python Context Managers (with)
The with statement is Python's elegant solution to
resource management — it guarantees that setup and teardown always happen, even if an exception occurs. You've
already seen it with files (with open(...)), but context managers work for
database connections, network sockets, thread locks, timers, temporary directories, and any resource that
needs guaranteed cleanup. This chapter teaches you to build your own context managers using both the class
protocol (__enter__/__exit__) and the
elegant @contextmanager generator approach.
The with Statement — Guaranteed Setup & Teardown
The with statement works through the context
manager protocol — any object with __enter__ and __exit__ methods can be used with with.
Python calls __enter__ at the start of the block and __exit__ at the end, always — even if an exception was raised.
# ── The classic file example ──────────────────────────
with open("data.txt", "w") as f:
f.write("Hello\n")
# f.close() called automatically — guaranteed!
# ── Without with — risky ──────────────────────────────
f = open("data.txt")
try:
content = f.read()
finally:
f.close() # must do this manually — with is better
# ── Multiple context managers in one with ─────────────
with (
open("input.txt", "r") as src,
open("output.txt", "w") as dst
):
dst.write(src.read().upper())
# ── The 'as' clause is optional ───────────────────────
import threading
lock = threading.Lock()
with lock: # no 'as' needed
# lock acquired
critical_section()
# lock released — even if exception occurred
# ── What Python does under the hood ──────────────────
# with EXPR as VAR:
# BODY
# translates to:
mgr = open("data.txt")
f = mgr.__enter__()
try:
pass # BODY
except:
if not mgr.__exit__(*sys.exc_info()):
raise
else:
mgr.__exit__(None, None, None)
Built-in Context Managers You Already Use
Python's standard library provides many built-in context managers. Knowing them saves you from reinventing the wheel:
import threading, sqlite3, tempfile, os
# ── Thread lock ───────────────────────────────────────
lock = threading.Lock()
with lock:
# only one thread can run this at a time
shared_list.append(42)
# ── SQLite — auto commit or rollback ──────────────────
with sqlite3.connect("app.db") as conn:
conn.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER)")
conn.execute("INSERT INTO users VALUES (1)")
# auto-committed if no exception; rolled back if one occurred
# ── Temporary directory ───────────────────────────────
with tempfile.TemporaryDirectory() as tmpdir:
print(f"Temp dir: {tmpdir}")
work_file = os.path.join(tmpdir, "data.txt")
with open(work_file, "w") as f:
f.write("temp data")
# tmpdir and all contents deleted here
# ── Decimal context — local precision ─────────────────
from decimal import Decimal, localcontext
with localcontext() as ctx:
ctx.prec = 50 # 50 decimal places locally
result = Decimal(1) / Decimal(7)
print(result)
# precision restored to default outside
Class-Based Context Managers — __enter__ & __exit__
Any class with __enter__ and __exit__ methods is a context manager. This is the most powerful approach —
full OOP, can hold state, reusable, and supports complex lifecycle management.
__init__ for configuration. Best when the CM has state, needs reuse, or
supports exception suppression.yield. Much less boilerplate.
Setup before yield, cleanup after. Can't suppress exceptions (easily).import time
# ── Timer context manager ─────────────────────────────
class Timer:
"""Measures elapsed time for a code block."""
def __init__(self, label: str = ""):
self.label = label
self.elapsed = 0.0
def __enter__(self):
self._start = time.perf_counter()
return self # 'as' variable = self
def __exit__(self, exc_type, exc_val, tb):
self.elapsed = time.perf_counter() - self._start
if self.label:
print(f"⏱ {self.label}: {self.elapsed:.4f}s")
return False # don't suppress exceptions
with Timer("List sort") as t:
data = sorted(range(1000000, 0, -1))
# ⏱ List sort: 0.0831s
print(f"Elapsed: {t.elapsed:.4f}s") # access after block
# ── Database transaction CM ───────────────────────────
class Transaction:
"""Wraps DB operations in a transaction."""
def __init__(self, connection):
self.conn = connection
def __enter__(self):
print("BEGIN transaction")
return self.conn.cursor()
def __exit__(self, exc_type, exc_val, tb):
if exc_type is None:
print("COMMIT")
self.conn.commit()
else:
print(f"ROLLBACK due to {exc_type.__name__}")
self.conn.rollback()
return False # let exception propagate
with Transaction(conn) as cursor:
cursor.execute("INSERT INTO orders VALUES (?)", (1,))
@contextmanager — The Generator Approach
The @contextlib.contextmanager decorator turns a
generator function with exactly one yield into a context manager. Code
before yield = __enter__; the yielded
value becomes the as variable; code after yield = __exit__.
from contextlib import contextmanager
import time, os
# ── Timer — generator approach ────────────────────────
@contextmanager
def timer(label: str):
start = time.perf_counter()
try:
yield # code in with block runs here
finally:
elapsed = time.perf_counter() - start
print(f"⏱ {label}: {elapsed:.4f}s")
with timer("Sorting"):
data = sorted(range(100000, 0, -1))
# ── Change directory temporarily ─────────────────────
@contextmanager
def cd(path):
"""Temporarily change the working directory."""
old = os.getcwd()
os.chdir(path)
try:
yield path
finally:
os.chdir(old) # always restore
with cd("/tmp") as p:
print(f"In: {p}")
print(f"Back: {os.getcwd()}")
# ── Yield a value ─────────────────────────────────────
@contextmanager
def managed_connection(url):
conn = create_connection(url) # setup
try:
yield conn # conn becomes 'as' variable
except Exception:
conn.rollback()
raise
finally:
conn.close() # always clean up
with managed_connection("db://localhost") as conn:
conn.execute("SELECT 1")
# ── Handle exceptions inside @contextmanager ─────────
@contextmanager
def ignore_errors(*exc_types):
try:
yield
except exc_types: # suppress these exceptions
pass
with ignore_errors(FileNotFoundError):
os.remove("maybe_missing.txt") # no crash if missing
__exit__ in Depth — Exception Handling
The __exit__ method receives exception information
when an exception occurred inside the with block. Its return value
determines whether the exception is suppressed or propagated.
# __exit__(self, exc_type, exc_val, traceback)
# exc_type — exception class (or None if no exception)
# exc_val — exception instance
# traceback — traceback object
# Return True → suppress the exception (swallow it)
# Return False → let the exception propagate
class SuppressValueError:
"""Suppress ValueError only — let everything else propagate."""
def __enter__(self): return self
def __exit__(self, exc_type, exc_val, tb):
if exc_type is ValueError:
print(f"ValueError suppressed: {exc_val}")
return True # ← suppress ValueError
return False # ← propagate everything else
with SuppressValueError():
raise ValueError("bad value") # suppressed — no crash
print("Execution continues!")
# ── Check if exception occurred ───────────────────────
class LogErrors:
def __enter__(self): return self
def __exit__(self, exc_type, exc_val, tb):
if exc_type is None:
print("Success — no exception")
else:
print(f"Exception: {exc_type.__name__}: {exc_val}")
return False # don't suppress
with LogErrors():
print("Working...") # Success — no exception
try:
with LogErrors():
raise RuntimeError("oops") # Exception: RuntimeError: oops
except RuntimeError:
pass
contextlib Utilities
The contextlib module provides ready-made tools for
common context manager patterns:
from contextlib import (
suppress, ExitStack, nullcontext,
redirect_stdout, asynccontextmanager
)
import io, os
# ── suppress — silently ignore specific exceptions ────
with suppress(FileNotFoundError, PermissionError):
os.remove("maybe_missing.txt") # no error if missing
# ── nullcontext — no-op CM (useful for optional CMs) ──
def process(data, use_lock=True):
import threading
lock = threading.Lock() if use_lock else nullcontext()
with lock: # works either way!
return data
# ── redirect_stdout — capture print() output ─────────
output = io.StringIO()
with redirect_stdout(output):
print("This goes to the buffer!")
print(f"Captured: {output.getvalue()!r}")
# ── ExitStack — manage dynamic number of CMs ──────────
with ExitStack() as stack:
# Open variable number of files
files = [
stack.enter_context(open(f, "r"))
for f in ["a.txt", "b.txt", "c.txt"]
]
# Register a cleanup callback
stack.callback(print, "All files processed")
# All files closed + callback run when block exits
# ── ExitStack for conditional CM ─────────────────────
with ExitStack() as stack:
if use_profiling:
stack.enter_context(profiler())
if use_lock:
stack.enter_context(lock)
# run code — cleanup happens regardless
run_code()
Real-World Context Manager Patterns
Context managers appear everywhere in production Python code. Here are the most important professional patterns:
from contextlib import contextmanager
import os, json, logging
# ── 1. Atomic file write — write to temp, rename ─────
@contextmanager
def atomic_write(filepath: str, encoding="utf-8"):
"""Write to temp file; rename on success, delete on failure."""
tmp = filepath + ".tmp"
try:
with open(tmp, "w", encoding=encoding) as f:
yield f
os.replace(tmp, filepath) # atomic rename
except Exception:
if os.path.exists(tmp):
os.remove(tmp)
raise
with atomic_write("config.json") as f:
json.dump({"version": "2.0"}, f, indent=2)
# config.json is either fully written or unchanged
# ── 2. Logging context ────────────────────────────────
@contextmanager
def log_operation(name: str):
"""Log start, end and duration of an operation."""
import time
logging.info(f"START {name}")
start = time.monotonic()
try:
yield
logging.info(f"OK {name} ({time.monotonic()-start:.3f}s)")
except Exception as e:
logging.error(f"FAIL {name}: {e}")
raise
with log_operation("data_import"):
import_records()
# ── 3. Environment variable override ─────────────────
@contextmanager
def env_override(**overrides):
"""Temporarily override environment variables."""
originals = {k: os.environ.get(k) for k in overrides}
os.environ.update({k: str(v) for k, v in overrides.items()})
try:
yield
finally:
for k, v in originals.items():
if v is None:
os.environ.pop(k, None)
else:
os.environ[k] = v
with env_override(DEBUG="1", DB_URL="sqlite:///:memory:"):
run_tests() # env vars restored after block
Practice Python Online — Free Compilers
Don't want to install anything yet? Practice all context manager examples from this chapter directly in your browser with these free Python environments.
- The with statement calls
__enter__on entry (its return value becomes theasvariable) and__exit__on exit — always, even if an exception occurred. - Any object with
__enter__and__exit__is a context manager. Built-ins:open(),threading.Lock(),sqlite3.connect(),tempfile.TemporaryDirectory(). - Class-based CM: implement
__enter__(self)(returns the resource) and__exit__(self, exc_type, exc_val, tb)(cleanup). Best for complex, stateful, reusable CMs. - @contextmanager generator: code before
yield= setup; yielded value =asvariable; code afteryield= cleanup. Usetry/finallyto guarantee teardown. __exit__returningTruesuppresses the exception;False/Nonepropagates it. Only suppress specific, intentional exception types.- contextlib tools:
suppress(exc),nullcontext()(no-op),redirect_stdout(),ExitStack(dynamic multiple CMs). ExitStackis essential when the number of context managers is determined at runtime — open N files, conditionally add profiling, register cleanup callbacks.- Real-world patterns: atomic file writes, DB transaction wrappers, operation logging, temp env var overrides, directory changers, connection pooling.
A class must implement the context manager protocol — two dunder methods:
__enter__(self)— called when entering thewithblock. Sets up and returns the resource (which becomes theasvariable). Should returnselfor the resource.__exit__(self, exc_type, exc_val, traceback)— called when leaving the block (always, even on exception). ReturnsTrueto suppress the exception,False/None to propagate it.
@contextlib.contextmanager turns a generator function into a
context manager. The function must yield exactly once:
- Code before
yield→ runs as__enter__(setup) - The yielded value → becomes the
asvariable - Code after
yield→ runs as__exit__(cleanup)
Wrap the yield in try/finally to guarantee cleanup even if an exception occurs in the
with block.
__exit__ returning True
suppresses the exception — execution continues after the with block as if no exception occurred. Only do this intentionally for
specific exception types.
__exit__ returning False (or None)
propagates the exception — it continues up the call stack. This is the correct
default for almost all context managers.
def __exit__(self, exc_type, exc_val, tb):
if exc_type is ValueError:
return True # ← suppress ValueError
return False # ← let everything else propagate
ExitStack is a context manager that manages a dynamic number of
other context managers. It's useful when you don't know at write-time how many CMs you need:
with ExitStack() as stack:
files = [stack.enter_context(open(f))
for f in file_list] # N files
stack.callback(cleanup_fn) # register callback
All registered CMs are cleaned up in reverse order when the with block exits.
nullcontext() is a no-op context manager — it does nothing on
enter and exit. It's useful when a CM is optional and you need to keep the with syntax consistent:
import threading
from contextlib import nullcontext
lock = threading.Lock() if use_lock else nullcontext()
with lock: # works in both cases!
critical_code()
Without nullcontext(), you'd need an
if/else to decide whether to use with or not.
with open("f.txt") as f:, what does the as f variable receive?@contextmanager generator, what does code in
the finally block after yield
guarantee?
Rate Us on Google
Join Our Learning Community
Connect with thousands of learners, get daily tips, ask questions and stay updated with new tutorials.