Tutorials Python Python Standard Library
Ch 31 / 58
Python Intermediate 14 min read Chapter 31 of 58

Python Standard Library

Python ships with "batteries included" — a massive standard library of 200+ modules covering almost everything you'll ever need: file system operations, date & time handling, JSON encoding, regular expressions, data structures, mathematics, networking, and more. The best part? No installation required. This chapter is your practical guide to the most essential standard library modules with real, copy-paste examples you'll use in professional projects every day.

What you'll learn in this chapter
os & pathlib — file system and path operations
sys — interpreter info, argv and sys.path
math & random — numbers and randomisation
datetime — dates, times and timedelta
json — encode and decode JSON data
re, collections, itertools, functools
Advertisement

Standard Library Overview — Batteries Included

Python's standard library is divided into functional categories. Every module listed here is part of a standard Python installation — no pip install needed. This is a curated map of the most useful modules you'll encounter in professional Python development:

File System
os, pathlib
shutil, glob
Interpreter
sys, gc
inspect, ast
Numbers
math, random
decimal, fractions
Date & Time
datetime
time, calendar
Data Formats
json, csv
xml, configparser
Text / Regex
re, string
textwrap, difflib
Data Structures
collections
heapq, bisect
Functional
itertools
functools, operator
Internet / IO
urllib, http
socket, email
Security
hashlib, hmac
secrets, ssl
Testing
unittest
doctest, mock
Profiling / Debug
logging, pdb
timeit, cProfile
How to explore the standard library Use help(module) in the Python REPL for full documentation, dir(module) to list all names, and visit docs.python.org/3/library/ for the complete reference.

os & pathlib — File System Operations

The os module provides a platform-independent interface to the operating system — directory traversal, file operations, environment variables, and process management. The modern pathlib module (Python 3.4+) provides an object-oriented path API that is cleaner and more intuitive than string-based paths.

Python — os_and_pathlib.py
import os

# Current directory and navigation
print(os.getcwd())                    # current working directory
os.chdir("/tmp")                      # change directory

# Directory operations
os.mkdir("new_dir")                    # create directory
os.makedirs("a/b/c", exist_ok=True)  # create nested dirs
print(os.listdir("."))                # list directory contents
os.rmdir("new_dir")                   # remove empty directory

# Path operations (legacy — prefer pathlib)
path = os.path.join("/home", "user", "file.txt")
print(os.path.exists(path))           # True/False
print(os.path.isfile(path))            # is it a file?
print(os.path.isdir("/home"))          # is it a directory?
print(os.path.basename(path))          # file.txt
print(os.path.dirname(path))           # /home/user
print(os.path.splitext("file.txt"))   # ('file', '.txt')
print(os.path.getsize(path))           # file size in bytes

# Environment variables
print(os.getenv("HOME"))              # /home/username
print(os.getenv("DB_URL", "default")) # with fallback
os.environ["MY_VAR"] = "hello"

# Walk a directory tree recursively
for root, dirs, files in os.walk("."):
    for f in files:
        print(os.path.join(root, f))

# ── pathlib — modern, object-oriented approach ────────
from pathlib import Path

p = Path("/home/user/docs/report.pdf")
print(p.name)       # report.pdf
print(p.stem)       # report (no extension)
print(p.suffix)     # .pdf
print(p.parent)     # /home/user/docs
print(p.exists())   # True/False

# Path joining with / operator (elegant!)
base = Path("/home/user")
config_file = base / "config" / "settings.json"
print(config_file)  # /home/user/config/settings.json

# Read/write with pathlib
p2 = Path("hello.txt")
p2.write_text("Hello, world!")
print(p2.read_text())                  # Hello, world!

# Glob — find files by pattern
for py_file in Path(".").glob("*.py"):
    print(py_file)

sys — Python Interpreter Interface

The sys module provides access to Python interpreter internals — command-line arguments, module search paths, stdin/stdout/stderr, the Python version, the recursion limit, and more.

Python — sys_module.py
import sys

# Python version and platform
print(sys.version)            # '3.12.0 (main, Oct 2 2023...)'
print(sys.version_info)       # sys.version_info(major=3, minor=12...)
print(sys.platform)           # 'linux', 'win32', 'darwin'

# Command-line arguments
# If called as: python script.py hello world
print(sys.argv)               # ['script.py', 'hello', 'world']
print(sys.argv[0])            # script.py — the script name

if len(sys.argv) > 1:
    print(f"First arg: {sys.argv[1]}")

# Exit the program
# sys.exit(0)   # 0 = success, non-zero = error
# sys.exit("Error: config not found")  # print message and exit

# Module search path
print(sys.path)               # list of directories Python searches for modules
sys.path.append("/my/custom/path")  # add custom module directory

# Stdin / stdout / stderr
sys.stdout.write("Hello\n")    # same as print()
sys.stderr.write("Error!\n")   # write to error stream

# Memory size of an object
print(sys.getsizeof([1,2,3]))  # size in bytes

# Recursion limit
print(sys.getrecursionlimit()) # 1000 (default)
sys.setrecursionlimit(5000)
Advertisement
K2infocom Python Masterclass & Backend Development
Master Python, Django & Backend APIs with AI
Go from absolute beginner to building production-ready REST APIs, web applications, and ML pipelines. Real projects, live mentoring in Vadodara, and placement support included.
2,400+
Students Enrolled
4.9 ⭐
Google Rating
6 Months
Full Program
100%
Placement Support
Python 3 Django & DRF FastAPI PostgreSQL Machine Learning Docker & AWS Real Projects AI Tools

math & random — Numbers and Randomisation

The math module provides mathematical functions — trigonometry, logarithms, rounding, constants. The random module generates pseudo-random numbers and implements random selection, shuffling, and sampling.

Python — math_and_random.py
import math

# Constants
print(math.pi)              # 3.141592653589793
print(math.e)               # 2.718281828459045
print(math.tau)             # 6.283185307 (2π)
print(math.inf)             # infinity

# Rounding and absolute
print(math.floor(3.9))     # 3  — round down
print(math.ceil(3.1))      # 4  — round up
print(math.trunc(3.7))     # 3  — truncate toward 0
print(math.fabs(-5.5))     # 5.5 — absolute value (float)

# Power and roots
print(math.sqrt(144))      # 12.0
print(math.pow(2, 10))     # 1024.0
print(math.isqrt(17))      # 4  — integer square root

# Logarithm
print(math.log(100, 10))   # 2.0  — log base 10
print(math.log2(8))         # 3.0  — log base 2
print(math.log10(1000))     # 3.0

# Trig
print(math.sin(math.pi / 2))  # 1.0
print(math.cos(0))            # 1.0
print(math.degrees(math.pi))  # 180.0
print(math.radians(90))       # 1.5707963...

# Combinatorics
print(math.factorial(6))      # 720
print(math.comb(10, 3))       # 120  — C(10,3)
print(math.perm(5, 2))        # 20   — P(5,2)
print(math.gcd(48, 18))       # 6

# ── random ───────────────────────────────────────────
import random

random.seed(42)                 # reproducible sequence

print(random.random())           # float in [0.0, 1.0)
print(random.uniform(1, 10))    # float in [1.0, 10.0]
print(random.randint(1, 6))     # int in [1, 6] — dice roll
print(random.randrange(0,100,5)) # 0,5,10,15...95

items = ["a", "b", "c", "d"]
print(random.choice(items))      # pick one randomly
print(random.sample(items, 2))   # pick 2, no replacement
print(random.choices(items, k=5))# pick 5, with replacement
random.shuffle(items)             # shuffle in-place
print(items)

datetime — Dates and Times

The datetime module provides four main classes: date (year/month/day), time (hour/minute/second), datetime (combined), and timedelta (duration). It's used for timestamps, scheduling, date arithmetic, and formatting.

Python — datetime_module.py
from datetime import date, time, datetime, timedelta

# date — year, month, day
today = date.today()
print(today)                          # 2025-05-16
print(today.year, today.month, today.day)
print(today.strftime("%d %B %Y"))   # 16 May 2025
print(today.isoformat())             # 2025-05-16
print(today.weekday())               # 0=Mon...6=Sun

# datetime — date + time
now = datetime.now()
print(now)                            # 2025-05-16 14:30:00.123456
print(now.strftime("%Y-%m-%d %H:%M:%S")) # formatted string

# Parse string to datetime
dt = datetime.strptime("2025-01-15", "%Y-%m-%d")
print(type(dt))                       # <class 'datetime'>

# timedelta — date arithmetic
delta    = timedelta(days=30, hours=5)
future   = today + delta
past     = today - timedelta(weeks=2)
print(f"30 days from today: {future}")
print(f"2 weeks ago: {past}")

# Difference between two dates
d1 = date(2025, 1, 1)
d2 = date(2025, 12, 31)
print((d2 - d1).days)                 # 364 days

# Timestamp → datetime
import time as t_module
ts = t_module.time()                  # Unix timestamp (float)
dt = datetime.fromtimestamp(ts)
print(dt)

# Format specifiers: %Y year, %m month, %d day
#   %H hour, %M minute, %S second, %A weekday name

json & re — Data Encoding and Pattern Matching

The json module converts Python objects to JSON strings and back — essential for web APIs, config files, and data storage. The re module provides regular expressions for pattern matching, validation, searching, and text processing.

Python — json_and_re.py
# ── json ─────────────────────────────────────────────
import json

# Python dict → JSON string
data     = {"name": "Alice", "age": 25, "active": True}
json_str = json.dumps(data)
print(json_str)  # '{"name": "Alice", "age": 25, "active": true}'

# Pretty-print with indent
print(json.dumps(data, indent=2))

# JSON string → Python dict
parsed = json.loads(json_str)
print(parsed["name"])          # Alice

# Write JSON to file
with open("data.json", "w") as f:
    json.dump(data, f, indent=2)

# Read JSON from file
with open("data.json") as f:
    loaded = json.load(f)
print(loaded)

# ── re — regular expressions ─────────────────────────
import re

# search — find pattern anywhere in string
match = re.search(r'\d+', "Phone: 12345")
print(match.group())         # 12345

# match — pattern at START of string
m = re.match(r'\d{4}', "2025-05-16")
print(m.group() if m else "no match")  # 2025

# findall — list all matches
nums = re.findall(r'\d+', "10 cats and 3 dogs")
print(nums)                  # ['10', '3']

# sub — replace matches
result = re.sub(r'\s+', "-", "hello world foo")
print(result)                # hello-world-foo

# compile — precompile pattern for reuse
email_re = re.compile(r'^[\w\.-]+@[\w\.-]+\.\w{2,}$')
print(bool(email_re.match("alice@example.com")))  # True
print(bool(email_re.match("not-an-email")))        # False

# Groups — extract parts of a match
m = re.search(r'(\d{4})-(\d{2})-(\d{2})', "Date: 2025-05-16")
if m:
    print(m.group(1), m.group(2), m.group(3))  # 2025 05 16

collections, itertools & functools

These three modules are the functional programming and data structure power trio. collections provides specialised containers, itertools provides efficient iterators and combinatorics, and functools provides higher-order function utilities.

Python — collections_itertools_functools.py
# ── collections ──────────────────────────────────────
from collections import Counter, defaultdict, deque, OrderedDict, namedtuple

# Counter — count occurrences
words = ["apple", "banana", "apple", "cherry", "banana", "apple"]
freq  = Counter(words)
print(freq)                   # Counter({'apple': 3, 'banana': 2, 'cherry': 1})
print(freq.most_common(2))    # [('apple', 3), ('banana', 2)]

# defaultdict — dict with default factory
groups = defaultdict(list)
for w in words:
    groups[w[0]].append(w)     # no KeyError for missing keys!
print(dict(groups))

# deque — double-ended queue, O(1) append/pop both ends
dq = deque([1,2,3], maxlen=5)
dq.appendleft(0)             # add to front
dq.append(4)                  # add to back
print(dq)                      # deque([0, 1, 2, 3, 4], maxlen=5)

# namedtuple — tuple with named fields
Point = namedtuple("Point", ["x", "y"])
p     = Point(3, 4)
print(p.x, p.y)                # 3 4  — named access

# ── itertools ────────────────────────────────────────
import itertools

# chain — iterate over multiple iterables as one
print(list(itertools.chain([1,2], [3,4], [5])))   # [1,2,3,4,5]

# combinations and permutations
print(list(itertools.combinations(["A","B","C"], 2)))
# [('A','B'), ('A','C'), ('B','C')]
print(list(itertools.permutations(["A","B","C"], 2)))

# product — Cartesian product
print(list(itertools.product([1,2], ["a","b"])))
# [(1,'a'),(1,'b'),(2,'a'),(2,'b')]

# groupby — group consecutive elements
data = [("a",1),("a",2),("b",3),("b",4)]
for key, group in itertools.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

# ── functools ────────────────────────────────────────
from functools import lru_cache, partial, reduce

# lru_cache — memoisation decorator
@lru_cache(maxsize=None)
def fib(n):
    if n <= 1: return n
    return fib(n-1) + fib(n-2)

print(fib(40))  # 102334155 — instant with cache

# partial — fix some arguments of a function
double = partial(pow, exp=2)
print(double(5))              # 25

# reduce — cumulative binary operation
print(reduce(lambda a,b: a*b, [1,2,3,4,5]))  # 120

Practice Python Online — Free Compilers

Don't want to install anything yet? Practice all standard library examples from this chapter directly in your browser — all stdlib modules are available in these online environments.

▶
Online Python 3 Compiler
Write, run and debug Python code in your browser. Standard library modules available. Test every example instantly.
Open Compiler
Replit
Full IDE online
Google Colab
Free GPU notebooks
JupyterLite
Notebooks in browser
Programiz
Beginner-friendly
Chapter Summary
  • Python ships "batteries included" — 200+ standard library modules available without any installation.
  • os — file system, directories, environment variables, directory walking. pathlib — modern OOP path handling with / operator for joining.
  • sys — Python version, command-line arguments (sys.argv), module search path (sys.path), exit, and object sizes.
  • math — constants (π, e), trig, logarithm, power, factorial, combinations, GCD. random — random numbers, choice, shuffle, sample with seed support.
  • datetime — date, time, datetime, timedelta for date arithmetic, formatting with strftime, parsing with strptime.
  • json — dumps() (dict→string), loads() (string→dict), dump() / load() for files. Essential for API work.
  • re — search(), match(), findall(), sub(), compile() for regular expression operations.
  • collections — Counter (counting), defaultdict (safe dict), deque (fast queue), namedtuple (readable tuple).
  • itertools — chain, combinations, permutations, product, groupby for efficient lazy iteration.
  • functools — @lru_cache (memoisation), partial() (pre-fill arguments), reduce() (fold).
⚡
Free Resource
Download our Python Standard Library Cheat Sheet
os, pathlib, sys, math, random, datetime, json, re, collections, itertools & functools — one printable page.
Get Sheet
Advertisement
Quick Check — Test Your Understanding
Question & Answer
✓ Answer

Both join path components in a cross-platform way, but they differ in style and interface:

  • os.path.join("/home", "user", "file.txt") — legacy, string-based, returns a string.
  • Path("/home") / "user" / "file.txt" — modern OOP approach, returns a Path object with methods like .exists(), .read_text(), .glob() directly on it.

pathlib (Python 3.4+) is the modern preferred approach — cleaner, more readable, and avoids string manipulation.

✓ Answer

Both serialise a Python object to JSON, but differ in destination:

  • json.dumps(data) — serialises to a string (the "s" stands for "string"). Use when you need the JSON as a Python string (e.g. to send over a network).
  • json.dump(data, file) — writes directly to a file object. Use when saving to disk.

Similarly, json.loads(string) parses a JSON string, while json.load(file) reads from a file.

✓ Answer

Counter is a subclass of dict that counts occurrences of elements. Pass it any iterable and it builds a frequency map:

from collections import Counter
c = Counter(["a","b","a","c","a"])
print(c)                  # Counter({'a': 3, 'b': 1, 'c': 1})
print(c.most_common(2))   # [('a', 3), ('b', 1)]

Use it for word frequency counts, letter frequency, vote tallying, or any scenario where you need to count how many times each value appears.

✓ Answer

@lru_cache (Least Recently Used cache) is a memoisation decorator — it caches the results of function calls. When the function is called with the same arguments again, it returns the cached result instead of recomputing.

It's especially powerful for recursive functions with overlapping subproblems like Fibonacci. Without the cache, fib(35) makes millions of calls. With it, each unique value is computed only once:

from functools import lru_cache
@lru_cache(maxsize=None)
def fib(n):
    if n <= 1: return n
    return fib(n-1) + fib(n-2)   # O(n) with cache
✓ Answer

Use Path.rglob() (recursive glob) to match files in all subdirectories:

from pathlib import Path

# All .py files in current dir and all subdirectories
for py_file in Path(".").rglob("*.py"):
    print(py_file)

# Non-recursive (current dir only)
for py_file in Path(".").glob("*.py"):
    print(py_file)
MCQ 1 of 3
Which function from the json module converts a Python dictionary to a JSON string?
MCQ 2 of 3
What does Counter("banana") return?
MCQ 3 of 3
Which re function returns all non-overlapping matches as a list?

Rate Us on Google

4.8 / 5
Enjoying this tutorial?
Your 5-star review helps other learners discover our free tutorials and encourages us to keep creating quality content.
80+ reviews
Write a Google Review
Join WhatsApp Channel