Connecting Python to Real Databases
Module M4 simulated SQL query building with plain strings. This module connects that knowledge to a real, running database — executing parameterised queries safely, managing transactions correctly, and using an ORM (SQLAlchemy) to work with rows as Python objects instead of raw SQL strings. By the end, you’ll know both the low-level DB-API approach and the higher-level ORM approach, and when each one is the right tool.
Python doesn’t talk to databases directly — it uses a database driver (a library specific to your database engine) that implements a common interface called the DB-API. Whether you’re using psycopg2 for PostgreSQL, cx_Oracle/oracledb for Oracle, or mysql-connector-python for MySQL, the core pattern is nearly identical across all of them — that consistency is the entire point of the DB-API standard.
Connections, Cursors, and the Core DB-API Pattern
Every DB-API driver follows the same shape: open a connection to the database, create a cursor from that connection (the cursor is what actually executes SQL and fetches results), run your query, fetch the results, and close both when done. This is conceptually the same lifecycle as the file-handling pattern from M5 — open, use, close — just one level more involved.
| Concept | What it represents |
|---|---|
| Connection | The live link to the database server — like having the phone line open |
| Cursor | An object that executes SQL statements and tracks the current position in the result set |
| .execute() | Runs a single SQL statement, optionally with parameters |
| .fetchone() / .fetchall() | Retrieves one row, or all remaining rows, from the last executed query |
| .commit() | Permanently saves any changes (INSERT/UPDATE/DELETE) made in the current transaction |
Parameterised Queries — Never Build SQL with String Formatting
This is the single most important habit in this entire module. Building a SQL query by inserting values directly into a string — with f-strings, .format(), or % — creates a SQL injection vulnerability the moment any part of that string comes from outside your code: user input, an API payload, even another database. Parameterised queries close this gap completely by sending the SQL structure and the data separately.
query = f”SELECT * FROM users
WHERE name = ‘{username}'”
cursor.execute(query)
query = “SELECT * FROM users
WHERE name = %s”
cursor.execute(query, (username,))
In the vulnerable version, if username is the string “x’; DROP TABLE users; –“, that text gets woven directly into the SQL and executed as part of the command. In the safe version, the placeholder (%s, ?, or :name depending on the driver) and the value are sent to the database separately — the database treats the value purely as data, never as executable SQL, no matter what it contains.
Transactions — All or Nothing
A transaction groups one or more database changes so they all succeed together or all fail together — there’s no in-between state where half the changes happened. This matters enormously whenever multiple related changes must stay consistent: transferring money between two accounts means debiting one and crediting the other — if the second part fails, the first part must be undone too, or you’ve created or destroyed money.
(transaction starts)
statements
✓ all saved
(transaction starts)
mid-transaction
✗ nothing saved
This connects directly to M6’s error handling: the standard pattern is a try/except block where the try runs your statements and calls .commit() only if everything succeeded, and the except calls .rollback() to undo any partial changes if something failed partway through.
SQLAlchemy — Working with Rows as Python Objects
An ORM (Object-Relational Mapper) lets you work with database rows as ordinary Python objects — directly connecting to the classes you learned in M7. Instead of writing SQL and manually unpacking tuples, you define a class once, and the ORM translates between that class and the corresponding database table automatically. SQLAlchemy is the most widely used ORM in the Python ecosystem.
__tablename__ = “employees”
id = Column(Integer,
primary_key=True)
name = Column(String)
salary = Column(Integer)
id INTEGER PRIMARY KEY,
name VARCHAR,
salary INTEGER
);
Once defined, you query using Python method chains instead of writing SQL strings: session.query(Employee).filter(Employee.salary > 80000).all() — SQLAlchemy translates this into the correct SQL automatically, parameterised safely by default. This is especially valuable for complex applications where queries need to be built dynamically and composed from reusable pieces.
Raw DB-API vs ORM — When to Use Each
Neither approach is universally “better” — they suit different situations, and most database developers use both depending on the task.
| Situation | Better fit |
|---|---|
| Quick scripts, reports, one-off queries | Raw DB-API — less setup, more direct |
| Large applications with many related tables | ORM — manages relationships and object mapping |
| Complex, highly-tuned analytical SQL | Raw DB-API — full control over exact query shape |
| CRUD-heavy application code | ORM — far less boilerplate per operation |
| Need database-agnostic code (swap PostgreSQL → MySQL easily) | ORM — abstracts away driver-specific SQL dialects |
DB-API — connecting, querying, and fetching
import sqlite3
# Connect and get a cursor
conn = sqlite3.connect("company.db")
cursor = conn.cursor()
# Parameterised SELECT — the ? placeholder, sqlite3's style
cursor.execute("SELECT name, salary FROM employees WHERE department = ?", ("IT",))
# Fetching results
row = cursor.fetchone() # single row as a tuple, or None if no rows
rows = cursor.fetchall() # all remaining rows as a list of tuples
for name, salary in rows:
print(f"{name}: ₹{salary:,}")
# Parameterised INSERT — multiple placeholders
cursor.execute(
"INSERT INTO employees (name, department, salary) VALUES (?, ?, ?)",
("Kiran Desai", "HR", 55000)
)
conn.commit() # changes are NOT saved until commit()
# Always close when finished
cursor.close()
conn.close()
# Using 'with' for automatic cleanup (sqlite3 supports this for the connection)
with sqlite3.connect("company.db") as conn:
cursor = conn.cursor()
cursor.execute("SELECT COUNT(*) FROM employees")
count = cursor.fetchone()[0]
print(f"Total employees: {count}")
Transactions — commit and rollback
import sqlite3
def transfer_funds(conn, from_id, to_id, amount):
"""
Transfer funds between two accounts as a single atomic transaction.
Either both updates succeed, or neither does.
"""
cursor = conn.cursor()
try:
cursor.execute("UPDATE accounts SET balance = balance - ? WHERE id = ?", (amount, from_id))
cursor.execute("UPDATE accounts SET balance = balance + ? WHERE id = ?", (amount, to_id))
conn.commit() # both updates succeeded — save permanently
return True
except sqlite3.Error as e:
conn.rollback() # undo BOTH updates — neither one stays
print(f"Transfer failed, rolled back: {e}")
return False
conn = sqlite3.connect("bank.db")
success = transfer_funds(conn, from_id=101, to_id=102, amount=5000)
SQLAlchemy ORM — defining models and querying
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import declarative_base, sessionmaker
Base = declarative_base()
# Define the model — maps directly to a database table
class Employee(Base):
__tablename__ = "employees"
id = Column(Integer, primary_key=True)
name = Column(String)
department = Column(String)
salary = Column(Integer)
# Connect and create a session (the ORM's equivalent of a cursor)
engine = create_engine("sqlite:///company.db")
Base.metadata.create_all(engine) # creates the table if it doesn't exist
Session = sessionmaker(bind=engine)
session = Session()
# INSERT — create and add a Python object, no SQL written
new_emp = Employee(name="Priya Sharma", department="IT", salary=85000)
session.add(new_emp)
session.commit()
# SELECT — query using Python method chains
all_employees = session.query(Employee).all()
it_staff = session.query(Employee).filter(Employee.department == "IT").all()
top_earner = session.query(Employee).order_by(Employee.salary.desc()).first()
for emp in it_staff:
print(f"{emp.name}: ₹{emp.salary:,}") # attribute access, not SQL
# UPDATE — modify the Python object, then commit
emp_to_raise = session.query(Employee).filter(Employee.name == "Priya Sharma").first()
emp_to_raise.salary = 92000
session.commit()
# DELETE
session.delete(emp_to_raise)
session.commit()
session.close()
All examples use Python’s built-in sqlite3 module so you can run them immediately with no separate database server to install — the exact same patterns apply directly to PostgreSQL, MySQL, or Oracle by swapping the driver and placeholder style.
Builds a simple employee search function that accepts user-style input and queries safely, then deliberately demonstrates what would happen with unsafe string-formatted SQL against the exact same malicious input — making the danger concrete rather than abstract.
import sqlite3
conn = sqlite3.connect(":memory:") # temporary in-memory database for this demo
cursor = conn.cursor()
cursor.execute("CREATE TABLE employees (id INTEGER, name TEXT, department TEXT)")
cursor.executemany(
"INSERT INTO employees VALUES (?, ?, ?)",
[(1, "Priya Sharma", "IT"), (2, "Rahul Mehta", "IT"), (3, "Anita Patel", "Finance")]
)
conn.commit()
def search_safe(name_query):
"""Search by name using a parameterised query — always safe."""
cursor.execute("SELECT * FROM employees WHERE name = ?", (name_query,))
return cursor.fetchall()
def search_unsafe(name_query):
"""DELIBERATELY VULNERABLE — for demonstration only, never use this pattern."""
query = f"SELECT * FROM employees WHERE name = '{name_query}'"
print(f" [unsafe query sent to DB]: {query}")
cursor.execute(query)
return cursor.fetchall()
# Normal search — both work identically
print("Normal search for 'Priya Sharma':")
print(" ", search_safe("Priya Sharma"))
# Malicious-style input — classic SQL injection payload
malicious_input = "x' OR '1'='1"
print(f"\nSearching with malicious input: {malicious_input!r}")
print("Safe version result:", search_safe(malicious_input)) # correctly finds nothing
print("Unsafe version result:", search_unsafe(malicious_input)) # leaks ALL rows!
[(1, ‘Priya Sharma’, ‘IT’)]
Searching with malicious input: “x’ OR ‘1’=’1”
Safe version result: []
[unsafe query sent to DB]: SELECT * FROM employees WHERE name = ‘x’ OR ‘1’=’1′
Unsafe version result: [(1, ‘Priya Sharma’, ‘IT’), (2, ‘Rahul Mehta’, ‘IT’), (3, ‘Anita Patel’, ‘Finance’)]
Reads employee records from a CSV (connecting back to M5) and loads them into a database table efficiently using executemany(), which sends all rows in a single batched operation rather than looping with individual execute() calls.
import sqlite3
import csv
import io
# Simulated CSV content (in practice this would be open("employees.csv"))
csv_content = """name,department,salary
Priya Sharma,IT,85000
Rahul Mehta,IT,92000
Anita Patel,Finance,78000
Suresh Kumar,IT,67000"""
def load_csv_to_db(csv_file, conn):
"""Read employee rows from a CSV file object and bulk-insert them."""
reader = csv.DictReader(csv_file)
rows = [(row["name"], row["department"], int(row["salary"])) for row in reader]
cursor = conn.cursor()
cursor.executemany(
"INSERT INTO employees (name, department, salary) VALUES (?, ?, ?)",
rows
)
conn.commit()
return len(rows)
# ── Run ──
conn = sqlite3.connect(":memory:")
conn.execute("CREATE TABLE employees (name TEXT, department TEXT, salary INTEGER)")
csv_file = io.StringIO(csv_content) # simulates an open file object
inserted = load_csv_to_db(csv_file, conn)
print(f"Inserted {inserted} rows")
cursor = conn.cursor()
cursor.execute("SELECT department, COUNT(*), AVG(salary) FROM employees GROUP BY department")
for dept, count, avg_sal in cursor.fetchall():
print(f" {dept:10} {count} employees, avg ₹{avg_sal:,.0f}")
Finance 1 employees, avg ₹78,000
IT 3 employees, avg ₹81,333
A complete transaction example showing both a successful transfer and a deliberately failing one (insufficient funds, checked before any write happens) — confirming the rollback leaves the database exactly as it was before the attempt.
import sqlite3
class InsufficientFundsError(Exception):
"""Raised when an account doesn't have enough balance for a transfer."""
pass
def get_balance(conn, account_id):
cursor = conn.cursor()
cursor.execute("SELECT balance FROM accounts WHERE id = ?", (account_id,))
return cursor.fetchone()[0]
def transfer(conn, from_id, to_id, amount):
"""Transfer funds atomically. Raises InsufficientFundsError without writing anything."""
cursor = conn.cursor()
try:
current_balance = get_balance(conn, from_id)
if current_balance < amount:
raise InsufficientFundsError(
f"Account {from_id} has ₹{current_balance}, cannot send ₹{amount}"
)
cursor.execute("UPDATE accounts SET balance = balance - ? WHERE id = ?", (amount, from_id))
cursor.execute("UPDATE accounts SET balance = balance + ? WHERE id = ?", (amount, to_id))
conn.commit()
print(f"✓ Transferred ₹{amount} from {from_id} to {to_id}")
except InsufficientFundsError as e:
conn.rollback()
print(f"✗ Transfer blocked: {e}")
# ── Run ──
conn = sqlite3.connect(":memory:")
conn.execute("CREATE TABLE accounts (id INTEGER, balance INTEGER)")
conn.execute("INSERT INTO accounts VALUES (101, 10000), (102, 2000)")
conn.commit()
transfer(conn, 101, 102, 3000) # succeeds — 101 has enough
transfer(conn, 102, 101, 50000) # fails — 102 doesn't have enough, nothing changes
print(f"\nFinal balances: 101=₹{get_balance(conn, 101)} 102=₹{get_balance(conn, 102)}")
✗ Transfer blocked: Account 102 has ₹5000, cannot send ₹50000
Final balances: 101=₹7000 102=₹5000
A full Create-Read-Update-Delete cycle using SQLAlchemy’s ORM, treating each employee as a normal Python object — directly extending the class-based modelling from M7 into a persisted database table.
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import declarative_base, sessionmaker
Base = declarative_base()
class Employee(Base):
__tablename__ = "employees"
id = Column(Integer, primary_key=True)
name = Column(String)
department = Column(String)
salary = Column(Integer)
def __repr__(self):
return f"{self.name} ({self.department}): ₹{self.salary:,}"
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
# CREATE
session.add_all([
Employee(name="Priya Sharma", department="IT", salary=85000),
Employee(name="Rahul Mehta", department="IT", salary=92000),
Employee(name="Anita Patel", department="Finance", salary=78000),
])
session.commit()
# READ
print("All employees:")
for emp in session.query(Employee).all():
print(f" {emp}")
print("\nIT department only:")
for emp in session.query(Employee).filter(Employee.department == "IT").all():
print(f" {emp}")
# UPDATE
priya = session.query(Employee).filter(Employee.name == "Priya Sharma").first()
priya.salary = 90000
session.commit()
print(f"\nAfter raise: {priya}")
# DELETE
suresh = session.query(Employee).filter(Employee.name == "Anita Patel").first()
session.delete(suresh)
session.commit()
print(f"\nRemaining count: {session.query(Employee).count()}")
Priya Sharma (IT): ₹85,000
Rahul Mehta (IT): ₹92,000
Anita Patel (Finance): ₹78,000
IT department only:
Priya Sharma (IT): ₹85,000
Rahul Mehta (IT): ₹92,000
After raise: Priya Sharma (IT): ₹90,000
Remaining count: 2
Combines everything: a context-managed connection, parameterised queries, and the error handling discipline from M6, producing a department salary report that degrades gracefully if the database is unavailable rather than crashing.
import sqlite3
class ReportError(Exception):
"""Raised when the salary report cannot be generated."""
pass
def generate_department_report(db_path, min_avg_salary=0):
"""
Connects to the database, runs a grouped salary report, and returns
departments whose average salary is at least min_avg_salary.
Raises:
ReportError: wraps any underlying database failure with a clear message.
"""
try:
with sqlite3.connect(db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"""SELECT department, COUNT(*) as headcount, AVG(salary) as avg_salary
FROM employees
GROUP BY department
HAVING AVG(salary) >= ?
ORDER BY avg_salary DESC""",
(min_avg_salary,)
)
return cursor.fetchall()
except sqlite3.OperationalError as e:
raise ReportError(f"Could not generate report — database issue: {e}") from e
# ── Set up demo data, then run the report ──
conn = sqlite3.connect("report_demo.db")
conn.execute("DROP TABLE IF EXISTS employees")
conn.execute("CREATE TABLE employees (name TEXT, department TEXT, salary INTEGER)")
conn.executemany(
"INSERT INTO employees VALUES (?, ?, ?)",
[("Priya", "IT", 85000), ("Rahul", "IT", 92000),
("Anita", "Finance", 78000), ("Meera", "Finance", 95000),
("Kiran", "HR", 55000)]
)
conn.commit()
conn.close()
try:
results = generate_department_report("report_demo.db", min_avg_salary=70000)
print("Departments with average salary ≥ ₹70,000:\n")
for dept, headcount, avg_sal in results:
print(f" {dept:10} {headcount} employees, avg ₹{avg_sal:,.0f}")
except ReportError as e:
print(f"Report generation failed: {e}")
Finance 2 employees, avg ₹86,500
IT 2 employees, avg ₹88,500
All exercises use sqlite3, which requires no installation or server setup — perfect for practice. For Exercise 3 and 4 you’ll need SQLAlchemy installed: pip install sqlalchemy.
📋 Order Processing System
Build a script called order_system.py that manages products and processes orders against a SQLite database, using parameterised queries, transactions, and proper error handling throughout. No string-formatted SQL is permitted anywhere in this assignment.
- Schema setup — create two tables: products (id, name, price, stock_qty) and orders (id, product_id, quantity, total_price, status). Seed products with at least 6 rows.
- find_product(name) — parameterised lookup returning the product row, or None if not found.
- Custom exception OrderError(Exception) — raised for any business-rule failure (product not found, insufficient stock, invalid quantity).
- place_order(product_name, quantity) — as a single atomic transaction: looks up the product, validates quantity is positive and stock is sufficient (raising OrderError otherwise, with a rollback), decrements stock_qty, inserts a row into orders with status “confirmed” and the calculated total_price, then commits. Any failure must leave both tables completely unchanged.
- order_summary() — returns a report (using GROUP BY) showing total orders and total revenue per product, for confirmed orders only.
- Demo script — place at least 5 orders, including at least 2 that should deliberately fail (one for insufficient stock, one for an unknown product), confirm via try/except that failures are caught and reported clearly, then print the final order summary and remaining stock levels.
The SQL injection question is the single most important thing to get right in this entire module — take your time with it.
