Virtualisation and Containers
Understand the 'works on my machine' problem and learn to package any application in a Docker container that runs identically everywhere.
The environment problem in depth
Software depends on its environment — the operating system, the runtime version, the system libraries, the installed packages, the environment variables, the file paths. When any of these differ between two machines, the same code can behave differently.
This problem compounds across a development team's lifecycle:
- Developer A uses Python 3.11 on macOS. Developer B uses Python 3.9 on Ubuntu. A feature that uses a 3.10+ API works for A and fails for B.
- The CI server runs Ubuntu and Python 3.11 but a different minor version. A subtle difference in a library's behaviour causes intermittent failures.
- The production server was set up two years ago with a different library version. A change that passed all tests breaks in production.
This is not a rare edge case — it is the default state of software development without explicit environment management. Containers solve it.
From bare metal to containers
Environment management has evolved through several generations:
| Approach | Mechanism | Pros | Cons |
|---|---|---|---|
| Bare metal | All applications share the OS directly | Simple, fast | No isolation, environment conflicts |
| Virtual machines | Each application gets its own OS on virtualised hardware | Strong isolation | Large (GBs), slow to start, resource-heavy |
| Containers | Applications share the host OS kernel but have isolated filesystems | Lightweight, fast, portable | Less isolation than VMs, Linux-centric |
A container does not include an operating system. It includes only the application and its dependencies — the libraries, configuration, and runtime it needs. It shares the host's Linux kernel. This is why containers are measured in megabytes, not gigabytes, and start in seconds, not minutes.
Before standardised shipping containers (invented in the 1950s), cargo was loaded and unloaded piece by piece at every port. Different ports had different equipment. Each shipment was a custom operation.
The standardised container transformed logistics: any crane could handle any container; any ship could carry any combination of containers; the contents were irrelevant to the infrastructure. Docker containers do the same for software: any host with Docker can run any container, regardless of what is inside.
Docker architecture
Docker uses a client-server architecture:
docker command you type. Sends instructions to the daemon via a REST API.python:3.11 come from here.Images and layers
A Docker image is not a single large file. It is a stack of layers. Each instruction in the Dockerfile creates a new layer. Layers are cached and reused:
Layer 5: COPY . . ← your application code
Layer 4: RUN pip install ← your dependencies
Layer 3: COPY requirements.txt .
Layer 2: WORKDIR /app
Layer 1: FROM python:3.11-slim ← base OS + Python
When you rebuild the image, Docker checks each layer. If the layer's input has not changed, the cached version is reused. This is why order matters: put things that change rarely (base image, system packages) before things that change often (application code).
A common pattern: copy requirements.txt first, run pip install, then copy the rest of the application. This way, the expensive pip install layer is only rebuilt when requirements.txt changes, not every time you change a line of application code.
Writing a Dockerfile
# Base image: official Python image, slim variant for smaller size
FROM python:3.11-slim
# Set environment variables
# Prevent Python from writing .pyc files
ENV PYTHONDONTWRITEBYTECODE=1
# Prevent Python from buffering stdout/stderr
ENV PYTHONUNBUFFERED=1
# Create and set the working directory inside the container
WORKDIR /app
# Copy ONLY requirements first (better layer caching)
COPY requirements.txt .
# Install dependencies
# --no-cache-dir reduces image size
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application
COPY . .
# Document which port the application uses (informational)
EXPOSE 8000
# The command to run when the container starts
CMD ["python", "-m", "uvicorn", "app:app", "--host", "0.0.0.0"]
Dockerfile instructions reference
COPY over ADD for simple file copies..dockerignore
Just as .gitignore tells Git which files to ignore, .dockerignore tells Docker which files to exclude from the build context. A smaller build context means faster builds:
.git
.gitignore
__pycache__
*.pyc
.pytest_cache
*.egg-info
dist/
build/
.env
*.md
tests/
# We do not need tests inside the production image
Docker commands in depth
# Build image, tag it, from Dockerfile in current dir
docker build -t my-app:latest .
# Build with a different Dockerfile
docker build -f Dockerfile.prod -t my-app:prod .
# Force rebuild — ignore all caches
docker build --no-cache -t my-app:latest .
# List all local images
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my-app latest a1b2c3d4 2 mins ago 142MB
# Run a container (exits when command completes)
docker run my-app:latest
# Run interactively with a terminal
docker run -it my-app:latest bash
# Run detached (in background)
docker run -d -p 8000:8000 my-app:latest
# Run and automatically delete container on exit
docker run --rm my-app:latest
# Pass environment variable
docker run -e APP_ENV=production my-app:latest
# List running containers
docker ps
# List all containers (including stopped)
docker ps -a
# Stop a running container
docker stop Volumes and bind mounts
Containers have isolated filesystems. When a container exits, any files written inside it are lost. Volumes and bind mounts persist data outside the container:
# Mount current directory into the container
# Changes to local files appear immediately inside the container
docker run -v $(pwd):/app my-app:latest
# Named volume for persistent data (e.g. database)
docker run -v db-data:/var/lib/postgresql/data postgres:16
Bind mounts are useful during development — you edit code on your machine and the running container sees the changes immediately. Named volumes are used for persistence (databases, uploaded files).
Networking basics
Containers on the same Docker network can communicate with each other by container name. By default, containers are isolated from the host network:
# -p host_port:container_port
# Forward requests to localhost:8000 → container:8000
docker run -p 8000:8000 my-app:latest
# Map to a different host port
docker run -p 3000:8000 my-app:latest
# localhost:3000 → container:8000
Key terms
Exercises
Part A: Exploring the runtime
- Pull and run the Python image interactively:
docker run -it python:3.11-slim bash - Inside the container, run
python3 --version,pip list, andls /. - Create a file:
echo hello > /tmp/test.txt. Exit the container. - Run the same image again. Is
/tmp/test.txtstill there? Why not?
Part B: Build your first image
- Write a Python script that prints a greeting and today's date.
- Write a Dockerfile for it using python:3.11-slim as the base.
- Add a
.dockerignoreexcluding__pycache__and.git. - Build it:
docker build -t hello:v1 . - Run it:
docker run --rm hello:v1
Part C: Layer caching
- Add a requirements.txt with one package (e.g. requests).
- Update your Dockerfile to copy requirements.txt first, install, then copy the rest.
- Build twice:
docker build -t hello:v2 .. How does the second build differ? - Change only your Python script and rebuild. Which layers are rebuilt? Which are cached?
Part D: Running a web application
- Install Flask locally and write a minimal Flask app that returns 'Hello from Docker'.
- Add Flask to requirements.txt.
- Update the Dockerfile CMD to run Flask:
["python3", "-m", "flask", "run", "--host=0.0.0.0"] - Build and run with port mapping:
docker run -p 5000:5000 hello:v3 - Visit http://localhost:5000 in your browser.