Docker Containers Explained: From Development to Production

In 2013, a developer named Solomon Hykes delivered a five-minute presentation at PyCon. He demonstrated a tool capable of packaging an application together with everything required to run it—libraries, configuration, runtime—into a portable unit that behaved identically on any Linux machine. The audience applauded politely. Docker was open-sourced two months later, and within five years it had become one of the most influential technologies in the history of software development.

The problem Docker addressed had affected practitioners for as long as software has existed: the recurring observation that code which runs correctly on a developer’s laptop fails in staging, that applications which behave one way in staging behave differently in production, and that new engineers spend days configuring local environments that never quite replicate the cloud target. Entire categories of defects existed because the environments in which code executed differed in invisible and difficult-to-reproduce ways.

Docker’s response was the container: an isolated, reproducible runtime environment that packages code and all its dependencies into a single artifact that behaves identically across hosts. A container built on a MacBook Pro will run identically on an Ubuntu server in AWS, on a Windows workstation, or on a Raspberry Pi running ARM Linux. Behavior, dependencies, and configuration remain constant across all targets.

In 2026, Docker and container technology are no longer optional knowledge for professional developers; they are foundational. The remainder of this post proceeds from first principles to production-ready patterns, covering the concepts and commands required to use Docker in real projects rather than to understand it only abstractly. For a companion piece that explores container internals, virtual machines versus containers, and layer caching strategies in greater depth, see the Docker containers explained guide.

Why Docker Changed Software Development Forever

To understand why Docker matters, one must first understand what it replaced. Before containers, deploying software typically involved one of two approaches.

Manual server configuration: An operator would connect to a server via SSH and install dependencies by hand, documenting the steps in a README and trusting that subsequent operators would follow them correctly. Engineers would later discover that production was running Python 3.8 while development was using Python 3.11, then spend days tracing the resulting behavioral differences. The approach was slow, error-prone, and impossible to scale.

Virtual Machines (VMs): Virtual machines address the consistency problem by virtualizing the entire hardware stack. A complete operating system image is packaged and executed inside another operating system. However, virtual machines are heavyweight. A typical image is gigabytes in size and takes minutes to boot. Running fifty isolated services as separate virtual machines requires fifty copies of a full operating system and consumes substantial resources.

Docker containers take a different approach: rather than virtualizing hardware, they virtualize the operating system. Containers share the host kernel but maintain isolated filesystems, processes, and network interfaces. The result is environments that are isolated like virtual machines yet lightweight like processes. A container starts in milliseconds rather than minutes and incurs overhead measured in megabytes rather than gigabytes.

This performance profile enables patterns that were impractical with virtual machines: operating fifty isolated microservices on a single server, instantiating ephemeral test environments for every pull request, and deploying code updates by replacing containers rather than executing update scripts. These patterns are now industry standard, and Docker is the technology that made them practical. For example, event-driven architectures based on Apache Kafka for stream processing or Apache Flink for complex event processing rely heavily on containerized deployments to scale individual pipeline stages independently.

Core Concepts: Images, Containers, and Registries

Docker’s conceptual model rests on three core entities. Conflating them is the most common source of error among newcomers, so each requires precise definition.

Docker Images: The Blueprint

A Docker image is a read-only template containing everything required to run an application: the operating system filesystem, application code, libraries, environment variables, and startup commands. An image is built once and can be instantiated as many containers. An image is analogous to a class definition in object-oriented programming—a blueprint rather than the entity itself.

Images are constructed in layers. Each instruction in a Dockerfile produces a new layer. Layers are cached and reused, so if application code changes but dependencies do not, Docker rebuilds only the modified layers. This layered cache is the reason Docker builds become fast after the initial one.

Docker Containers: The Running Instance

A container is a running instance of an image. When an image is executed, Docker creates a writable layer above the image’s read-only layers and starts the specified process. The container possesses an isolated filesystem, network interface, and process namespace. Multiple containers can run concurrently from the same image, each maintaining its own writable state.

An important property: containers are ephemeral by design. When a container stops, any data written to its filesystem is lost unless stored in a volume, which is discussed later. This ephemerality is a deliberate property rather than a defect. It allows containers to be destroyed and recreated without concern for accumulated state. Persistent data belongs in volumes, and application state belongs in external databases.

Docker Registries: The Distribution Layer

A registry is a storage system for Docker images. Docker Hub is the default public registry, hosting hundreds of thousands of community and official images, including Ubuntu, Node.js, PostgreSQL, Redis, and nginx. Private registries such as AWS ECR, Google Artifact Registry, and GitHub Container Registry store proprietary images within an organization’s own infrastructure.

The workflow is straightforward: an image is built locally, pushed to a registry, and pulled from that registry on any machine that needs to run it. This is how code travels from a developer’s laptop to a production server without manual file copying or SSH-based deployment scripts.

Writing a First Dockerfile

A Dockerfile is a text file containing instructions for building a Docker image, with each instruction producing a layer. The following example builds a Python web application image step by step using FastAPI, which is examined in detail in the companion FastAPI guide.

# Start from an official Python runtime as the base image
FROM python:3.12-slim

# Set the working directory inside the container
WORKDIR /app

# Copy dependency files first (for better layer caching)
COPY requirements.txt .

# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the application code
COPY . .

# Create a non-root user for security
RUN useradd --create-home appuser && chown -R appuser /app
USER appuser

# Tell Docker what port the app uses (documentation only)
EXPOSE 8000

# Command to run when container starts
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Several decisions embedded in this Dockerfile are important for production use.

python:3.12-slim rather than python:3.12: The slim variant omits documentation, test files, and other non-essential components, reducing image size from approximately 900 MB to roughly 130 MB. Smaller images build faster, transfer faster, and present a smaller attack surface. For practitioners considering a compiled language to produce leaner containers, the Python and Rust comparison examines how Rust’s static binaries can yield single-digit-megabyte images.

Copying requirements.txt before the application code: Docker rebuilds only the layers that have changed and any layers that follow them. Copying dependencies before source code allows the expensive pip install step to remain cached as long as requirements.txt is unchanged, even when application code changes. The result is substantially faster iterative builds.

Running as a non-root user: Processes in containers run as root by default. This poses a security risk: an attacker who exploits an application vulnerability obtains root access inside the container. Creating a non-root user and switching to it is a low-effort improvement with meaningful security benefit.

The image can then be built and executed as follows.

# Build the image, tagging it as "myapp:latest"
docker build -t myapp:latest .

# Run the container, mapping host port 8080 to container port 8000
docker run -p 8080:8000 myapp:latest

# Run in detached mode (background)
docker run -d -p 8080:8000 --name myapp myapp:latest

# View running containers
docker ps

# View container logs
docker logs myapp

# Stop the container
docker stop myapp

Docker Compose: Multi-Container Applications

Real applications rarely run in isolation. A typical web application requires a database, a cache, possibly a background worker, and sometimes a reverse proxy. Running and connecting such services manually with docker run commands becomes unmanageable. Docker Compose addresses this by defining and running multi-container applications from a single YAML configuration file.

The following docker-compose.yml defines a FastAPI application paired with PostgreSQL and Redis.

services:
  # The web application
  web:
    build: .
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgresql://postgres:secret@db:5432/appdb
      REDIS_URL: redis://redis:6379/0
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    volumes:
      - ./src:/app/src  # Mount source for hot reload in development

  # PostgreSQL database
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb
    volumes:
      - postgres_data:/var/lib/postgresql/data  # Persist data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

  # Redis cache
  redis:
    image: redis:7-alpine
    volumes:
      - redis_data:/data

# Named volumes persist data between container restarts
volumes:
  postgres_data:
  redis_data:

Several patterns in this configuration warrant attention.

Service discovery by name: The web service connects to the database using db as the hostname, visible in DATABASE_URL: postgresql://...@db:5432/.... Docker Compose creates an internal network on which each service is reachable by its service name, removing the need for hardcoded IP addresses.

Health checks with depends_on: Declaring depends_on: db alone only waits for the database container to start, not for PostgreSQL to be ready to accept connections. Combining condition: service_healthy with a health check ensures the web service does not start until the database is actually responsive.

Volume mounts for development: Mounting ./src:/app/src ensures that source code changes on the host machine are immediately reflected inside the container, enabling hot reload without rebuilding the image for every change.

# Start all services (detached)
docker compose up -d

# View logs from all services
docker compose logs -f

# View logs from a specific service
docker compose logs -f web

# Stop all services
docker compose down

# Stop and remove volumes (WARNING: deletes data)
docker compose down -v

# Rebuild images after Dockerfile changes
docker compose up -d --build

# Run a one-off command in a service container
docker compose exec web python manage.py migrate

Networking: How Containers Communicate

Docker’s networking model rests on a few concepts that frequently cause confusion among developers encountering container networking for the first time.

Each container has its own network namespace. Inside a container, localhost refers to the container itself rather than the host machine. This often surprises developers: a web server inside a container cannot connect to a database running on the host using localhost:5432 because the database is not “local” from the container’s perspective.

Docker Compose creates a default network. All services declared in a docker-compose.yml file are automatically connected to a shared bridge network on which services reach one another by service name. The web service connects to db using the hostname db, not localhost.

Port publishing exposes containers to the host. The ports: - "8000:8000" syntax publishes container port 8000 on host port 8000. Without this directive, the service is reachable only from within the Docker network and not from a browser on the host machine.

Internal services should not publish ports in production. A database container does not need to be reachable from outside Docker in production; only the web application requires external access. Omitting port publishing for internal services such as databases, caches, and workers substantially reduces the attack surface.

Persistent Data with Volumes

Containers are ephemeral: when a container is removed, its writable layer disappears, and any data written directly to the container filesystem is lost. Databases, file uploads, configuration, and any other data that must survive container restarts require volumes.

Docker provides two persistence mechanisms.

Named volumes are managed by Docker and stored in its storage area on the host, typically at /var/lib/docker/volumes/. They are the recommended mechanism for persisting database data because Docker manages their lifecycle independently of any particular container. In the Compose example above, postgres_data and redis_data are named volumes.

Bind mounts map a specific directory on the host machine to a path inside the container. The ./src:/app/src mount in the development configuration is a bind mount. Changes on the host are immediately visible inside the container. Bind mounts are well suited to development because they enable live code reload, but they are less appropriate for production because they introduce a dependency on the host filesystem structure.

# List all volumes
docker volume ls

# Inspect a named volume (shows where data is stored on host)
docker volume inspect myapp_postgres_data

# Back up a named volume
docker run --rm \
  -v myapp_postgres_data:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/postgres_backup.tar.gz /data

# Remove unused volumes (careful — this deletes data!)
docker volume prune

Production Best Practices: Adjustments for Live Environments

A Docker configuration that performs well in development can still fail in production in unexpected ways. The gap between “the application runs in Docker” and “the application runs reliably in production Docker” is bridged by several important practices.

Multi-Stage Builds: Separating Build from Runtime

Many applications require build tools that are unnecessary at runtime, including compilers, test frameworks, and build system dependencies. Multi-stage builds allow a heavy build environment to produce artifacts that are then copied into a minimal runtime image.

# Stage 1: Build stage (can be large)
FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build  # Produces /app/dist

# Stage 2: Production runtime (minimal)
FROM node:20-alpine AS production
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev  # Only production dependencies
COPY --from=builder /app/dist ./dist  # Copy only build output
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

The final image contains only the Node.js runtime, production dependencies, and compiled output, with no TypeScript compiler, development dependencies, or source files. The reduction in image size can move from more than 1 GB to under 200 MB.

Avoiding Secrets in Images

One of the most common security errors, and a violation of clean code principles, is embedding credentials, API keys, or passwords in a Dockerfile or in the image itself. Docker image layers are readable by anyone with access to the image. Even if the secret is added in one layer and removed in another, it remains accessible in the intermediate layer’s history.

# WRONG: Secret baked into image
ENV API_KEY=sk-super-secret-key-12345

# RIGHT: Pass secrets at runtime as environment variables
# In docker run:
docker run -e API_KEY="${API_KEY}" myapp

# In Docker Compose with an .env file:
# .env file (never commit this to git):
# API_KEY=sk-super-secret-key-12345

# docker-compose.yml:
# environment:
#   API_KEY: ${API_KEY}  # Reads from .env file

Container Health Checks in Production

In production environments that employ container orchestration such as Kubernetes, Docker Swarm, or AWS ECS, the orchestrator requires a mechanism to determine container health. Without a health check, the orchestrator assumes that the container is healthy as long as the process is running, even when the application returns HTTP 500 errors for every request.

HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8000/health || exit 1

The application should expose a /health endpoint that returns HTTP 200 when it is ready to serve requests and can reach its dependencies. The orchestrator will restart unhealthy containers and direct traffic away from them.

Resource Limits

Without resource limits, a misbehaving container can consume all available memory or CPU on a host, starving other services. Memory and CPU limits should always be configured in production.

services:
  web:
    image: myapp:latest
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: "1.0"
        reservations:
          memory: 256M
          cpus: "0.5"

Common Patterns: Web App, API with Database, Worker Queue

Pattern 1: Web App with Nginx Reverse Proxy

It is standard practice in production to run a reverse proxy such as nginx or Caddy in front of the application. The proxy handles SSL termination, static file serving, request buffering, and load balancing, allowing the application server to focus on business logic.

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
      - ./certs:/etc/nginx/certs
    depends_on:
      - web

  web:
    build: .
    # Note: NO ports published — only nginx reaches this container
    expose:
      - "8000"

Pattern 2: Background Worker with Celery and Redis

Long-running tasks such as sending emails, processing images, or generating reports should not block HTTP request handlers. The standard pattern queues these tasks and processes them asynchronously through a worker process.

services:
  web:
    build: .
    command: uvicorn main:app --host 0.0.0.0 --port 8000

  worker:
    build: .  # Same image, different command
    command: celery -A tasks worker --loglevel=info
    depends_on:
      - redis
      - db

  redis:
    image: redis:7-alpine

  db:
    image: postgres:16-alpine

The web and worker services share the same Docker image but execute different commands. This is a common pattern for Python applications: one image, multiple process types, all defined in a single Compose file.

Debugging Containers: Diagnosing Failures

Every Docker practitioner accumulates a set of debugging commands. The following are the most frequently used.

# Open an interactive shell inside a running container
docker exec -it container_name bash
# or if bash isn't available (Alpine-based images):
docker exec -it container_name sh

# Inspect container details (env vars, mounts, network settings)
docker inspect container_name

# View real-time resource usage (CPU, memory, network I/O)
docker stats

# Check what files are different from the base image
docker diff container_name

# Start a stopped container to investigate its state
docker start -ai container_name

# Run a debugging container with access to all host namespaces
docker run -it --rm --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh

# Build with verbose output (shows each layer build step)
docker build --progress=plain .

# Check why a layer is cache-busting (useful for slow builds)
docker history myapp:latest

The most common debugging scenario is a container that exits immediately after starting. The remedy is to run it interactively in order to surface the error.

# Override the CMD to drop into a shell instead of running the app
docker run -it --rm myapp:latest bash

# Or check the logs of an exited container
docker logs container_name

From Development to Production: A Mental Model

Docker’s value lies not in any single feature but in the consistency it establishes across the entire software delivery lifecycle. The same image that runs on a developer’s laptop is the image that is tested in continuous integration and deployed to production. The environment, comprising the operating system, libraries, and configuration structure, is defined once in a Dockerfile and reproduced exactly across all targets.

The conceptual shift that Docker enables is the treatment of infrastructure as code. The Dockerfile is a precise, version-controlled specification of the application’s runtime environment. The docker-compose.yml is a precise, version-controlled specification of how services connect. Both reside in the repository, are reviewed in pull requests in accordance with Git and GitHub best practices, and are reproduced identically by any developer on the team within minutes through docker compose up.

This consistency eliminates entire categories of defects, simplifies onboarding considerably, and renders the deployment pipeline reliable in ways that manual server configuration could not achieve. These factors explain why Docker adoption progressed from zero to ubiquitous in under a decade. The tool addressed real problems that developers encountered daily, and the developer experience was favorable.

The path from this point to production-ready containers is straightforward: learn the Dockerfile instructions, understand Compose networking, master the debugging commands, and apply the production best practices outlined above. For a more detailed examination of container internals, virtual machine comparisons, and image optimization strategies, consult the companion Docker containers explained from development to production guide. The concepts are few and the practical return is substantial. Starting with a single application and containerizing it is the most direct way to understand why Solomon Hykes’ five-minute PyCon demonstration influenced an industry.

References

• Docker Official Documentation—docs.docker.com
• Docker Dockerfile Best Practices
• Docker Compose File Reference
• OWASP Docker Security Cheat Sheet
• Kane, Sean P. and Karl Matthias. Docker: Up and Running, 3rd Edition. O’Reilly Media, 2023.
• Docker Awesome Compose—Official Sample Applications
• Aqua Security. “Docker Security Best Practices.” Aqua Security Blog, 2024.