Getting everything ready for you…

Loading courses, problems and your progress. This takes just a moment.

0s

Guides › Cloud Computing & DevOps

Docker & Kubernetes: Complete Guide

Master Docker and Kubernetes from fundamentals to production concepts. Learn containers, images, Dockerfiles, registries, volumes, networking, Docker Compose, Kubernetes architecture, Pods, Deployments, Services, ConfigMaps, Secrets, storage, autoscaling, security, troubleshooting, and hands-on depl

Docker & Kubernetes: Complete Guide

1. Introduction

Docker and Kubernetes solve related but different problems.

Docker is a platform and ecosystem used to build, package, distribute, and run applications in containers.

Kubernetes is a container orchestration platform that manages containerized workloads across a cluster of machines.

A typical workflow is:

  1. Write application code.
  2. Package the application into a container image.
  3. Test the image locally.
  4. Store the image in a container registry.
  5. Deploy the image to a server or Kubernetes cluster.
  6. Monitor, update, and troubleshoot the application.

Containers improve consistency, but they do not automatically guarantee security, performance, or portability. Host operating systems, CPU architectures, external services, permissions, and configuration still matter.

2. Containers vs Virtual Machines

A virtual machine includes a guest operating system running on virtualized hardware. A container normally shares the host kernel while isolating application processes and resources.

Virtual Machines

Advantages:

Trade-offs:

Containers

Advantages:

Trade-offs:

3. Install and Verify Docker

Install Docker Engine or Docker Desktop using the official instructions for your operating system. On Linux, installation and service-management steps differ by distribution.

Verify installation:

docker --version
docker info
docker run --rm hello-world

The hello-world image confirms that Docker can run a basic container. The first run may download the image.

If commands fail, check that the Docker service is running and that your account has permission to access the Docker daemon. Membership in the Docker group can effectively grant root-level control on many Linux systems, so do not grant it casually.

4. Core Docker Concepts

Image

An image is a packaged, versioned filesystem and configuration used to create containers. Images are built from layers.

Container

A container is a running or stopped instance of an image with its own process configuration and isolated runtime environment.

Registry

A registry stores and distributes images. Examples include Docker Hub and private registries provided by cloud platforms.

Volume

A volume provides persistent storage outside the writable layer of an individual container.

Network

A Docker network controls how containers communicate with one another and with other systems.

Dockerfile

A Dockerfile describes instructions for building an image.

5. Essential Docker Commands

Images

docker pull nginx:stable
docker images
docker image inspect nginx:stable
docker image rm nginx:stable

Containers

docker run -d --name web -p 8080:80 nginx:stable
docker ps
docker ps -a
docker logs web
docker logs -f web
docker exec -it web sh
docker stop web
docker start web
docker restart web
docker rm -f web

Open http://localhost:8080 in a browser when the container is running and the port is available.

-d runs the container in the background. -p 8080:80 maps host port 8080 to container port 80. Use a different host port if 8080 is already in use.

Resource and Cleanup Commands

docker stats
docker inspect web
docker system df
docker container prune
docker image prune

Cleanup commands can delete stopped containers or unused resources. Review what will be removed before running destructive commands.

6. Writing Your First Dockerfile

Create a folder containing index.html:

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Docker Demo</title>
</head>
<body>
  <h1>Hello from Docker!</h1>
  <p>This page is served from a container.</p>
</body>
</html>

Create a file named Dockerfile in the same folder:

FROM nginx:stable
COPY index.html /usr/share/nginx/html/index.html
EXPOSE 80

Build and run it:

docker build -t codify-web:1.0 .
docker run --rm -d --name codify-demo -p 8080:80 codify-web:1.0

Visit http://localhost:8080. Stop the demo with:

docker stop codify-demo

EXPOSE documents the container's intended listening port; it does not publish that port to the host. Publishing is done with -p or the appropriate runtime configuration.

7. Dockerfile Instructions

Prefer a small, maintained base image, pin important versions, avoid embedding secrets, and use .dockerignore to keep unnecessary files out of the build context.

Example .dockerignore:

.git
.env
node_modules
__pycache__
*.log

Adapt the exclusions to your project. Do not exclude files required to build the application.

8. Build a Python Application Image

Create app.py:

from http.server import BaseHTTPRequestHandler, HTTPServer
import os

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        body = b"Hello from the Codify container!\n"
        self.send_response(200)
        self.send_header("Content-Type", "text/plain; charset=utf-8")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

port = int(os.getenv("PORT", "8000"))
server = HTTPServer(("0.0.0.0", port), Handler)
server.serve_forever()

Create Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY app.py .
ENV PORT=8000
EXPOSE 8000
USER 10001
CMD ["python", "app.py"]

Build and run:

docker build -t codify-python:1.0 .
docker run --rm -p 8000:8000 codify-python:1.0

Test in another terminal:

curl http://localhost:8000

Expected response:

Hello from the Codify container!

This example is for learning. For production, use an appropriate production server, graceful shutdown handling, health checks, and a more complete application configuration.

9. Docker Image Layers and Caching

Docker builds images in layers. If an instruction and its inputs have not changed, the build may reuse the cached result.

Good practices:

A multi-stage build separates compilation from the final runtime image, reducing unnecessary files and dependencies.

10. Docker Volumes and Persistent Data

Containers can be removed and recreated. Data that must survive container replacement should live in persistent storage.

Create and inspect a volume:

docker volume create codify-data
docker volume ls
docker volume inspect codify-data

Run a test container using the volume:

docker run --rm \
  -v codify-data:/data \
  alpine \
  sh -c "echo persistent-data > /data/example.txt"

Read the data again:

docker run --rm \
  -v codify-data:/data \
  alpine \
  cat /data/example.txt

The data remains in the named volume after each temporary container exits. Deleting a volume is destructive, so back up important data before removing it.

11. Docker Networking

Common network types include:

Create a user-defined bridge network:

docker network create codify-net
docker network ls
docker network inspect codify-net

Containers attached to the same user-defined network can generally reach each other by container name or network alias.

Do not publish database ports to every host interface unless the use case requires it. Bind to localhost for local-only testing when appropriate.

12. Docker Compose

Docker Compose describes a multi-container application in a YAML file.

Example compose.yaml:

services:
  web:
    image: nginx:stable
    ports:
      - "8080:80"
    restart: unless-stopped

  test:
    image: alpine:3
    command: ["sh", "-c", "echo Compose is working"]

Start the web service:

docker compose up -d web
docker compose ps
docker compose logs web

Run the test service:

docker compose run --rm test

Stop and remove the Compose-managed resources:

docker compose down

Use Compose for local development, demonstrations, and suitable single-host deployments. It is not a replacement for Kubernetes in every environment.

13. Environment Variables and Secrets

Use environment variables for configuration that differs between environments, such as a listening port or feature flag.

Avoid placing passwords, private keys, or API tokens in:

For local development, an ignored .env file can be convenient, but it is not a complete production secrets-management solution. Use a suitable secret manager or orchestrator-supported secret mechanism for sensitive production credentials.

14. Image Registries and Versioning

Use meaningful, traceable tags:

codify-api:1.0.0
codify-api:1.1.0
codify-api:2.0.0

A typical push workflow is:

docker login
docker tag codify-api:1.0.0 YOUR_REGISTRY/codify-api:1.0.0
docker push YOUR_REGISTRY/codify-api:1.0.0

Replace the placeholder with the actual registry and repository. Avoid publishing private images to public repositories.

For stronger reproducibility, pin image digests where suitable and maintain a policy for image updates and vulnerability scanning.

15. Container Security Best Practices

A container is not automatically a security boundary strong enough for every threat model.

16. What Is Kubernetes?

Kubernetes automates the deployment, scaling, networking, and lifecycle management of containerized applications.

Important capabilities include:

Kubernetes introduces operational complexity. For a small application, a managed app platform or simpler container service may be more appropriate.

17. Kubernetes Architecture

A Kubernetes cluster has a control plane and worker nodes.

Control Plane Components

Managed Kubernetes providers may operate some or all control-plane components on the customer's behalf.

Worker Node Components

The precise architecture depends on the cluster distribution and provider.

18. Kubernetes Core Objects

Pod

A Pod is the smallest deployable unit in Kubernetes. It can contain one or more closely related containers that share certain resources, such as networking.

Pods are generally disposable. Create them through workload controllers rather than relying on manually created Pods for long-running applications.

Deployment

A Deployment manages a desired number of replicated Pods and supports controlled updates.

ReplicaSet

A ReplicaSet maintains a specified number of Pod replicas. Deployments normally manage ReplicaSets for you.

Service

A Service provides a stable network endpoint for a set of Pods.

Common types include:

ConfigMap

A ConfigMap stores non-confidential configuration data.

Secret

A Secret is intended for confidential configuration. Access controls, encryption configuration, and cluster security remain important; base64 encoding alone is not encryption.

Namespace

Namespaces group namespaced resources and help separate workloads within a cluster. A namespace by itself is not a complete security boundary.

PersistentVolume and PersistentVolumeClaim

These objects represent storage resources and requests for storage. Actual provisioning depends on the cluster and storage driver.

19. Install kubectl and Access a Cluster

kubectl is the command-line client for Kubernetes. Use the official installation instructions and a local learning environment or managed cluster.

Verify the client:

kubectl version --client
kubectl config get-contexts
kubectl config current-context

Check connectivity to the selected cluster:

kubectl cluster-info
kubectl get nodes
kubectl get namespaces

Always check the current context before running commands against a shared or production cluster.

20. Deploy an Application to Kubernetes

Create deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: codify-web
spec:
  replicas: 2
  selector:
    matchLabels:
      app: codify-web
  template:
    metadata:
      labels:
        app: codify-web
    spec:
      containers:
        - name: nginx
          image: nginx:stable
          ports:
            - containerPort: 80
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 3
            periodSeconds: 5
          resources:
            requests:
              cpu: "100m"
              memory: "64Mi"
            limits:
              cpu: "500m"
              memory: "128Mi"

Apply it:

kubectl apply -f deployment.yaml
kubectl get deployments
kubectl get pods -l app=codify-web
kubectl rollout status deployment/codify-web

The image tag is illustrative. In production, select a maintained version and follow your image pinning policy.

21. Expose the Application with a Service

Create service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: codify-web
spec:
  type: ClusterIP
  selector:
    app: codify-web
  ports:
    - port: 80
      targetPort: 80

Apply and inspect it:

kubectl apply -f service.yaml
kubectl get service codify-web

For local testing, forward a local port:

kubectl port-forward service/codify-web 8080:80

Open http://localhost:8080 while port forwarding is running. For public access, choose and secure an appropriate Ingress, Gateway, or LoadBalancer setup supported by the cluster.

22. Kubernetes Configuration and Secrets

Example ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: codify-config
data:
  APP_MODE: "development"

Create a Secret using an appropriate secure workflow rather than committing real credentials into source control.

For example, in a disposable local cluster, a demonstration Secret can be created with:

kubectl create secret generic codify-demo \
  --from-literal=DEMO_VALUE=replace-me

Do not use actual passwords or tokens in tutorials, screenshots, or public repositories. Restrict who can read Secrets and configure encryption at rest where supported.

23. Kubernetes Health Probes

Use endpoints that reflect the actual application state. An overly aggressive liveness probe can repeatedly restart an otherwise recoverable service.

24. Kubernetes Scaling and Updates

Scale a Deployment:

kubectl scale deployment/codify-web --replicas=3
kubectl get pods

Inspect rollout history:

kubectl rollout history deployment/codify-web
kubectl rollout status deployment/codify-web

Roll back when appropriate:

kubectl rollout undo deployment/codify-web

A rollback of the workload specification does not automatically reverse database schema changes or other external side effects. Plan those separately.

For automatic scaling, learn Horizontal Pod Autoscaler, metrics availability, resource requests, and workload behavior under load.

25. Resource Requests and Limits

Requests inform scheduling and resource allocation. Limits constrain resource consumption according to Kubernetes' resource model.

Set requests based on observed application behavior. Memory limits that are too low can cause out-of-memory termination, while CPU limits can throttle workloads.

Monitor real usage before selecting production values.

26. Kubernetes Storage

Learn:

Do not assume that a persistent volume is automatically backed up. Confirm the storage provider's capabilities and test restoration.

27. Troubleshooting Kubernetes

Useful commands:

kubectl get pods -A
kubectl describe pod POD_NAME
kubectl logs POD_NAME
kubectl logs POD_NAME --previous
kubectl get events --sort-by=.metadata.creationTimestamp
kubectl describe deployment codify-web
kubectl get endpointslices
kubectl top pods
kubectl top nodes

Replace placeholders with actual resource names. Some commands require metrics components or suitable cluster permissions.

Common Problems

ImagePullBackOff

CrashLoopBackOff

Pending Pod

Service Has No Working Backends

Application Is Unreachable

28. Kubernetes Security

29. Docker and Kubernetes Together

A common delivery workflow:

  1. Build the image in CI.
  2. Run tests and security checks.
  3. Tag the image with a version or commit identifier.
  4. Push the image to a registry.
  5. Update the Kubernetes workload.
  6. Monitor rollout health.
  7. Roll back if the deployment fails.
  8. Record the release and its changes.

Do not rebuild different images under the same mutable tag for separate environments without a clear release policy.

30. Practical Projects

Project 1: Containerized Static Website

Build an Nginx image, serve a custom page, and document the build and run commands.

Project 2: Docker Compose Application

Run a simple API and database using Compose. Use persistent storage, configuration variables, health checks, and a safe development-only database credential.

Project 3: Kubernetes Web Deployment

Deploy a web server with multiple replicas, a Service, health probes, resource requests, and a documented rollback process.

Project 4: CI/CD Pipeline

Automate tests, image building, scanning, registry publishing, and a controlled deployment to a test environment.

Project 5: Troubleshooting Lab

Create intentional failures in a local environment: invalid image tag, wrong Service selector, and insufficient resource allocation. Diagnose them using logs, events, and kubectl describe.

31. Learning Roadmap

Stage Focus
1 Linux, networking, and Git
2 Docker installation and commands
3 Dockerfiles, image layers, and volumes
4 Networking, registries, and Compose
5 Container security and image scanning
6 Kubernetes architecture and core objects
7 Deployments, Services, and configuration
8 Probes, resources, scaling, and rollbacks
9 Storage, networking, and troubleshooting
10 CI/CD, monitoring, and portfolio projects

32. Final Checklist

Conclusion

Docker packages applications into consistent containers, while Kubernetes orchestrates those containers across a cluster. Learn Docker first, then Kubernetes fundamentals, then security, automation, monitoring, and production troubleshooting.

More guides