Guides › Cloud Computing & DevOps
Docker & Kubernetes: Complete Guide
Master Docker and Kubernetes from fundamentals to production concepts. Learn containers, images, Dockerfiles, registries, volumes, networking, Docker Compose, Kubernetes architecture, Pods, Deployments, Services, ConfigMaps, Secrets, storage, autoscaling, security, troubleshooting, and hands-on depl
Docker & Kubernetes: Complete Guide
1. Introduction
Docker and Kubernetes solve related but different problems.
Docker is a platform and ecosystem used to build, package, distribute, and run applications in containers.
Kubernetes is a container orchestration platform that manages containerized workloads across a cluster of machines.
A typical workflow is:
- Write application code.
- Package the application into a container image.
- Test the image locally.
- Store the image in a container registry.
- Deploy the image to a server or Kubernetes cluster.
- Monitor, update, and troubleshoot the application.
Containers improve consistency, but they do not automatically guarantee security, performance, or portability. Host operating systems, CPU architectures, external services, permissions, and configuration still matter.
2. Containers vs Virtual Machines
A virtual machine includes a guest operating system running on virtualized hardware. A container normally shares the host kernel while isolating application processes and resources.
Virtual Machines
Advantages:
- Strong separation between guest operating systems.
- Support for different operating systems on compatible hypervisors.
- Useful for workloads requiring full OS control.
Trade-offs:
- Often need more resources.
- Require management of guest operating systems and their updates.
Containers
Advantages:
- Usually lightweight and quick to start.
- Package application dependencies consistently.
- Support repeatable build and deployment workflows.
Trade-offs:
- Share the host kernel.
- Require careful isolation and permissions.
- Do not remove the need for host and dependency patching.
3. Install and Verify Docker
Install Docker Engine or Docker Desktop using the official instructions for your operating system. On Linux, installation and service-management steps differ by distribution.
Verify installation:
docker --version
docker info
docker run --rm hello-world
The hello-world image confirms that Docker can run a basic container. The first run may download the image.
If commands fail, check that the Docker service is running and that your account has permission to access the Docker daemon. Membership in the Docker group can effectively grant root-level control on many Linux systems, so do not grant it casually.
4. Core Docker Concepts
Image
An image is a packaged, versioned filesystem and configuration used to create containers. Images are built from layers.
Container
A container is a running or stopped instance of an image with its own process configuration and isolated runtime environment.
Registry
A registry stores and distributes images. Examples include Docker Hub and private registries provided by cloud platforms.
Volume
A volume provides persistent storage outside the writable layer of an individual container.
Network
A Docker network controls how containers communicate with one another and with other systems.
Dockerfile
A Dockerfile describes instructions for building an image.
5. Essential Docker Commands
Images
docker pull nginx:stable
docker images
docker image inspect nginx:stable
docker image rm nginx:stable
Containers
docker run -d --name web -p 8080:80 nginx:stable
docker ps
docker ps -a
docker logs web
docker logs -f web
docker exec -it web sh
docker stop web
docker start web
docker restart web
docker rm -f web
Open http://localhost:8080 in a browser when the container is running and the port is available.
-d runs the container in the background. -p 8080:80 maps host port 8080 to container port 80. Use a different host port if 8080 is already in use.
Resource and Cleanup Commands
docker stats
docker inspect web
docker system df
docker container prune
docker image prune
Cleanup commands can delete stopped containers or unused resources. Review what will be removed before running destructive commands.
6. Writing Your First Dockerfile
Create a folder containing index.html:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Docker Demo</title>
</head>
<body>
<h1>Hello from Docker!</h1>
<p>This page is served from a container.</p>
</body>
</html>
Create a file named Dockerfile in the same folder:
FROM nginx:stable
COPY index.html /usr/share/nginx/html/index.html
EXPOSE 80
Build and run it:
docker build -t codify-web:1.0 .
docker run --rm -d --name codify-demo -p 8080:80 codify-web:1.0
Visit http://localhost:8080. Stop the demo with:
docker stop codify-demo
EXPOSE documents the container's intended listening port; it does not publish that port to the host. Publishing is done with -p or the appropriate runtime configuration.
7. Dockerfile Instructions
FROM: Selects a base image.WORKDIR: Sets the working directory for subsequent instructions.COPY: Copies files from the build context.RUN: Executes commands during image building.ENV: Defines environment variables.ARG: Defines build-time variables.EXPOSE: Documents a listening port.USER: Sets the runtime user for subsequent instructions.CMD: Supplies the default command or arguments.ENTRYPOINT: Defines the main executable.HEALTHCHECK: Defines a health-check command where supported.
Prefer a small, maintained base image, pin important versions, avoid embedding secrets, and use .dockerignore to keep unnecessary files out of the build context.
Example .dockerignore:
.git
.env
node_modules
__pycache__
*.log
Adapt the exclusions to your project. Do not exclude files required to build the application.
8. Build a Python Application Image
Create app.py:
from http.server import BaseHTTPRequestHandler, HTTPServer
import os
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
body = b"Hello from the Codify container!\n"
self.send_response(200)
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
port = int(os.getenv("PORT", "8000"))
server = HTTPServer(("0.0.0.0", port), Handler)
server.serve_forever()
Create Dockerfile:
FROM python:3.12-slim
WORKDIR /app
COPY app.py .
ENV PORT=8000
EXPOSE 8000
USER 10001
CMD ["python", "app.py"]
Build and run:
docker build -t codify-python:1.0 .
docker run --rm -p 8000:8000 codify-python:1.0
Test in another terminal:
curl http://localhost:8000
Expected response:
Hello from the Codify container!
This example is for learning. For production, use an appropriate production server, graceful shutdown handling, health checks, and a more complete application configuration.
9. Docker Image Layers and Caching
Docker builds images in layers. If an instruction and its inputs have not changed, the build may reuse the cached result.
Good practices:
- Copy dependency manifests before frequently changing source files.
- Install dependencies in a reproducible way.
- Use
.dockerignore. - Avoid downloading unnecessary tools into the final image.
- Use multi-stage builds when build tools are not required at runtime.
A multi-stage build separates compilation from the final runtime image, reducing unnecessary files and dependencies.
10. Docker Volumes and Persistent Data
Containers can be removed and recreated. Data that must survive container replacement should live in persistent storage.
Create and inspect a volume:
docker volume create codify-data
docker volume ls
docker volume inspect codify-data
Run a test container using the volume:
docker run --rm \
-v codify-data:/data \
alpine \
sh -c "echo persistent-data > /data/example.txt"
Read the data again:
docker run --rm \
-v codify-data:/data \
alpine \
cat /data/example.txt
The data remains in the named volume after each temporary container exits. Deleting a volume is destructive, so back up important data before removing it.
11. Docker Networking
Common network types include:
- Bridge: Commonly used for communication between containers on a Docker host.
- Host: Uses the host network namespace on supported platforms.
- None: Disables normal container networking.
- Overlay: Supports networking across hosts in compatible orchestration setups.
Create a user-defined bridge network:
docker network create codify-net
docker network ls
docker network inspect codify-net
Containers attached to the same user-defined network can generally reach each other by container name or network alias.
Do not publish database ports to every host interface unless the use case requires it. Bind to localhost for local-only testing when appropriate.
12. Docker Compose
Docker Compose describes a multi-container application in a YAML file.
Example compose.yaml:
services:
web:
image: nginx:stable
ports:
- "8080:80"
restart: unless-stopped
test:
image: alpine:3
command: ["sh", "-c", "echo Compose is working"]
Start the web service:
docker compose up -d web
docker compose ps
docker compose logs web
Run the test service:
docker compose run --rm test
Stop and remove the Compose-managed resources:
docker compose down
Use Compose for local development, demonstrations, and suitable single-host deployments. It is not a replacement for Kubernetes in every environment.
13. Environment Variables and Secrets
Use environment variables for configuration that differs between environments, such as a listening port or feature flag.
Avoid placing passwords, private keys, or API tokens in:
- Dockerfiles.
- Image layers.
- Git repositories.
- Public logs.
- Unprotected environment files.
For local development, an ignored .env file can be convenient, but it is not a complete production secrets-management solution. Use a suitable secret manager or orchestrator-supported secret mechanism for sensitive production credentials.
14. Image Registries and Versioning
Use meaningful, traceable tags:
codify-api:1.0.0
codify-api:1.1.0
codify-api:2.0.0
A typical push workflow is:
docker login
docker tag codify-api:1.0.0 YOUR_REGISTRY/codify-api:1.0.0
docker push YOUR_REGISTRY/codify-api:1.0.0
Replace the placeholder with the actual registry and repository. Avoid publishing private images to public repositories.
For stronger reproducibility, pin image digests where suitable and maintain a policy for image updates and vulnerability scanning.
15. Container Security Best Practices
- Use maintained base images.
- Run as a non-root user where possible.
- Scan images and dependencies for vulnerabilities.
- Do not embed secrets in images.
- Limit Linux capabilities and runtime privileges.
- Avoid privileged containers unless absolutely necessary.
- Keep the host and container runtime updated.
- Use read-only filesystems when appropriate.
- Restrict network access.
- Use resource limits.
- Sign or verify images when supported by your delivery process.
- Keep an inventory of deployed images.
A container is not automatically a security boundary strong enough for every threat model.
16. What Is Kubernetes?
Kubernetes automates the deployment, scaling, networking, and lifecycle management of containerized applications.
Important capabilities include:
- Scheduling workloads on cluster nodes.
- Restarting failed containers.
- Scaling replicas.
- Service discovery and load distribution.
- Rolling updates and rollbacks.
- Configuration and secret distribution.
- Persistent storage integration.
Kubernetes introduces operational complexity. For a small application, a managed app platform or simpler container service may be more appropriate.
17. Kubernetes Architecture
A Kubernetes cluster has a control plane and worker nodes.
Control Plane Components
- API server: Entry point for Kubernetes API requests.
- etcd: Stores cluster state and configuration data.
- Scheduler: Chooses suitable nodes for Pods.
- Controller manager: Runs controllers that reconcile desired and actual state.
- Cloud controller manager: Integrates with cloud infrastructure where used.
Managed Kubernetes providers may operate some or all control-plane components on the customer's behalf.
Worker Node Components
- Kubelet: Ensures assigned Pods are running as specified.
- Container runtime: Runs containers.
- Network implementation: Provides Pod networking and related connectivity.
The precise architecture depends on the cluster distribution and provider.
18. Kubernetes Core Objects
Pod
A Pod is the smallest deployable unit in Kubernetes. It can contain one or more closely related containers that share certain resources, such as networking.
Pods are generally disposable. Create them through workload controllers rather than relying on manually created Pods for long-running applications.
Deployment
A Deployment manages a desired number of replicated Pods and supports controlled updates.
ReplicaSet
A ReplicaSet maintains a specified number of Pod replicas. Deployments normally manage ReplicaSets for you.
Service
A Service provides a stable network endpoint for a set of Pods.
Common types include:
- ClusterIP.
- NodePort.
- LoadBalancer.
- ExternalName.
ConfigMap
A ConfigMap stores non-confidential configuration data.
Secret
A Secret is intended for confidential configuration. Access controls, encryption configuration, and cluster security remain important; base64 encoding alone is not encryption.
Namespace
Namespaces group namespaced resources and help separate workloads within a cluster. A namespace by itself is not a complete security boundary.
PersistentVolume and PersistentVolumeClaim
These objects represent storage resources and requests for storage. Actual provisioning depends on the cluster and storage driver.
19. Install kubectl and Access a Cluster
kubectl is the command-line client for Kubernetes. Use the official installation instructions and a local learning environment or managed cluster.
Verify the client:
kubectl version --client
kubectl config get-contexts
kubectl config current-context
Check connectivity to the selected cluster:
kubectl cluster-info
kubectl get nodes
kubectl get namespaces
Always check the current context before running commands against a shared or production cluster.
20. Deploy an Application to Kubernetes
Create deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: codify-web
spec:
replicas: 2
selector:
matchLabels:
app: codify-web
template:
metadata:
labels:
app: codify-web
spec:
containers:
- name: nginx
image: nginx:stable
ports:
- containerPort: 80
readinessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 3
periodSeconds: 5
resources:
requests:
cpu: "100m"
memory: "64Mi"
limits:
cpu: "500m"
memory: "128Mi"
Apply it:
kubectl apply -f deployment.yaml
kubectl get deployments
kubectl get pods -l app=codify-web
kubectl rollout status deployment/codify-web
The image tag is illustrative. In production, select a maintained version and follow your image pinning policy.
21. Expose the Application with a Service
Create service.yaml:
apiVersion: v1
kind: Service
metadata:
name: codify-web
spec:
type: ClusterIP
selector:
app: codify-web
ports:
- port: 80
targetPort: 80
Apply and inspect it:
kubectl apply -f service.yaml
kubectl get service codify-web
For local testing, forward a local port:
kubectl port-forward service/codify-web 8080:80
Open http://localhost:8080 while port forwarding is running. For public access, choose and secure an appropriate Ingress, Gateway, or LoadBalancer setup supported by the cluster.
22. Kubernetes Configuration and Secrets
Example ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: codify-config
data:
APP_MODE: "development"
Create a Secret using an appropriate secure workflow rather than committing real credentials into source control.
For example, in a disposable local cluster, a demonstration Secret can be created with:
kubectl create secret generic codify-demo \
--from-literal=DEMO_VALUE=replace-me
Do not use actual passwords or tokens in tutorials, screenshots, or public repositories. Restrict who can read Secrets and configure encryption at rest where supported.
23. Kubernetes Health Probes
- Startup probe: Determines whether a slow-starting application has finished starting.
- Readiness probe: Determines whether a Pod should receive traffic.
- Liveness probe: Helps detect a container that needs restarting.
Use endpoints that reflect the actual application state. An overly aggressive liveness probe can repeatedly restart an otherwise recoverable service.
24. Kubernetes Scaling and Updates
Scale a Deployment:
kubectl scale deployment/codify-web --replicas=3
kubectl get pods
Inspect rollout history:
kubectl rollout history deployment/codify-web
kubectl rollout status deployment/codify-web
Roll back when appropriate:
kubectl rollout undo deployment/codify-web
A rollback of the workload specification does not automatically reverse database schema changes or other external side effects. Plan those separately.
For automatic scaling, learn Horizontal Pod Autoscaler, metrics availability, resource requests, and workload behavior under load.
25. Resource Requests and Limits
Requests inform scheduling and resource allocation. Limits constrain resource consumption according to Kubernetes' resource model.
Set requests based on observed application behavior. Memory limits that are too low can cause out-of-memory termination, while CPU limits can throttle workloads.
Monitor real usage before selecting production values.
26. Kubernetes Storage
Learn:
- StorageClass.
- PersistentVolume.
- PersistentVolumeClaim.
- Dynamic provisioning.
- Access modes.
- Snapshots and restore procedures.
- Storage performance and availability.
- Stateful workloads.
Do not assume that a persistent volume is automatically backed up. Confirm the storage provider's capabilities and test restoration.
27. Troubleshooting Kubernetes
Useful commands:
kubectl get pods -A
kubectl describe pod POD_NAME
kubectl logs POD_NAME
kubectl logs POD_NAME --previous
kubectl get events --sort-by=.metadata.creationTimestamp
kubectl describe deployment codify-web
kubectl get endpointslices
kubectl top pods
kubectl top nodes
Replace placeholders with actual resource names. Some commands require metrics components or suitable cluster permissions.
Common Problems
ImagePullBackOff
- Check the image name and tag.
- Confirm registry access and image-pull credentials.
- Check network connectivity and registry availability.
CrashLoopBackOff
- Inspect current and previous logs.
- Review the container command, environment, permissions, and resource limits.
- Check whether probes are misconfigured.
Pending Pod
- Inspect events.
- Check CPU and memory requests.
- Review node selectors, taints, tolerations, and storage claims.
Service Has No Working Backends
- Check Service selectors against Pod labels.
- Confirm readiness status.
- Inspect EndpointSlices and network policies.
Application Is Unreachable
- Check Service ports and target ports.
- Test from inside the cluster.
- Inspect Ingress or Gateway configuration and firewall rules.
- Verify DNS and TLS settings.
28. Kubernetes Security
- Apply least-privilege RBAC.
- Restrict API access.
- Use namespaces thoughtfully.
- Configure NetworkPolicies where supported.
- Run containers as non-root where possible.
- Drop unnecessary capabilities.
- Avoid privileged containers and host access.
- Use image scanning and trusted registries.
- Protect Secrets and cluster credentials.
- Set resource requests and limits.
- Keep Kubernetes and add-ons updated.
- Enable audit logging and monitoring.
- Test backup and restore procedures.
29. Docker and Kubernetes Together
A common delivery workflow:
- Build the image in CI.
- Run tests and security checks.
- Tag the image with a version or commit identifier.
- Push the image to a registry.
- Update the Kubernetes workload.
- Monitor rollout health.
- Roll back if the deployment fails.
- Record the release and its changes.
Do not rebuild different images under the same mutable tag for separate environments without a clear release policy.
30. Practical Projects
Project 1: Containerized Static Website
Build an Nginx image, serve a custom page, and document the build and run commands.
Project 2: Docker Compose Application
Run a simple API and database using Compose. Use persistent storage, configuration variables, health checks, and a safe development-only database credential.
Project 3: Kubernetes Web Deployment
Deploy a web server with multiple replicas, a Service, health probes, resource requests, and a documented rollback process.
Project 4: CI/CD Pipeline
Automate tests, image building, scanning, registry publishing, and a controlled deployment to a test environment.
Project 5: Troubleshooting Lab
Create intentional failures in a local environment: invalid image tag, wrong Service selector, and insufficient resource allocation. Diagnose them using logs, events, and kubectl describe.
31. Learning Roadmap
| Stage | Focus |
|---|---|
| 1 | Linux, networking, and Git |
| 2 | Docker installation and commands |
| 3 | Dockerfiles, image layers, and volumes |
| 4 | Networking, registries, and Compose |
| 5 | Container security and image scanning |
| 6 | Kubernetes architecture and core objects |
| 7 | Deployments, Services, and configuration |
| 8 | Probes, resources, scaling, and rollbacks |
| 9 | Storage, networking, and troubleshooting |
| 10 | CI/CD, monitoring, and portfolio projects |
32. Final Checklist
- Explain images, containers, registries, and volumes.
- Write and debug a Dockerfile.
- Build and run a container.
- Use persistent storage and container networks.
- Configure a Compose application.
- Explain Kubernetes control plane and worker nodes.
- Create Deployments and Services.
- Use ConfigMaps and Secrets safely.
- Configure health probes and resource requests.
- Diagnose common Pod and networking failures.
- Demonstrate a safe update and rollback.
- Build a documented container deployment project.
Conclusion
Docker packages applications into consistent containers, while Kubernetes orchestrates those containers across a cluster. Learn Docker first, then Kubernetes fundamentals, then security, automation, monitoring, and production troubleshooting.
More guides
- How to Use Codify: A 7-Day Start Plan — New here? Pick a roadmap, make a study plan, start the DSA sheet, revise with notes, practise for placements and test yourself — one step a day, then a daily routine.
- Infosys Preparation Guide: Online Test, Pseudocode and Interview — Roles (SE, DSE, SP), what each section of the Infosys online test checks, how to crack pseudocode and reasoning, a 4-week plan and the interview.
- Wipro Elite NLTH Preparation: Aptitude, Essay and Coding — The Wipro fresher test round by round — quant, logical and verbal, the written essay, two coding problems — plus a 3-week plan and interview tips.
- Accenture Preparation Guide: Cognitive, Technical, Coding and Communication — Every Accenture round explained — cognitive, MS Office/networking/cloud technical MCQs, pseudocode, coding and the spoken-English test — with a 4-week plan.