diff --git a/.gitignore b/.gitignore index 45f14ccf09..097aa73434 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,18 @@ __pycache__/ # Do not commit real inventory with IPs if you don't want # ansible/inventory/hosts.ini + +# --- Lab 18 / Nix --- +result +result-* +.direnv/ +labs/lab18/app_python/venv*/ +labs/lab18/app_python/freeze*.txt +labs/lab18/app_python/requirements-unpinned.txt +labs/lab18/app_python/*.tar +labs/lab18/app_python/*.tar.gz + +# Lab 18 generated local files +labs/lab18/app_python/venv*/ +labs/lab18/app_python/result +result diff --git a/labs/lab18.md b/labs/lab18.md index 3491394659..864df70baa 100644 --- a/labs/lab18.md +++ b/labs/lab18.md @@ -1,430 +1,1306 @@ -# Lab 18 — Decentralized Hosting with 4EVERLAND & IPFS +# Lab 18 — Reproducible Builds with Nix ![difficulty](https://img.shields.io/badge/difficulty-intermediate-yellow) -![topic](https://img.shields.io/badge/topic-Web3%20Infrastructure-blue) -![points](https://img.shields.io/badge/points-20-orange) -![type](https://img.shields.io/badge/type-Exam%20Alternative-purple) +![topic](https://img.shields.io/badge/topic-Nix%20%26%20Reproducibility-blue) +![points](https://img.shields.io/badge/points-12-orange) -> Deploy content to the decentralized web using IPFS and 4EVERLAND for permanent, censorship-resistant hosting. +> **Goal:** Learn to create truly reproducible builds using Nix, eliminating "works on my machine" problems and achieving bit-for-bit reproducibility. +> **Deliverable:** A PR/MR from `feature/lab18` to the course repo with `labs/submission18.md` containing build artifacts, hash comparisons, Nix expressions, and analysis. Submit the PR/MR link via Moodle. -## Overview - -The decentralized web (Web3) offers an alternative to traditional hosting where content is stored across a distributed network rather than centralized servers. IPFS (InterPlanetary File System) is the foundation, and 4EVERLAND provides a user-friendly gateway to this ecosystem. +--- -**This is an Exam Alternative Lab** — Complete both Lab 17 and Lab 18 to replace the final exam. +## Overview -**What You'll Learn:** -- IPFS fundamentals and content addressing -- Decentralized storage concepts -- Pinning services and persistence -- 4EVERLAND hosting platform -- Centralized vs decentralized trade-offs +In this lab you will practice: +- Installing Nix and understanding the Nix philosophy +- Writing Nix derivations to build software reproducibly +- Creating reproducible Docker images using Nix +- Using Nix Flakes for modern, declarative dependency management +- **Comparing Nix with your previous work from Labs 1-2** -**Prerequisites:** Basic understanding of web hosting, completed Docker lab +**Why Nix?** Traditional build tools (Docker, npm, pip, etc.) claim to be reproducible, but they're not: +- `Dockerfile` with `apt-get install nodejs` gets different versions over time +- `pip install -r requirements.txt` without hash pinning can vary +- Docker builds include timestamps and vary across machines -**Tech Stack:** IPFS | 4EVERLAND | Docker | Content Addressing +**Nix solves this:** Every build is isolated in a sandbox with exact dependencies. The same Nix expression produces **identical binaries** on any machine, forever. -**Provided Files:** -- `labs/lab18/index.html` — A beautiful course landing page ready to deploy +**Building on Your Work:** Throughout this lab, you'll revisit your DevOps Info Service from Lab 1 and compare: +- **Lab 1**: `requirements.txt` vs Nix derivations for dependency management +- **Lab 2**: Traditional `Dockerfile` vs Nix `dockerTools` for containerization +- **Lab 10** *(bonus task)*: Helm `values.yaml` version pinning vs Nix Flakes locking --- -## Exam Alternative Requirements +## Prerequisites -| Requirement | Details | -|-------------|---------| -| **Deadline** | 1 week before exam date | -| **Minimum Score** | 16/20 points | -| **Must Complete** | Both Lab 17 AND Lab 18 | -| **Total Points** | 40 pts (replaces 40 pt exam) | +- **Required:** Completed Labs 1-16 (all required course labs) +- **Key Labs Referenced:** + - Lab 1: Python DevOps Info Service (you'll rebuild with Nix) + - Lab 2: Docker containerization (you'll compare with Nix dockerTools) + - Lab 10: Helm charts (you'll compare version pinning with Nix Flakes) +- Linux, macOS, or WSL2 +- Basic understanding of package managers +- Your `app_python/` directory from Lab 1-2 available --- ## Tasks -### Task 1 — IPFS Fundamentals (3 pts) +### Task 1 — Build Reproducible Python App (Revisiting Lab 1) (6 pts) + +**Objective:** Use Nix to build your DevOps Info Service from Lab 1 and compare Nix's reproducibility guarantees with traditional `pip install -r requirements.txt`. + +**Why This Matters:** You've already built this app in Lab 1 using `requirements.txt`. Now you'll see how Nix provides **true reproducibility** that `pip` cannot guarantee - the same derivation produces bit-for-bit identical results across different machines and times. + +#### 1.1: Install Nix Package Manager + +> ⚠️ **Important Installation Requirements:** +> - Requires sudo/admin access on your machine +> - Creates `/nix` directory at system root (Linux/macOS) or `C:\nix` (Windows WSL) +> - Modifies shell configuration files (`~/.bashrc`, `~/.zshrc`, etc.) +> - Installation size: ~500MB-1GB for base system +> - **Cannot be installed in home directory only** +> - Uninstallation requires manual cleanup (see [official guide](https://nixos.org/manual/nix/stable/installation/uninstall.html)) + +1. **Install Nix using the Determinate Systems installer (recommended):** + + ```bash + curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install + ``` + + > **Why Determinate Nix?** It enables flakes by default and provides better defaults for modern Nix usage. + +
+ 🐧 Alternative: Official Nix installer + + ```bash + sh <(curl -L https://nixos.org/nix/install) --daemon + ``` + + Then enable flakes by adding to `~/.config/nix/nix.conf`: + ``` + experimental-features = nix-command flakes + ``` + +
+ +2. **Verify Installation:** + + ```bash + nix --version + ``` + + You should see Nix 2.x or higher. + + **Restart your terminal** after installation to load Nix into your PATH. + +3. **Test Basic Nix Usage:** + + ```bash + # Try running a program without installing it + nix run nixpkgs#hello + ``` + + This downloads and runs `hello` without installing it permanently. + +#### 1.2: Prepare Your Python Application + +1. **Copy your Lab 1 app to the lab18 directory:** + + ```bash + mkdir -p labs/lab18/app_python + cp -r app_python/* labs/lab18/app_python/ + cd labs/lab18/app_python + ``` + + You should have: + - `app.py` - Your DevOps Info Service + - `requirements.txt` - Your Python dependencies (Flask/FastAPI) + +2. **Review your traditional workflow (Lab 1):** + + Recall how you built this in Lab 1: + ```bash + python -m venv venv + source venv/bin/activate + pip install -r requirements.txt + python app.py + ``` + + **Problems with this approach:** + - Different Python versions on different machines + - `pip install` without hashes can pull different package versions + - Virtual environment is not portable + - No guarantee of reproducibility over time + +#### 1.3: Write a Nix Derivation for Your Python App + +1. **Create a Nix derivation:** + + Create `default.nix` in `labs/lab18/app_python/`: + +
+ 📚 Where to learn Nix Python derivation syntax + + - [nix.dev - Python](https://nix.dev/tutorials/nixos/building-and-running-python-apps) + - [nixpkgs Python documentation](https://nixos.org/manual/nixpkgs/stable/#python) + - [Nix Pills - Chapter 6: Our First Derivation](https://nixos.org/guides/nix-pills/our-first-derivation.html) + + **Key concepts you need:** + - `python3Packages.buildPythonApplication` - Function to build Python apps + - `propagatedBuildInputs` - Python dependencies (Flask/FastAPI) + - `makeWrapper` - Wraps Python script with interpreter + - `pname` - Package name + - `version` - Package version + - `src` - Source code location (use `./.` for current directory) + - `format = "other"` - For apps without setup.py + + **Translating requirements.txt to Nix:** + Your Lab 1 `requirements.txt` might have: + ``` + Flask==3.1.0 + Werkzeug>=2.0 + click + ``` + + In Nix, you reference packages from nixpkgs (not exact PyPI versions): + - `Flask==3.1.0` → `pkgs.python3Packages.flask` + - `fastapi==0.115.0` → `pkgs.python3Packages.fastapi` + - `uvicorn[standard]` → `pkgs.python3Packages.uvicorn` + + **Note:** Nix uses versions from the pinned nixpkgs, not PyPI directly. This is intentional for reproducibility. + + **Example structure (Flask):** + ```nix + { pkgs ? import {} }: + + pkgs.python3Packages.buildPythonApplication { + pname = "devops-info-service"; + version = "1.0.0"; + src = ./.; + + format = "other"; + + propagatedBuildInputs = with pkgs.python3Packages; [ + flask + ]; + + nativeBuildInputs = [ pkgs.makeWrapper ]; + + installPhase = '' + mkdir -p $out/bin + cp app.py $out/bin/devops-info-service + + # Wrap with Python interpreter so it can execute + wrapProgram $out/bin/devops-info-service \ + --prefix PYTHONPATH : "$PYTHONPATH" + ''; + } + ``` + + **Example for FastAPI:** + ```nix + propagatedBuildInputs = with pkgs.python3Packages; [ + fastapi + uvicorn + ]; + ``` + + **Hint:** If you get "command not found" errors, make sure you're using `makeWrapper` in the installPhase. + +
+ +2. **Build your application with Nix:** + + ```bash + nix-build + ``` + + This creates a `result` symlink pointing to the Nix store path. + +3. **Run the Nix-built application:** + + ```bash + ./result/bin/devops-info-service + ``` + + Visit `http://localhost:5000` (or your configured port) - it should work identically to your Lab 1 version! + +#### 1.4: Prove Reproducibility (Compare with Lab 1 approach) + +1. **Record the Nix store path:** + + ```bash + readlink result + ``` + + Note the store path (e.g., `/nix/store/abc123-devops-info-service-1.0.0/`) + +2. **Build again and compare:** + + ```bash + rm result + nix-build + readlink result + ``` + + **Observation:** The store path is **identical**! But wait - did Nix rebuild it or reuse it? + + **Answer: Nix reused the cached build!** Same inputs = same hash = reuse existing store path. + +3. **Force an actual rebuild to prove reproducibility:** + + ```bash + # First, find your build's store path + STORE_PATH=$(readlink result) + echo "Original store path: $STORE_PATH" + + # Delete it from the Nix store + nix-store --delete $STORE_PATH + + # Now rebuild (this forces actual compilation) + rm result + nix-build + readlink result + ``` + + **Observation:** Same store path returns! Nix rebuilt it from scratch and got the exact same hash. -**Objective:** Understand IPFS concepts and run a local node. +3. **Compare with traditional pip approach:** -**Requirements:** + **Demonstrate pip's limitations:** -1. **Study IPFS Concepts** - - Content addressing vs location addressing - - CIDs (Content Identifiers) - - Pinning and garbage collection - - IPFS gateways + ```bash + # Test 1: Install without version pins (shows immediate non-reproducibility) + echo "flask" > requirements-unpinned.txt # No version specified -2. **Run Local IPFS Node** - - Use Docker to run IPFS node - - Access the Web UI - - Understand node configuration + python -m venv venv1 + source venv1/bin/activate + pip install -r requirements-unpinned.txt + pip freeze | grep -i flask > freeze1.txt + deactivate -3. **Add Content Locally** - - Add a file to your local IPFS node - - Retrieve the CID - - Access via local gateway + # Simulate time passing: clear pip cache + pip cache purge 2>/dev/null || rm -rf ~/.cache/pip + + python -m venv venv2 + source venv2/bin/activate + pip install -r requirements-unpinned.txt + pip freeze | grep -i flask > freeze2.txt + deactivate + + # Compare Flask versions + diff freeze1.txt freeze2.txt + ``` + + **Observation:** + - Without version pins, you get whatever's latest + - **Even with pinned versions** in requirements.txt, you only pin direct dependencies + - Transitive dependencies (dependencies of your dependencies) can still drift + - Over weeks/months, `pip install -r requirements.txt` can produce different environments + + **The fundamental problem:** + ``` + Lab 1 approach: requirements.txt pins what YOU install + Problem: Doesn't pin what FLASK installs (Werkzeug, Click, etc.) + Result: Different machines = different transitive dependency versions + + Nix approach: Pins EVERYTHING in the entire dependency tree + Result: Bit-for-bit identical on all machines, forever + ``` + +4. **Understand Nix's caching behavior:** + + **Key insight:** Nix uses content-addressable storage: + ``` + Store path format: /nix/store/-- + Example: /nix/store/abc123xyz-devops-info-service-1.0.0 + + The is computed from: + - All source code + - All dependencies (transitively!) + - Build instructions + - Compiler flags + - Everything needed to reproduce the build + + Same inputs → Same hash → Reuse existing build (cache hit) + Different inputs → Different hash → New build required + ``` + +5. **Nix's guarantee:** + + ```bash + # Hash the entire Nix output + nix-hash --type sha256 result + ``` + + This hash will be **identical** on any machine, any time, forever - if the inputs don't change. + + This is why Nix can safely share binary caches (cache.nixos.org) - the hash proves the content! + +**📊 Comparison Table - Lab 1 vs Lab 18:** + +| Aspect | Lab 1 (pip + venv) | Lab 18 (Nix) | +|--------|-------------------|--------------| +| Python version | System-dependent | Pinned in derivation | +| Dependency resolution | Runtime (`pip install`) | Build-time (pure) | +| Reproducibility | Approximate (with lockfiles) | Bit-for-bit identical | +| Portability | Requires same OS + Python | Works anywhere Nix runs | +| Binary cache | No | Yes (cache.nixos.org) | +| Isolation | Virtual environment | Sandboxed build | +| Store path | N/A | Content-addressable hash | + +#### 1.5: Optional - Go Application (If you completed Lab 1 Bonus)
-💡 Hints +🎁 For students who built the Go version in Lab 1 Bonus -**IPFS Concepts:** -- **Content Addressing:** Files identified by hash of content, not location -- **CID:** Unique identifier derived from content hash (e.g., `QmXxx...` or `bafyxxx...`) -- **Pinning:** Marking content to keep it (prevent garbage collection) -- **Gateway:** HTTP interface to IPFS network +If you implemented the compiled language bonus in Lab 1, you can also build it with Nix: -**Run IPFS with Docker:** -```bash -docker run -d --name ipfs \ - -p 4001:4001 \ - -p 8080:8080 \ - -p 5001:5001 \ - ipfs/kubo:latest - -# Web UI at http://localhost:5001/webui -# Gateway at http://localhost:8080 -``` +1. **Copy your Go app:** + ```bash + mkdir -p labs/lab18/app_go + cp -r app_go/* labs/lab18/app_go/ + cd labs/lab18/app_go + ``` -**Add Content:** -```bash -# Create test file -echo "Hello IPFS from DevOps course!" > hello.txt +2. **Create `default.nix` for Go:** + ```nix + { pkgs ? import {} }: -# Add to IPFS -docker exec ipfs ipfs add /hello.txt -# Returns: added QmXxx... hello.txt + pkgs.buildGoModule { + pname = "devops-info-service-go"; + version = "1.0.0"; + src = ./.; -# Access via gateway -curl http://localhost:8080/ipfs/QmXxx... -``` + vendorHash = null; # or use pkgs.lib.fakeHash if you have dependencies + } + ``` -**Resources:** -- [IPFS Docs](https://docs.ipfs.tech/) -- [IPFS Concepts](https://docs.ipfs.tech/concepts/) +3. **Build and compare binary size:** + ```bash + nix-build + ls -lh result/bin/ + ``` + + Compare this with your multi-stage Docker build from Lab 2 Bonus!
+In `labs/submission18.md`, document: +- Installation steps and verification output +- Your `default.nix` file with explanations of each field +- Store path from multiple builds (prove they're identical) +- Comparison table: `pip install` vs Nix derivation +- Why does `requirements.txt` provide weaker guarantees than Nix? +- Screenshots showing your Lab 1 app running from Nix-built version +- Explanation of the Nix store path format and what each part means +- **Reflection:** How would Nix have helped in Lab 1 if you had used it from the start? + --- -### Task 2 — 4EVERLAND Setup (3 pts) +### Task 2 — Reproducible Docker Images (Revisiting Lab 2) (4 pts) + +**Objective:** Use Nix's `dockerTools` to containerize your DevOps Info Service and compare with your traditional Dockerfile from Lab 2. + +**Why This Matters:** In Lab 2, you created a `Dockerfile` that built your Python app. While Docker provides isolation, it's **not reproducible**: +- Build timestamps differ between builds +- Base image tags like `python:3.13-slim` can point to different versions over time +- `apt-get` installs latest packages, which change +- Two builds of the same Dockerfile can produce different image hashes + +Nix's `dockerTools` creates **truly reproducible** container images with content-addressable layers. + +#### 2.1: Review Your Lab 2 Dockerfile + +1. **Find your Dockerfile from Lab 2:** + + ```bash + # From repository root directory + cat app_python/Dockerfile + ``` + + You likely have something like: + ```dockerfile + FROM python:3.13-slim + RUN useradd -m appuser + WORKDIR /app + COPY requirements.txt . + RUN pip install -r requirements.txt + COPY app.py . + USER appuser + EXPOSE 5000 + CMD ["python", "app.py"] + ``` + +
+ 💡 Don't have your Lab 2 Dockerfile? + + If you lost your Lab 2 work, create a minimal Dockerfile now: + + ```dockerfile + FROM python:3.13-slim + WORKDIR /app + COPY requirements.txt app.py ./ + RUN pip install -r requirements.txt + EXPOSE 5000 + CMD ["python", "app.py"] + ``` + + Save as `app_python/Dockerfile`. + +
+ +2. **Test Lab 2 Dockerfile reproducibility:** + + ```bash + # Make sure you're in repository root + cd ~/path/to/DevOps-Core-Course # Adjust to your path + + # Build from app_python directory + docker build -t lab2-app:v1 ./app_python + docker inspect lab2-app:v1 | grep Created + + # Wait a few seconds, then rebuild + sleep 5 + docker build -t lab2-app:v2 ./app_python + docker inspect lab2-app:v2 | grep Created + ``` + + **Observation:** Different creation timestamps! The image hashes are different even though the content is identical. + +#### 2.2: Build Docker Image with Nix + +1. **Create a Nix Docker image using `dockerTools`:** + + Create `labs/lab18/app_python/docker.nix`: + +
+ 📚 Where to learn about dockerTools + + - [nix.dev - Building Docker images](https://nix.dev/tutorials/nixos/building-and-running-docker-images.html) + - [nixpkgs dockerTools documentation](https://ryantm.github.io/nixpkgs/builders/images/dockertools/) + + **Key concepts:** + - `pkgs.dockerTools.buildLayeredImage` - Builds efficient layered images + - `name` - Image name + - `tag` - Image tag (optional, defaults to latest) + - `contents` - Packages/derivations to include in the image + - `config.Cmd` - Default command to run + - `config.ExposedPorts` - Ports to expose + + **Critical for reproducibility:** + - **DO NOT** use `created = "now"` - this breaks reproducibility! + - **DO** use `created = "1970-01-01T00:00:01Z"` for reproducible builds + - **DO** use exact derivations (from Task 1) instead of arbitrary packages + + **Example structure:** + ```nix + { pkgs ? import {} }: + + let + app = import ./default.nix { inherit pkgs; }; + in + pkgs.dockerTools.buildLayeredImage { + name = "devops-info-service-nix"; + tag = "1.0.0"; + + contents = [ app ]; + + config = { + Cmd = [ "${app}/bin/devops-info-service" ]; + ExposedPorts = { + "5000/tcp" = {}; + }; + }; + + created = "1970-01-01T00:00:01Z"; # Reproducible timestamp + } + ``` + +
+ +2. **Build the Nix Docker image:** + + ```bash + cd labs/lab18/app_python + nix-build docker.nix + ``` + + This creates a tarball in `result`. + +3. **Load into Docker:** + + ```bash + docker load < result + ``` + + Output shows the image was loaded with a specific tag. + +4. **Run both containers side-by-side:** + + ```bash + # First, clean up any existing containers to avoid port conflicts + docker stop lab2-container nix-container 2>/dev/null || true + docker rm lab2-container nix-container 2>/dev/null || true + + # Run Lab 2 traditional Docker image on port 5000 + docker run -d -p 5000:5000 --name lab2-container lab2-app:v1 + + # Run Nix-built image on port 5001 (mapped to container's 5000) + docker run -d -p 5001:5000 --name nix-container devops-info-service-nix:1.0.0 + ``` + + Test both: + ```bash + curl http://localhost:5000/health # Lab 2 version + curl http://localhost:5001/health # Nix version + ``` + + Both should work identically! + + **Troubleshooting:** + - If port 5000 is in use: `lsof -i :5000` to find the process + - Container won't start: Check logs with `docker logs lab2-container` + - Permission denied: Make sure Docker daemon is running + +#### 2.3: Compare Reproducibility - Lab 2 vs Lab 18 + +**Test 1: Rebuild Reproducibility** -**Objective:** Set up 4EVERLAND account and explore the platform. +1. **Rebuild Nix image multiple times:** -**Requirements:** + ```bash + rm result + nix-build docker.nix + sha256sum result -1. **Create Account** - - Sign up at [4everland.org](https://www.4everland.org/) - - Connect with GitHub or wallet - - Explore dashboard + rm result + nix-build docker.nix + sha256sum result + ``` -2. **Understand Services** - - Hosting: Deploy websites/apps - - Storage: IPFS pinning - - Gateway: Access IPFS content + **Observation:** Identical SHA256 hashes! The tarball is bit-for-bit identical. -3. **Explore Free Tier** - - Understand limits and capabilities - - Review pricing for reference +2. **Compare with Lab 2 Dockerfile:** + + ```bash + # Make sure you're in repository root + # Build Lab 2 Dockerfile twice and compare saved image hashes + + docker build -t lab2-app:test1 ./app_python/ + docker save lab2-app:test1 | sha256sum + + sleep 2 # Wait a moment + + docker build -t lab2-app:test2 ./app_python/ + docker save lab2-app:test2 | sha256sum + ``` + + **Observation:** Different hashes! Even though the Dockerfile and source are identical, Lab 2's approach is not reproducible. + +**Test 2: Image Size Comparison** + +```bash +docker images | grep -E "lab2-app|devops-info-service-nix" +``` + +Create a comparison table: + +| Metric | Lab 2 Dockerfile | Lab 18 Nix dockerTools | +|--------|------------------|------------------------| +| Image size | ~150MB (with python:3.13-slim) | ~50-80MB (minimal closure) | +| Reproducibility | ❌ Different hashes each build | ✅ Identical hashes | +| Build caching | Layer-based (timestamp-dependent) | Content-addressable | +| Base image dependency | Yes (python:3.13-slim) | No base image needed | + +**Test 3: Layer Analysis** + +1. **Examine Lab 2 image layers:** + + ```bash + docker history lab2-app:v1 + ``` + + Note the timestamps in the "CREATED" column - they vary between builds! + +2. **Examine Nix image layers:** + + ```bash + docker history devops-info-service-nix:1.0.0 + ``` + + Nix uses content-addressable layers - same content = same layer hash. + +#### 2.4: Advanced Comparison - Multi-Stage Builds
-💡 Hints +🎁 Optional: Compare with Lab 2 Bonus Multi-Stage Build -**4EVERLAND Services:** -- **Hosting:** Deploy from Git repos, automatic builds -- **Bucket (Storage):** Upload files, get IPFS CIDs -- **Gateway:** Access content via 4everland.link +If you completed the Lab 2 bonus with Go and multi-stage builds, you can compare: -**Dashboard:** -- Projects: Your deployed sites -- Bucket: File storage -- Domains: Custom domain setup +**Your Lab 2 multi-stage Dockerfile:** +```dockerfile +FROM golang:1.22 AS builder +COPY . . +RUN go build -o app main.go -**Free Tier Includes:** -- 100 deployments/month -- 5GB storage -- 100GB bandwidth +FROM alpine:latest +COPY --from=builder /app/app /app +ENTRYPOINT ["/app"] +``` + +**Problems:** +- `golang:1.22` and `alpine:latest` change over time +- Build includes timestamps +- Not reproducible across machines + +**Nix equivalent (fully reproducible):** +```nix +pkgs.dockerTools.buildLayeredImage { + name = "go-app-nix"; + contents = [ goApp ]; # Built in Task 1.5 + config.Cmd = [ "${goApp}/bin/go-app" ]; + created = "1970-01-01T00:00:01Z"; +} +``` -**Resources:** -- [4EVERLAND Docs](https://docs.4everland.org/) +Same result size, but **fully reproducible**!
+**📊 Comprehensive Comparison - Lab 2 vs Lab 18:** + +| Aspect | Lab 2 Traditional Dockerfile | Lab 18 Nix dockerTools | +|--------|------------------------------|------------------------| +| **Base images** | `python:3.13-slim` (changes over time) | No base image (pure derivations) | +| **Timestamps** | Different on each build | Fixed or deterministic | +| **Package installation** | `pip install` at build time | Nix store paths (immutable) | +| **Reproducibility** | ❌ Same Dockerfile → Different images | ✅ Same docker.nix → Identical images | +| **Caching** | Layer-based (breaks on timestamp) | Content-addressable (perfect caching) | +| **Image size** | ~150MB+ with full base image | ~50-80MB with minimal closure | +| **Portability** | Requires Docker | Requires Nix (then loads to Docker) | +| **Security** | Base image vulnerabilities | Minimal dependencies, easier auditing | +| **Lab 2 Learning** | Best practices, non-root user | Build on Lab 2 knowledge | + +In `labs/submission18.md`, document: +- Your `docker.nix` file with explanations of each field +- Side-by-side comparison: Lab 2 Dockerfile vs Nix docker.nix +- SHA256 hash comparison proving Nix reproducibility +- Image size comparison table with analysis +- `docker history` output for both approaches +- Screenshots showing both containers running simultaneously +- **Analysis:** Why can't traditional Dockerfiles achieve bit-for-bit reproducibility? +- **Reflection:** If you could redo Lab 2 with Nix, what would you do differently? +- Practical scenarios where Nix's reproducibility matters (CI/CD, security audits, rollbacks) + --- -### Task 3 — Deploy Static Content (4 pts) +### Bonus Task — Modern Nix with Flakes (Includes Lab 10 Comparison) (2 pts) + +**Objective:** Modernize your Nix expressions using Flakes for better dependency locking and reproducibility. Compare Nix Flakes with Helm's version pinning approach from Lab 10. + +**Why This Matters:** Nix Flakes are the modern standard (2026) for Nix projects. They provide: +- Automatic dependency locking via `flake.lock` +- Standardized project structure +- Better reproducibility across time +- Easier sharing and collaboration + +**Comparison with Lab 10:** In Lab 10 (Helm), you used `values.yaml` to pin image versions. Flakes take this concept further by locking **all** dependencies, not just container images. + +#### Bonus.1: Convert to Flake + +1. **Create a `flake.nix`:** + + Create `labs/lab18/app_python/flake.nix`: + +
+ 📚 Where to learn about Flakes + + - [Zero to Nix - Flakes](https://zero-to-nix.com/concepts/flakes) + - [NixOS Wiki - Flakes](https://wiki.nixos.org/wiki/Flakes) + - [Nix Flakes explained](https://nix.dev/concepts/flakes) + + **Key structure:** + ```nix + { + description = "DevOps Info Service - Reproducible Build"; + + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.11"; # Pin exact nixpkgs version + }; + + outputs = { self, nixpkgs }: + let + # ⚠️ Architecture note: This example uses x86_64-linux + # - Works on: Linux (x86_64), WSL2 + # - Mac Intel: Change to "x86_64-darwin" + # - Mac M1/M2/M3: Change to "aarch64-darwin" + # - For multi-system support, see: https://github.com/numtide/flake-utils + system = "x86_64-linux"; + pkgs = nixpkgs.legacyPackages.${system}; + in + { + packages.${system} = { + default = import ./default.nix { inherit pkgs; }; + dockerImage = import ./docker.nix { inherit pkgs; }; + }; + + # Development shell with all dependencies + devShells.${system}.default = pkgs.mkShell { + buildInputs = with pkgs; [ + python313 + python313Packages.flask # or fastapi + ]; + }; + }; + } + ``` + + **Platform-specific adjustments:** + - **Linux/WSL2**: Use `system = "x86_64-linux";` (shown above) + - **Mac Intel**: Use `system = "x86_64-darwin";` + - **Mac ARM (M1/M2/M3)**: Use `system = "aarch64-darwin";` + + **Hint:** Use `nix flake init` to generate a template, then modify it. + +
+ +2. **Generate lock file:** + + ```bash + cd labs/lab18/app_python + nix flake update + ``` + + This creates `flake.lock` with pinned dependencies. + +3. **Build using flake:** + + ```bash + nix build # Builds default package + nix build .#dockerImage # Builds Docker image + ./result/bin/devops-info-service # Run the app + ``` + +#### Bonus.2: Compare with Lab 10 Helm Values + +**Lab 10 Helm approach to version pinning:** + +In `k8s/mychart/values.yaml`: +```yaml +image: + repository: yourusername/devops-info-service + tag: "1.0.0" # Pin specific version + pullPolicy: IfNotPresent + +# Environment-specific overrides +# values-prod.yaml: +image: + tag: "1.0.0" # Explicit version for prod +``` + +**Limitations:** +- Only pins the container image tag +- Doesn't lock Python dependencies inside the image +- Doesn't lock Helm chart dependencies +- Image tag `1.0.0` could point to different content if rebuilt + +**Nix Flakes approach:** + +`flake.lock` locks **everything**: +```json +{ + "nodes": { + "nixpkgs": { + "locked": { + "lastModified": 1704321342, + "narHash": "sha256-abc123...", + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "52e3e80afff4b16ccb7c52e9f0f5220552f03d04", + "type": "github" + } + } + } +} +``` + +This locks: +- ✅ Exact nixpkgs revision (all 80,000+ packages) +- ✅ Python version and all dependencies +- ✅ Build tools and compilers +- ✅ Everything in the closure + +**Combined Approach:** + +You can use both together! +1. Build reproducible image with Nix: `nix build .#dockerImage` +2. Load to Docker and tag: `docker load < result` +3. Reference in Helm with content hash: `image.tag: "sha256-abc123..."` + +This gives you: +- Helm's declarative Kubernetes deployment +- Nix's perfect reproducibility for the image + +Create a comparison table in your submission. + +#### Bonus.3: Test Cross-Machine Reproducibility + +1. **Commit your flake to git:** + + ```bash + git add flake.nix flake.lock default.nix docker.nix + git commit -m "feat: add Nix flake for reproducible builds" + git push + ``` + +2. **Test on another machine or ask a classmate:** + + ```bash + # Build directly from GitHub + nix build github:yourusername/DevOps-Core-Course?dir=labs/lab18/app_python#default + ``` + +3. **Compare store paths:** + + ```bash + readlink result + ``` + + Both machines should get **identical store paths** - same hash, same content! + +#### Bonus.4: Add Development Shell + +1. **Enter the dev shell:** + + ```bash + nix develop + ``` + + This gives you an isolated environment with exact Python version and dependencies. -**Objective:** Deploy a static site to 4EVERLAND. +2. **Compare with Lab 1 virtual environment:** -**Requirements:** + **Lab 1 approach:** + ```bash + python -m venv venv + source venv/bin/activate + pip install -r requirements.txt + ``` -1. **Use the Provided Static Site** - - A course landing page is provided at `labs/lab18/index.html` - - Review the HTML/CSS to understand the structure - - You may customize it or create your own + **Lab 18 Nix approach:** + ```bash + nix develop + # Python and all dependencies instantly available + # Same environment on every machine + ``` -2. **Deploy via 4EVERLAND** - - Connect your GitHub repository - - Configure build settings - - Deploy to IPFS via 4EVERLAND +3. **Try it:** -3. **Verify Deployment** - - Access via 4EVERLAND URL - - Access via IPFS gateway - - Note the CID + ```bash + nix develop + python --version # Exact pinned version + python -c "import flask; print(flask.__version__)" + ``` -4. **Test Permanence** - - Understand that content with same hash = same CID - - Make a change, redeploy, observe new CID + Exit and enter again - same versions, always! + +**📊 Dependency Management Comparison:** + +| Aspect | Lab 1 (venv + requirements.txt) | Lab 10 (Helm values.yaml) | Lab 18 (Nix Flakes) | +|--------|--------------------------------|---------------------------|---------------------| +| **Locks Python version** | ❌ Uses system Python | ❌ Uses image Python | ✅ Pinned in flake | +| **Locks dependencies** | ⚠️ Approximate (versions drift) | ❌ Only image tag | ✅ Exact hashes | +| **Locks build tools** | ❌ No | ❌ No | ✅ Yes | +| **Reproducibility** | ⚠️ Probabilistic | ⚠️ Tag-based | ✅ Cryptographic | +| **Cross-machine** | ❌ Varies | ⚠️ Depends on image | ✅ Identical | +| **Dev environment** | ✅ Yes (venv) | ❌ No | ✅ Yes (nix develop) | +| **Time-stable** | ❌ Packages update | ⚠️ Tags can change | ✅ Locked forever | + +In `labs/submission18.md`, document: +- Your complete `flake.nix` with explanations +- `flake.lock` snippet showing locked dependencies (especially nixpkgs revision) +- Build outputs from `nix build` +- Proof that builds are identical across machines/time +- Dev shell experience: Compare `nix develop` vs Lab 1's `venv` +- Comparison with Lab 10 Helm values.yaml approach (Bonus.2) +- **Reflection:** How do Flakes improve upon traditional dependency management? +- Practical scenarios where flake.lock prevented a "works on my machine" problem + +--- + +## Troubleshooting Common Issues
-💡 Hints - -**Provided Static Site:** -The course provides a beautiful landing page at `labs/lab18/index.html` that you can deploy. It includes: -- Modern responsive design -- Course curriculum overview -- Learning roadmap -- "Deployed on IPFS" badge - -**Deployment Steps:** -1. Go to 4EVERLAND Dashboard → Hosting -2. Click "New Project" -3. Import from GitHub -4. Select your repository and branch -5. Configure: - - Framework: None (static) - - Build command: (leave empty for static) - - Output directory: `labs/lab18` (or root if you moved the file) -6. Deploy - -**Alternative: Create Your Own** -You can also create your own static site. Keep it simple: -```html - - - - My DevOps Portfolio - - -

Welcome to My DevOps Journey

-

Deployed on IPFS via 4EVERLAND

- - +🔧 Python app doesn't run: "command not found" or "No such file or directory" + +**Problem:** Your `app.py` doesn't have a shebang line and isn't being wrapped with Python interpreter. + +**Solution:** Ensure you're using `makeWrapper` in your `default.nix`: + +```nix +nativeBuildInputs = [ pkgs.makeWrapper ]; + +installPhase = '' + mkdir -p $out/bin + cp app.py $out/bin/devops-info-service + + wrapProgram $out/bin/devops-info-service \ + --prefix PYTHONPATH : "$PYTHONPATH" +''; ``` -**Access URLs:** -- 4EVERLAND: `https://your-project.4everland.app` -- IPFS Gateway: `https://ipfs.4everland.link/ipfs/CID` +Alternatively, add a shebang to your `app.py`: +```python +#!/usr/bin/env python3 +```
---- +
+🔧 "error: hash mismatch in fixed-output derivation" + +**Problem:** The hash you specified doesn't match the actual content. + +**Solution:** +1. Use `pkgs.lib.fakeHash` initially to get the correct hash +2. Nix will fail and tell you the expected hash +3. Replace `fakeHash` with the correct hash from the error message + +Example: +```nix +vendorHash = pkgs.lib.fakeHash; # Start with this +# Error will say: "got: sha256-abc123..." +# Then use: vendorHash = "sha256-abc123..."; +``` -### Task 4 — IPFS Pinning (4 pts) +
-**Objective:** Use 4EVERLAND's storage (Bucket) for IPFS pinning. +
+🔧 Docker image doesn't load or fails to run -**Requirements:** +**Common causes:** -1. **Upload Files to Bucket** - - Upload multiple files (images, documents, etc.) - - Get CIDs for each file +1. **Image tarball not built:** Check `result` is a `.tar.gz` file + ```bash + file result + # Should show: gzip compressed data + ``` -2. **Create a Directory Structure** - - Upload a folder with multiple files - - Understand directory CIDs +2. **Wrong Cmd path:** Verify the app path in docker.nix + ```nix + config.Cmd = [ "${app}/bin/devops-info-service" ]; + # Make sure this matches your installPhase output + ``` -3. **Access via Multiple Gateways** - - Access your content via: - - 4EVERLAND gateway - - Public IPFS gateways (ipfs.io, dweb.link) - - Understand gateway differences +3. **Missing dependencies in image:** Add required packages to `contents` + ```nix + contents = [ app pkgs.coreutils ]; # Add tools if needed + ``` -4. **Verify Pinning** - - Confirm content is pinned - - Understand pinning vs local storage +
-💡 Hints +🔧 Port conflicts when running containers -**Bucket Upload:** -1. Dashboard → Bucket -2. Create new bucket -3. Upload files or folders -4. Get CID from file details +**Problem:** Port 5000 or 5001 already in use. -**Multiple Gateways:** +**Solution:** ```bash -# 4EVERLAND -https://ipfs.4everland.link/ipfs/QmXxx... - -# IPFS.io -https://ipfs.io/ipfs/QmXxx... +# Find what's using the port +lsof -i :5000 -# Cloudflare -https://cloudflare-ipfs.com/ipfs/QmXxx... +# Stop old containers +docker stop $(docker ps -aq) 2>/dev/null -# DWeb.link -https://dweb.link/ipfs/QmXxx... +# Or use different ports +docker run -d -p 5002:5000 --name my-container my-image ``` -**Directory Upload:** -- Upload entire folder -- Get directory CID -- Access files: `gateway/ipfs/DirCID/filename` +
+ +
+🔧 Flakes don't work: "experimental features" error -**Pinning Importance:** -- Unpinned content may be garbage collected -- Pinning services keep content available -- Multiple pins = more redundancy +**Problem:** Flakes not enabled in your Nix configuration. + +**Solution:** +```bash +# Check if flakes are enabled +nix flake --help + +# If error, enable flakes: +mkdir -p ~/.config/nix +echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf + +# Restart terminal +```
---- +
+🔧 Build fails on macOS: "unsupported system" -### Task 5 — IPNS & Updates (3 pts) +**Problem:** Flake hardcodes `x86_64-linux` but you're on macOS. -**Objective:** Understand mutable content with IPNS. +**Solution:** Change the system in `flake.nix`: +```nix +# For Mac Intel: +system = "x86_64-darwin"; -**Requirements:** +# For Mac M1/M2/M3: +system = "aarch64-darwin"; +``` -1. **Understand IPNS** - - IPFS = immutable (content changes = new CID) - - IPNS = mutable pointer to IPFS content - - IPNS name stays same, content can change +
-2. **Explore 4EVERLAND Domains** - - Custom domains for your deployment - - How 4EVERLAND handles updates +
+🔧 "cannot build derivation: no builder for this system" + +**Problem:** Trying to build Linux binaries on macOS or vice versa. -3. **Update Deployment** - - Make changes to your static site - - Redeploy - - Observe: same URL, new CID +**Solution:** Either: +1. Match your system architecture in the flake +2. Use Docker builds which work cross-platform +3. Use Nix's cross-compilation features (advanced) + +
-💡 Hints +🔧 Don't have Lab 1/2 artifacts to use + +**No problem!** Create a minimal example: -**IPFS vs IPNS:** -- **IPFS CID:** `QmXxx...` - changes when content changes -- **IPNS Name:** `/ipns/k51xxx...` - stays same, points to current CID +1. **Create simple Flask app:** + ```python + # app.py + from flask import Flask, jsonify + app = Flask(__name__) -**4EVERLAND Handles This:** -- Your project URL stays constant -- Behind scenes, updates the IPNS pointer -- Users always get latest version + @app.route('/health') + def health(): + return jsonify({"status": "healthy"}) -**Domain Configuration:** -1. Dashboard → Hosting → Your Project -2. Settings → Domains -3. Add custom domain or use provided subdomain + if __name__ == '__main__': + app.run(host='0.0.0.0', port=5000) + ``` + +2. **Create requirements.txt:** + ``` + flask + ``` + +3. **Create basic Dockerfile:** + ```dockerfile + FROM python:3.13-slim + WORKDIR /app + COPY requirements.txt app.py ./ + RUN pip install -r requirements.txt + EXPOSE 5000 + CMD ["python", "app.py"] + ``` + +Now you can proceed with the lab using these minimal examples!
--- -### Task 6 — Documentation & Analysis (3 pts) +## How to Submit -**Objective:** Document your work and analyze decentralized hosting. +1. Create a branch for this lab and push it: -**Create `4EVERLAND.md` with:** + ```bash + git switch -c feature/lab18 + # create labs/submission18.md with your findings + git add labs/submission18.md labs/lab18/ + git commit -m "docs: add lab18 submission - Nix reproducible builds" + git push -u origin feature/lab18 + ``` -1. **Deployment Summary** - - What you deployed - - URLs (4EVERLAND and IPFS gateways) - - CIDs obtained +2. **Open a PR (GitHub) or MR (GitLab)** from your fork's `feature/lab18` branch → **course repository's main branch**. -2. **Screenshots** - - 4EVERLAND dashboard - - Deployed site - - Bucket storage - - Multiple gateway access +3. In the PR/MR description, include: -3. **Centralized vs Decentralized Comparison** + ```text + Platform: [GitHub / GitLab] -| Aspect | Traditional Hosting | IPFS/4EVERLAND | -|--------|---------------------|----------------| -| Content addressing | | | -| Single point of failure | | | -| Censorship resistance | | | -| Update mechanism | | | -| Cost model | | | -| Speed/latency | | | -| Best use cases | | | + - [x] Task 1 — Build Reproducible Artifacts from Scratch (6 pts) + - [x] Task 2 — Reproducible Docker Images with Nix (4 pts) + - [ ] Bonus Task — Modern Nix with Flakes (2 pts) [if completed] + ``` -4. **Use Case Analysis** - - When decentralized hosting makes sense - - When traditional hosting is better - - Your recommendations +4. **Copy the PR/MR URL** and submit it via **Moodle before the deadline**. --- -## Checklist +## Acceptance Criteria -- [ ] IPFS concepts understood -- [ ] Local IPFS node running -- [ ] Content added to local IPFS -- [ ] 4EVERLAND account created -- [ ] Static site deployed via 4EVERLAND -- [ ] Files uploaded to Bucket -- [ ] Content accessed via multiple gateways -- [ ] IPNS/updates understood -- [ ] `4EVERLAND.md` documentation complete -- [ ] Comparison analysis complete +- ✅ Branch `feature/lab18` exists with commits for each task +- ✅ File `labs/submission18.md` contains required outputs and analysis for all completed tasks +- ✅ Directory `labs/lab18/` contains your application code and Nix expressions +- ✅ Nix derivations successfully build reproducible artifacts +- ✅ Docker image built with Nix and compared to traditional Dockerfile +- ✅ Hash comparisons prove reproducibility +- ✅ **Bonus (if attempted):** `flake.nix` and `flake.lock` present and working +- ✅ PR/MR from `feature/lab18` → **course repo main branch** is open +- ✅ PR/MR link submitted via Moodle before the deadline --- -## Rubric - -| Criteria | Points | -|----------|--------| -| **IPFS Fundamentals** | 3 pts | -| **4EVERLAND Setup** | 3 pts | -| **Static Deployment** | 4 pts | -| **IPFS Pinning** | 4 pts | -| **IPNS & Updates** | 3 pts | -| **Documentation** | 3 pts | -| **Total** | **20 pts** | +## Rubric (12 pts max) -**Grading:** -- **18-20:** Excellent understanding, thorough deployment, insightful analysis -- **16-17:** Working deployment, good documentation -- **14-15:** Basic deployment, incomplete analysis -- **<14:** Incomplete deployment +| Criterion | Points | +| --------------------------------------------------- | -----: | +| Task 1 — Build Reproducible Artifacts from Scratch | **6** | +| Task 2 — Reproducible Docker Images with Nix | **4** | +| Bonus Task — Modern Nix with Flakes | **2** | +| **Total** | **12** | --- -## Resources +## Guidelines + +- Use clear Markdown headers to organize sections in `submission18.md` +- Include command outputs and written analysis for each task +- Explain WHY Nix provides better reproducibility than traditional tools +- Compare before/after results when proving reproducibility +- Document challenges encountered and how you solved them +- Include code snippets with explanations, not just paste
-📚 IPFS Documentation +📚 Helpful Resources + +**Official Documentation:** +- [nix.dev - Official tutorials](https://nix.dev/) +- [Zero to Nix - Beginner-friendly guide](https://zero-to-nix.com/) +- [Nix Pills - Deep dive](https://nixos.org/guides/nix-pills/) +- [NixOS Package Search](https://search.nixos.org/) + +**Docker with Nix:** +- [Building Docker images - nix.dev](https://nix.dev/tutorials/nixos/building-and-running-docker-images.html) +- [dockerTools reference](https://ryantm.github.io/nixpkgs/builders/images/dockertools/) + +**Flakes:** +- [Nix Flakes - NixOS Wiki](https://wiki.nixos.org/wiki/Flakes) +- [Flakes - Zero to Nix](https://zero-to-nix.com/concepts/flakes) +- [Practical Nix Flakes](https://serokell.io/blog/practical-nix-flakes) -- [IPFS Docs](https://docs.ipfs.tech/) -- [IPFS Concepts](https://docs.ipfs.tech/concepts/) -- [Content Addressing](https://docs.ipfs.tech/concepts/content-addressing/) -- [IPNS](https://docs.ipfs.tech/concepts/ipns/) +**Community:** +- [awesome-nix - Curated resources](https://github.com/nix-community/awesome-nix) +- [NixOS Discourse](https://discourse.nixos.org/)
-🌐 4EVERLAND +💡 Nix Tips -- [4EVERLAND Docs](https://docs.4everland.org/) -- [Hosting Guide](https://docs.4everland.org/hosting/overview) -- [Bucket (Storage)](https://docs.4everland.org/storage/bucket) +1. **Store paths are content-addressable:** Same inputs = same output hash +2. **Use `nix-shell -p pkg` for quick testing** before adding to derivations +3. **Garbage collect unused builds:** `nix-collect-garbage -d` +4. **Search for packages:** `nix search nixpkgs golang` +5. **Read error messages carefully:** Nix errors are verbose but informative +6. **Use `lib.fakeHash` initially** when you don't know the hash yet +7. **Avoid network access in builds:** Nix sandboxes block network by default +8. **Pin nixpkgs version** for maximum reproducibility
-🔗 Public Gateways +🔧 Troubleshooting + +**If Nix installation fails:** +- Ensure you have multi-user support (daemon mode recommended) +- Check `/nix` directory permissions +- Try the Determinate Systems installer instead of official + +**If builds fail with "hash mismatch":** +- Update the hash in your derivation to match the error message +- Use `lib.fakeHash` to discover the correct hash + +**If Docker load fails:** +- Verify result is a valid tarball: `file result` +- Check Docker daemon is running: `docker info` +- Try `docker load -i result` instead of `docker load < result` + +**If flakes don't work:** +- Ensure experimental features are enabled in `~/.config/nix/nix.conf` +- Run `nix flake check` to validate flake syntax +- Make sure your flake is in a git repository -- [IPFS Gateway Checker](https://ipfs.github.io/public-gateway-checker/) -- [Gateway List](https://docs.ipfs.tech/concepts/ipfs-gateway/#gateway-providers) +**If cross-machine builds differ:** +- Check nixpkgs input is locked in `flake.lock` +- Verify both machines use same Nix version +- Ensure no `created = "now"` or timestamps in image builds
---- +
+🎯 Understanding Reproducibility + +**What makes a build reproducible?** +- ✅ Deterministic inputs (exact versions, hashes) +- ✅ Isolated environment (no system dependencies) +- ✅ No timestamps or random values +- ✅ Same compiler, same flags, same libraries +- ✅ Content-addressable storage + +**Why traditional tools fail:** +```bash +# Docker - timestamps in layers +docker build . # Different timestamp = different image hash + +# npm - lockfiles help but aren't perfect +npm install # Still uses local cache, system libraries + +# apt/yum - version drift +apt-get install nodejs # Gets different version next week +``` -**Good luck!** 🌐 +**How Nix succeeds:** +```bash +# Nix - pure, sandboxed, content-addressed +nix-build # Same inputs = bit-for-bit identical output + # Today, tomorrow, on any machine +``` + +**Real-world impact:** +- **CI/CD:** No more "works on my machine" +- **Security:** Audit exact dependency tree +- **Rollback:** Atomic updates with perfect rollbacks +- **Collaboration:** Everyone gets identical environment + +
+ +
+🌟 Advanced Concepts (Optional Reading) + +**Content-Addressable Store:** +- Every package has a unique hash based on its inputs +- `/nix/store/abc123...` where `abc123` = hash of inputs +- Same inputs = same hash = reuse existing build + +**Sandboxing:** +- Builds run in isolated namespaces +- No network access (except for fixed-output derivations) +- No access to `/home`, `/tmp`, or system paths +- Only declared dependencies are available + +**Lazy Evaluation:** +- Nix expressions are lazily evaluated +- Only builds what's actually needed +- Enables massive codebase (all of nixpkgs) without performance issues + +**Binary Cache:** +- cache.nixos.org provides pre-built binaries +- If your build matches a cached hash, download instead of rebuild +- Set up private caches for your team + +**Cross-Compilation:** +- Nix makes cross-compilation trivial +- `pkgs.pkgsCross.aarch64-multiplatform.hello` +- Same reproducibility guarantees across architectures -> **Remember:** Decentralized hosting trades some convenience for resilience and censorship resistance. Content-addressed storage ensures integrity - the same content always has the same identifier. +
diff --git a/labs/lab18/app_python/Dockerfile b/labs/lab18/app_python/Dockerfile new file mode 100644 index 0000000000..07605a7262 --- /dev/null +++ b/labs/lab18/app_python/Dockerfile @@ -0,0 +1,31 @@ +# Production-oriented image for a small Flask app. +# Pin a specific Python version for reproducible builds. +FROM python:3.13.1-slim + +# Python runtime defaults: no .pyc files, unbuffered logs (better for containers) +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 + +WORKDIR /app + +# Create a dedicated non-root user with numeric UID/GID. +RUN addgroup --system --gid 10001 app \ + && adduser --system --uid 10001 --ingroup app --no-create-home app \ + && mkdir -p /data /config \ + && chown -R 10001:10001 /app /data /config + +# Install dependencies first to leverage Docker layer caching. +COPY requirements.txt ./ +RUN pip install --no-cache-dir -r requirements.txt + +# Copy only the application code needed at runtime. +COPY app.py ./ + +# Drop privileges using a numeric user for Kubernetes runAsNonRoot validation. +USER 10001:10001 + +# Document the port (Flask defaults to 5000 in this repo) +EXPOSE 5000 + +# Start the application +CMD ["python", "app.py"] diff --git a/labs/lab18/app_python/README.md b/labs/lab18/app_python/README.md new file mode 100644 index 0000000000..920ec38dc4 --- /dev/null +++ b/labs/lab18/app_python/README.md @@ -0,0 +1,13 @@ +# Lab 18 Nix App + +This directory contains a copy of the Lab 1/2 Python DevOps Info Service and the Nix expressions required by Lab 18. + +Files: + +- `app.py` — Flask service copied from `app_python/`. +- `requirements.txt` — original pip dependency file for comparison. +- `Dockerfile` — original Lab 2 Dockerfile for comparison. +- `default.nix` — reproducible Nix build for the Python app. +- `docker.nix` — reproducible Docker image built with Nix `dockerTools`. + +Bonus flakes are intentionally not included. diff --git a/labs/lab18/app_python/app.py b/labs/lab18/app_python/app.py new file mode 100644 index 0000000000..12d111a9d5 --- /dev/null +++ b/labs/lab18/app_python/app.py @@ -0,0 +1,580 @@ +""" +DevOps Info Service +Main application module (Flask) + +Endpoints: +- GET / : service + system + runtime + request info +- GET /health : health check (for probes/monitoring) +- GET /metrics : Prometheus metrics endpoint +- GET /visits : current persisted visit counter value +""" + +from __future__ import annotations + +import json +import logging +import os +import platform +import socket +import sys +import tempfile +import threading +import time +from datetime import datetime, timezone +from pathlib import Path +from typing import Any, Dict + +from flask import Flask, Response, g, jsonify, request +from prometheus_client import CONTENT_TYPE_LATEST, Counter, Gauge, Histogram, generate_latest + +# ----------------------------------------------------------------------------- +# App & Config +# ----------------------------------------------------------------------------- + +app = Flask(__name__) + +HOST: str = os.getenv("HOST", "0.0.0.0") +PORT: int = int(os.getenv("PORT", "5000")) +DEBUG: bool = os.getenv("DEBUG", "False").strip().lower() == "true" + +SERVICE_NAME = os.getenv("SERVICE_NAME", "devops-info-service") +SERVICE_VERSION = os.getenv("SERVICE_VERSION", "1.1.0") +SERVICE_DESCRIPTION = os.getenv("SERVICE_DESCRIPTION", "DevOps course info service") +SERVICE_FRAMEWORK = "Flask" +APP_VARIANT = os.getenv("APP_VARIANT", "primary") +APP_MESSAGE = os.getenv("APP_MESSAGE", "running") +APP_ENV = os.getenv("APP_ENV", "dev") +LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO") +FEATURE_VISITS_ENDPOINT = os.getenv("FEATURE_VISITS_ENDPOINT", "true") +CONFIG_FILE_PATH = os.getenv("CONFIG_FILE_PATH", "/config/config.json") +VISITS_FILE = Path(os.getenv("VISITS_FILE", "/data/visits")) + +START_TIME_UTC = datetime.now(timezone.utc) +VISITS_LOCK = threading.Lock() + + +# ----------------------------------------------------------------------------- +# Prometheus metrics +# ----------------------------------------------------------------------------- + +HTTP_REQUESTS_TOTAL = Counter( + "http_requests_total", + "Total HTTP requests processed by the service.", + ["method", "endpoint", "status_code"], +) + +HTTP_REQUEST_DURATION_SECONDS = Histogram( + "http_request_duration_seconds", + "HTTP request duration in seconds.", + ["method", "endpoint"], +) + +HTTP_REQUESTS_IN_PROGRESS = Gauge( + "http_requests_in_progress", + "HTTP requests currently being processed.", +) + +DEVOPS_INFO_ENDPOINT_CALLS_TOTAL = Counter( + "devops_info_endpoint_calls_total", + "Total endpoint calls for the DevOps info service.", + ["endpoint"], +) + +DEVOPS_INFO_SYSTEM_COLLECTION_SECONDS = Histogram( + "devops_info_system_collection_seconds", + "Time spent collecting system information.", +) + +DEVOPS_INFO_UPTIME_SECONDS = Gauge( + "devops_info_uptime_seconds", + "Current service uptime in seconds.", +) + + +# ----------------------------------------------------------------------------- +# Logging +# ----------------------------------------------------------------------------- + + +def iso_utc_now_z() -> str: + """Return current UTC time in ISO format with 'Z' suffix.""" + return datetime.now(timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z") + + +class JSONFormatter(logging.Formatter): + """Small JSON formatter for container-friendly structured logs.""" + + EXTRA_FIELDS = ( + "event", + "service", + "version", + "method", + "path", + "status_code", + "client_ip", + "duration_ms", + "user_agent", + ) + + def format(self, record: logging.LogRecord) -> str: + payload: Dict[str, Any] = { + "timestamp": iso_utc_now_z(), + "level": record.levelname, + "logger": record.name, + "message": record.getMessage(), + } + + for field in self.EXTRA_FIELDS: + value = getattr(record, field, None) + if value is not None: + payload[field] = value + + if record.exc_info: + payload["exception"] = self.formatException(record.exc_info) + + return json.dumps(payload, ensure_ascii=False) + + + +def configure_logging() -> logging.Logger: + handler = logging.StreamHandler(sys.stdout) + handler.setFormatter(JSONFormatter()) + + root_logger = logging.getLogger() + root_logger.handlers.clear() + root_logger.addHandler(handler) + root_logger.setLevel(logging.DEBUG if DEBUG else logging.INFO) + + for logger_name in ("werkzeug", "gunicorn.error", "gunicorn.access"): + current_logger = logging.getLogger(logger_name) + current_logger.handlers.clear() + current_logger.propagate = True + current_logger.setLevel(logging.DEBUG if DEBUG else logging.INFO) + + return logging.getLogger(SERVICE_NAME) + + +logger = configure_logging() + + +# ----------------------------------------------------------------------------- +# Request hooks +# ----------------------------------------------------------------------------- + + +def get_client_ip() -> str: + """ + Best-effort client IP resolution. + Prefers X-Forwarded-For (common behind reverse proxies). + """ + forwarded_for = request.headers.get("X-Forwarded-For", "") + if forwarded_for: + # "client, proxy1, proxy2" + return forwarded_for.split(",")[0].strip() + return request.remote_addr or "unknown" + + + +def normalize_endpoint() -> str: + """ + Keep endpoint labels low-cardinality for Prometheus. + Uses the Flask route template when available and groups unknown paths. + """ + if request.url_rule and request.url_rule.rule: + return request.url_rule.rule + + if request.path == "/": + return "/" + + return "unmatched" + + +@app.before_request +def log_request_started() -> None: + g.request_started_at = time.perf_counter() + g.normalized_endpoint = normalize_endpoint() + g.skip_http_metrics = request.path == "/metrics" + g.active_request_metric_registered = False + + if not g.skip_http_metrics: + HTTP_REQUESTS_IN_PROGRESS.inc() + g.active_request_metric_registered = True + + logger.debug( + "request_started", + extra={ + "event": "request_started", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + "method": request.method, + "path": request.path, + "client_ip": get_client_ip(), + "user_agent": request.headers.get("User-Agent", "unknown"), + }, + ) + + +@app.teardown_request +def track_request_finished(_error: Exception | None) -> None: + if getattr(g, "active_request_metric_registered", False): + HTTP_REQUESTS_IN_PROGRESS.dec() + g.active_request_metric_registered = False + + +@app.after_request +def add_headers(response): + endpoint = getattr(g, "normalized_endpoint", normalize_endpoint()) + duration_seconds = time.perf_counter() - getattr(g, "request_started_at", time.perf_counter()) + duration_ms = round(duration_seconds * 1000, 2) + + if not getattr(g, "skip_http_metrics", False): + HTTP_REQUESTS_TOTAL.labels( + method=request.method, + endpoint=endpoint, + status_code=str(response.status_code), + ).inc() + HTTP_REQUEST_DURATION_SECONDS.labels( + method=request.method, + endpoint=endpoint, + ).observe(duration_seconds) + DEVOPS_INFO_ENDPOINT_CALLS_TOTAL.labels(endpoint=endpoint).inc() + + DEVOPS_INFO_UPTIME_SECONDS.set(get_uptime()["seconds"]) + + if response.mimetype == "application/json": + response.headers["Content-Type"] = "application/json; charset=utf-8" + + logger.info( + "request_completed", + extra={ + "event": "request_completed", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + "method": request.method, + "path": request.path, + "status_code": response.status_code, + "client_ip": get_client_ip(), + "duration_ms": duration_ms, + "user_agent": request.headers.get("User-Agent", "unknown"), + }, + ) + return response + + +# ----------------------------------------------------------------------------- +# Helpers +# ----------------------------------------------------------------------------- + + +def get_uptime() -> Dict[str, Any]: + """Calculate service uptime since START_TIME_UTC.""" + delta = datetime.now(timezone.utc) - START_TIME_UTC + seconds = int(delta.total_seconds()) + hours = seconds // 3600 + minutes = (seconds % 3600) // 60 + + hours_part = f"{hours} hour" + ("" if hours == 1 else "s") + minutes_part = f"{minutes} minute" + ("" if minutes == 1 else "s") + + return { + "seconds": seconds, + "human": f"{hours_part}, {minutes_part}", + } + + + +def get_system_info() -> Dict[str, Any]: + """Collect system information using Python standard library.""" + started_at = time.perf_counter() + try: + return { + "hostname": socket.gethostname(), + "platform": platform.system(), + "platform_version": platform.platform(), + "architecture": platform.machine(), + "cpu_count": os.cpu_count() or 0, + "python_version": platform.python_version(), + } + finally: + DEVOPS_INFO_SYSTEM_COLLECTION_SECONDS.observe(time.perf_counter() - started_at) + + + +def ensure_visits_storage() -> int: + """Create the visits file if missing and return the initial counter value.""" + VISITS_FILE.parent.mkdir(parents=True, exist_ok=True) + if not VISITS_FILE.exists(): + write_visits_count(0) + return 0 + return read_visits_count() + + + +def read_visits_count() -> int: + """Read the persisted visit counter from disk.""" + try: + content = VISITS_FILE.read_text(encoding="utf-8").strip() + except FileNotFoundError: + return 0 + + if not content: + return 0 + + try: + return int(content) + except ValueError: + logger.warning( + "invalid_visits_file_content", + extra={ + "event": "invalid_visits_file_content", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + }, + ) + return 0 + + + +def write_visits_count(value: int) -> None: + """Persist the visit counter using an atomic replace operation.""" + VISITS_FILE.parent.mkdir(parents=True, exist_ok=True) + with tempfile.NamedTemporaryFile("w", encoding="utf-8", dir=str(VISITS_FILE.parent), delete=False) as handle: + handle.write(str(value)) + handle.flush() + os.fsync(handle.fileno()) + temp_path = Path(handle.name) + + os.replace(temp_path, VISITS_FILE) + + + +def increment_visits_count() -> int: + """Read, increment, and persist the visit counter with in-process locking.""" + with VISITS_LOCK: + current_value = read_visits_count() + next_value = current_value + 1 + write_visits_count(next_value) + return next_value + + + +def get_runtime_config() -> Dict[str, Any]: + """Return file-based and environment-based configuration details.""" + config_payload: Dict[str, Any] = { + "app_env": APP_ENV, + "log_level": LOG_LEVEL, + "feature_visits_endpoint": FEATURE_VISITS_ENDPOINT, + "config_file_path": CONFIG_FILE_PATH, + "config_file_loaded": False, + } + + config_path = Path(CONFIG_FILE_PATH) + if not config_path.exists(): + config_payload["config_file_error"] = "config file not found" + return config_payload + + try: + config_payload["config_file"] = json.loads(config_path.read_text(encoding="utf-8")) + config_payload["config_file_loaded"] = True + except (OSError, json.JSONDecodeError) as exc: + config_payload["config_file_error"] = str(exc) + + return config_payload + + + +def build_endpoints() -> list[Dict[str, str]]: + return [ + {"path": "/", "method": "GET", "description": "Service information and visit counter increment"}, + {"path": "/visits", "method": "GET", "description": "Current persisted visit count"}, + {"path": "/health", "method": "GET", "description": "Liveness health check"}, + {"path": "/ready", "method": "GET", "description": "Readiness health check"}, + {"path": "/metrics", "method": "GET", "description": "Prometheus metrics"}, + ] + + +INITIAL_VISITS = ensure_visits_storage() + + +# ----------------------------------------------------------------------------- +# Routes +# ----------------------------------------------------------------------------- + +@app.get("/") +def index(): + """Main endpoint - service and system information.""" + uptime = get_uptime() + current_visits = increment_visits_count() + + payload: Dict[str, Any] = { + "service": { + "name": SERVICE_NAME, + "version": SERVICE_VERSION, + "description": SERVICE_DESCRIPTION, + "framework": SERVICE_FRAMEWORK, + "variant": APP_VARIANT, + "message": APP_MESSAGE, + }, + "system": get_system_info(), + "runtime": { + "uptime_seconds": uptime["seconds"], + "uptime_human": uptime["human"], + "current_time": iso_utc_now_z(), + "timezone": "UTC", + }, + "request": { + "client_ip": get_client_ip(), + "user_agent": request.headers.get("User-Agent", "unknown"), + "method": request.method, + "path": request.path, + }, + "visits": { + "count": current_visits, + "file": str(VISITS_FILE), + }, + "configuration": get_runtime_config(), + "endpoints": build_endpoints(), + } + + return jsonify(payload), 200 + + +@app.get("/visits") +def visits(): + """Return the current persisted visit counter without incrementing it.""" + with VISITS_LOCK: + current_visits = read_visits_count() + + return jsonify( + { + "visits": current_visits, + "file": str(VISITS_FILE), + "timestamp": iso_utc_now_z(), + } + ), 200 + + +@app.get("/health") +def health(): + """Health check endpoint (for probes/monitoring).""" + uptime = get_uptime() + return jsonify( + { + "status": "healthy", + "timestamp": iso_utc_now_z(), + "uptime_seconds": uptime["seconds"], + "variant": APP_VARIANT, + } + ), 200 + + +@app.get("/ready") +def ready(): + """Readiness endpoint used by Kubernetes readiness probes.""" + uptime = get_uptime() + return jsonify( + { + "status": "ready", + "timestamp": iso_utc_now_z(), + "uptime_seconds": uptime["seconds"], + "variant": APP_VARIANT, + "message": APP_MESSAGE, + } + ), 200 + + +@app.get("/metrics") +def metrics() -> Response: + """Expose Prometheus metrics for scraping.""" + DEVOPS_INFO_UPTIME_SECONDS.set(get_uptime()["seconds"]) + return Response(generate_latest(), content_type=CONTENT_TYPE_LATEST) + + +# ----------------------------------------------------------------------------- +# Error Handlers +# ----------------------------------------------------------------------------- + +@app.errorhandler(404) +def not_found(_error): + logger.warning( + "endpoint_not_found", + extra={ + "event": "endpoint_not_found", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + "method": request.method, + "path": request.path, + "status_code": 404, + "client_ip": get_client_ip(), + }, + ) + return ( + jsonify( + { + "error": "Not Found", + "message": "Endpoint does not exist", + "timestamp": iso_utc_now_z(), + } + ), + 404, + ) + + +@app.errorhandler(500) +def internal_error(_error): + logger.exception( + "unhandled_error", + extra={ + "event": "unhandled_error", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + "method": request.method, + "path": request.path, + "status_code": 500, + "client_ip": get_client_ip(), + }, + ) + return ( + jsonify( + { + "error": "Internal Server Error", + "message": "An unexpected error occurred", + "timestamp": iso_utc_now_z(), + } + ), + 500, + ) + + +# ----------------------------------------------------------------------------- +# Entrypoint +# ----------------------------------------------------------------------------- + + +def main() -> None: + logger.info( + "service_starting", + extra={ + "event": "service_starting", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + }, + ) + logger.info( + "runtime_configuration host=%s port=%s debug=%s visits_file=%s initial_visits=%s", + HOST, + PORT, + DEBUG, + VISITS_FILE, + INITIAL_VISITS, + extra={ + "event": "runtime_configuration", + "service": SERVICE_NAME, + "version": SERVICE_VERSION, + }, + ) + app.run(host=HOST, port=PORT, debug=DEBUG) + + +if __name__ == "__main__": + main() diff --git a/labs/lab18/app_python/default.nix b/labs/lab18/app_python/default.nix new file mode 100644 index 0000000000..774d092f08 --- /dev/null +++ b/labs/lab18/app_python/default.nix @@ -0,0 +1,40 @@ +{ pkgs ? import {} }: + +let + pythonEnv = pkgs.python3.withPackages (ps: with ps; [ + flask + prometheus-client + ]); +in +pkgs.stdenv.mkDerivation { + pname = "devops-info-service"; + version = "1.0.0"; + src = ./.; + + nativeBuildInputs = [ pkgs.makeWrapper ]; + + installPhase = '' + runHook preInstall + + mkdir -p $out/share/devops-info-service + cp app.py $out/share/devops-info-service/app.py + + mkdir -p $out/bin + makeWrapper ${pythonEnv}/bin/python $out/bin/devops-info-service \ + --add-flags $out/share/devops-info-service/app.py \ + --set HOST 0.0.0.0 \ + --set PORT 5000 \ + --set SERVICE_NAME devops-info-service \ + --set SERVICE_VERSION 1.0.0-nix \ + --set APP_ENV lab18-nix \ + --set VISITS_FILE /tmp/devops-info-service-visits + + runHook postInstall + ''; + + meta = with pkgs.lib; { + description = "DevOps Info Service built reproducibly with Nix"; + mainProgram = "devops-info-service"; + platforms = platforms.linux; + }; +} diff --git a/labs/lab18/app_python/docker.nix b/labs/lab18/app_python/docker.nix new file mode 100644 index 0000000000..5128256d74 --- /dev/null +++ b/labs/lab18/app_python/docker.nix @@ -0,0 +1,34 @@ +{ pkgs ? import {} }: + +let + app = import ./default.nix { inherit pkgs; }; +in +pkgs.dockerTools.buildLayeredImage { + name = "devops-info-service-nix"; + tag = "1.0.0"; + + contents = [ app ]; + + # Fixed timestamp is required for reproducible image tarballs. + created = "1970-01-01T00:00:01Z"; + + extraCommands = '' + mkdir -p tmp + chmod 1777 tmp + ''; + + config = { + Cmd = [ "${app}/bin/devops-info-service" ]; + Env = [ + "HOST=0.0.0.0" + "PORT=5000" + "SERVICE_NAME=devops-info-service" + "SERVICE_VERSION=1.0.0-nix-docker" + "APP_ENV=lab18-nix-docker" + "VISITS_FILE=/tmp/devops-info-service-visits" + ]; + ExposedPorts = { + "5000/tcp" = {}; + }; + }; +} diff --git a/labs/lab18/app_python/requirements.txt b/labs/lab18/app_python/requirements.txt new file mode 100644 index 0000000000..46c776bf8d --- /dev/null +++ b/labs/lab18/app_python/requirements.txt @@ -0,0 +1,2 @@ +Flask==3.1.0 +prometheus-client==0.23.1 diff --git a/labs/lab18/evidence/.gitkeep b/labs/lab18/evidence/.gitkeep new file mode 100644 index 0000000000..e69de29bb2 diff --git a/labs/lab18/evidence/01-nix-version.txt b/labs/lab18/evidence/01-nix-version.txt new file mode 100644 index 0000000000..3c21b8b95e --- /dev/null +++ b/labs/lab18/evidence/01-nix-version.txt @@ -0,0 +1 @@ +nix (Determinate Nix 3.20.0) 2.34.6 diff --git a/labs/lab18/evidence/02-nix-hello.txt b/labs/lab18/evidence/02-nix-hello.txt new file mode 100644 index 0000000000..af5626b4a1 --- /dev/null +++ b/labs/lab18/evidence/02-nix-hello.txt @@ -0,0 +1 @@ +Hello, world! diff --git a/labs/lab18/evidence/03-nix-build-first.txt b/labs/lab18/evidence/03-nix-build-first.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/03-nix-build-first.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/04-store-path-first.txt b/labs/lab18/evidence/04-store-path-first.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/04-store-path-first.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/05-nix-output-hash-first.txt b/labs/lab18/evidence/05-nix-output-hash-first.txt new file mode 100644 index 0000000000..c5ac605261 --- /dev/null +++ b/labs/lab18/evidence/05-nix-output-hash-first.txt @@ -0,0 +1 @@ +dd2bd2abba01a00297ed14ad60f29a24b67ce457b1211cdb52dc3b5c2eb0a0fd diff --git a/labs/lab18/evidence/06-nix-build-second.txt b/labs/lab18/evidence/06-nix-build-second.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/06-nix-build-second.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/07-store-path-second.txt b/labs/lab18/evidence/07-store-path-second.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/07-store-path-second.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/08-nix-output-hash-second.txt b/labs/lab18/evidence/08-nix-output-hash-second.txt new file mode 100644 index 0000000000..c5ac605261 --- /dev/null +++ b/labs/lab18/evidence/08-nix-output-hash-second.txt @@ -0,0 +1 @@ +dd2bd2abba01a00297ed14ad60f29a24b67ce457b1211cdb52dc3b5c2eb0a0fd diff --git a/labs/lab18/evidence/09-store-path-before-delete.txt b/labs/lab18/evidence/09-store-path-before-delete.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/09-store-path-before-delete.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/10-nix-store-delete.txt b/labs/lab18/evidence/10-nix-store-delete.txt new file mode 100644 index 0000000000..9d5659c998 --- /dev/null +++ b/labs/lab18/evidence/10-nix-store-delete.txt @@ -0,0 +1 @@ +1 store paths deleted, 17.2 KiB freed diff --git a/labs/lab18/evidence/11-nix-build-after-delete.txt b/labs/lab18/evidence/11-nix-build-after-delete.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/11-nix-build-after-delete.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/12-store-path-after-delete.txt b/labs/lab18/evidence/12-store-path-after-delete.txt new file mode 100644 index 0000000000..13034d0e30 --- /dev/null +++ b/labs/lab18/evidence/12-store-path-after-delete.txt @@ -0,0 +1 @@ +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 diff --git a/labs/lab18/evidence/13-nix-output-hash-after-delete.txt b/labs/lab18/evidence/13-nix-output-hash-after-delete.txt new file mode 100644 index 0000000000..c5ac605261 --- /dev/null +++ b/labs/lab18/evidence/13-nix-output-hash-after-delete.txt @@ -0,0 +1 @@ +dd2bd2abba01a00297ed14ad60f29a24b67ce457b1211cdb52dc3b5c2eb0a0fd diff --git a/labs/lab18/evidence/14-pip-comparison.txt b/labs/lab18/evidence/14-pip-comparison.txt new file mode 100644 index 0000000000..4417980fce --- /dev/null +++ b/labs/lab18/evidence/14-pip-comparison.txt @@ -0,0 +1,17 @@ +--- freeze1.txt --- +Flask==3.1.3 +Jinja2==3.1.6 +MarkupSafe==3.0.3 +Werkzeug==3.1.8 +blinker==1.9.0 +click==8.3.3 +itsdangerous==2.2.0 +--- freeze2.txt --- +Flask==3.1.3 +Jinja2==3.1.6 +MarkupSafe==3.0.3 +Werkzeug==3.1.8 +blinker==1.9.0 +click==8.3.3 +itsdangerous==2.2.0 +--- diff freeze1 freeze2 --- diff --git a/labs/lab18/evidence/15-lab2-docker-build-v1.txt b/labs/lab18/evidence/15-lab2-docker-build-v1.txt new file mode 100644 index 0000000000..e69de29bb2 diff --git a/labs/lab18/evidence/16-lab2-created-v1.txt b/labs/lab18/evidence/16-lab2-created-v1.txt new file mode 100644 index 0000000000..bb19365162 --- /dev/null +++ b/labs/lab18/evidence/16-lab2-created-v1.txt @@ -0,0 +1 @@ +2026-04-16T12:49:26.205010483Z diff --git a/labs/lab18/evidence/17-lab2-docker-build-v2.txt b/labs/lab18/evidence/17-lab2-docker-build-v2.txt new file mode 100644 index 0000000000..e69de29bb2 diff --git a/labs/lab18/evidence/18-lab2-created-v2.txt b/labs/lab18/evidence/18-lab2-created-v2.txt new file mode 100644 index 0000000000..bb19365162 --- /dev/null +++ b/labs/lab18/evidence/18-lab2-created-v2.txt @@ -0,0 +1 @@ +2026-04-16T12:49:26.205010483Z diff --git a/labs/lab18/evidence/19-lab2-save-hash-v1.txt b/labs/lab18/evidence/19-lab2-save-hash-v1.txt new file mode 100644 index 0000000000..580da159d8 --- /dev/null +++ b/labs/lab18/evidence/19-lab2-save-hash-v1.txt @@ -0,0 +1 @@ +5d0bd401bee017ef30287e8411a576af3bc1a5f0545cadae4f6f705916977ae6 - diff --git a/labs/lab18/evidence/20-lab2-save-hash-v2.txt b/labs/lab18/evidence/20-lab2-save-hash-v2.txt new file mode 100644 index 0000000000..8332c504ce --- /dev/null +++ b/labs/lab18/evidence/20-lab2-save-hash-v2.txt @@ -0,0 +1 @@ +81572d01b29834d3ae5db13a7b98c4f545b9ae0a4f7930d49b1372cd34485797 - diff --git a/labs/lab18/evidence/21-nix-docker-build-first.txt b/labs/lab18/evidence/21-nix-docker-build-first.txt new file mode 100644 index 0000000000..fd1c043212 --- /dev/null +++ b/labs/lab18/evidence/21-nix-docker-build-first.txt @@ -0,0 +1 @@ +/nix/store/a44ki7ymm2wdabqyxz0p6lmsvly6aps6-devops-info-service-nix.tar.gz diff --git a/labs/lab18/evidence/22-nix-docker-hash-first.txt b/labs/lab18/evidence/22-nix-docker-hash-first.txt new file mode 100644 index 0000000000..bb93ed2b80 --- /dev/null +++ b/labs/lab18/evidence/22-nix-docker-hash-first.txt @@ -0,0 +1 @@ +aa3638c60f34d8cfda3136c3e29a797f4edd5ee4b215b9603cfc3e1a62450eb5 result diff --git a/labs/lab18/evidence/23-nix-docker-load.txt b/labs/lab18/evidence/23-nix-docker-load.txt new file mode 100644 index 0000000000..31b7bee292 --- /dev/null +++ b/labs/lab18/evidence/23-nix-docker-load.txt @@ -0,0 +1 @@ +Loaded image: devops-info-service-nix:1.0.0 diff --git a/labs/lab18/evidence/24-nix-docker-build-second.txt b/labs/lab18/evidence/24-nix-docker-build-second.txt new file mode 100644 index 0000000000..fd1c043212 --- /dev/null +++ b/labs/lab18/evidence/24-nix-docker-build-second.txt @@ -0,0 +1 @@ +/nix/store/a44ki7ymm2wdabqyxz0p6lmsvly6aps6-devops-info-service-nix.tar.gz diff --git a/labs/lab18/evidence/25-nix-docker-hash-second.txt b/labs/lab18/evidence/25-nix-docker-hash-second.txt new file mode 100644 index 0000000000..bb93ed2b80 --- /dev/null +++ b/labs/lab18/evidence/25-nix-docker-hash-second.txt @@ -0,0 +1 @@ +aa3638c60f34d8cfda3136c3e29a797f4edd5ee4b215b9603cfc3e1a62450eb5 result diff --git a/labs/lab18/evidence/26-lab2-container-id.txt b/labs/lab18/evidence/26-lab2-container-id.txt new file mode 100644 index 0000000000..42c746855c --- /dev/null +++ b/labs/lab18/evidence/26-lab2-container-id.txt @@ -0,0 +1 @@ +892f8c44796102f904ae1c7c91e60ba4536d13d22b09c1b97276e3fd624f2b6a diff --git a/labs/lab18/evidence/27-nix-container-id.txt b/labs/lab18/evidence/27-nix-container-id.txt new file mode 100644 index 0000000000..7d06152240 --- /dev/null +++ b/labs/lab18/evidence/27-nix-container-id.txt @@ -0,0 +1 @@ +9a7ceb173dd2aa4b14c61e2fdd28f83ea2aec505f8b1c5398245d65870e83ae9 diff --git a/labs/lab18/evidence/28-container-health-checks.txt b/labs/lab18/evidence/28-container-health-checks.txt new file mode 100644 index 0000000000..e3972fd262 --- /dev/null +++ b/labs/lab18/evidence/28-container-health-checks.txt @@ -0,0 +1,10 @@ +--- docker ps --- +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +9a7ceb173dd2 devops-info-service-nix:1.0.0 "/nix/store/06c4hlva…" 9 seconds ago Up 8 seconds 0.0.0.0:5001->5000/tcp, [::]:5001->5000/tcp nix-container +892f8c447961 lab2-app:v1 "python app.py" 10 seconds ago Up 9 seconds 0.0.0.0:5000->5000/tcp, [::]:5000->5000/tcp lab2-container +--- lab2 container health --- +{"status":"healthy","timestamp":"2026-05-14T20:17:35.129Z","uptime_seconds":8,"variant":"primary"} + +--- nix container health --- +{"status":"healthy","timestamp":"2026-05-14T20:17:35.144Z","uptime_seconds":8,"variant":"primary"} + diff --git a/labs/lab18/evidence/29-image-size-comparison.txt b/labs/lab18/evidence/29-image-size-comparison.txt new file mode 100644 index 0000000000..c8a10ebe8f --- /dev/null +++ b/labs/lab18/evidence/29-image-size-comparison.txt @@ -0,0 +1,3 @@ +lab2-app v2 3d1599933b32 4 weeks ago 189MB +lab2-app v1 8fc4113f4485 4 weeks ago 189MB +devops-info-service-nix 1.0.0 d5f5ad80fafd 56 years ago 396MB diff --git a/labs/lab18/evidence/30-lab2-docker-history.txt b/labs/lab18/evidence/30-lab2-docker-history.txt new file mode 100644 index 0000000000..2ea54d5187 --- /dev/null +++ b/labs/lab18/evidence/30-lab2-docker-history.txt @@ -0,0 +1,19 @@ +IMAGE CREATED CREATED BY SIZE COMMENT +8fc4113f4485 4 weeks ago CMD ["python" "app.py"] 0B buildkit.dockerfile.v0 + 4 weeks ago EXPOSE &{[{{28 0} {28 0}}] 0xc001a908c0} 0B buildkit.dockerfile.v0 + 4 weeks ago USER 10001:10001 0B buildkit.dockerfile.v0 + 4 weeks ago COPY app.py ./ # buildkit 28.7kB buildkit.dockerfile.v0 + 4 weeks ago RUN /bin/sh -c pip install --no-cache-dir -r… 6.16MB buildkit.dockerfile.v0 + 4 weeks ago COPY requirements.txt ./ # buildkit 12.3kB buildkit.dockerfile.v0 + 4 weeks ago RUN /bin/sh -c addgroup --system --gid 10001… 57.3kB buildkit.dockerfile.v0 + 3 months ago WORKDIR /app 8.19kB buildkit.dockerfile.v0 + 3 months ago ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFER… 0B buildkit.dockerfile.v0 + 16 months ago CMD ["python3"] 0B buildkit.dockerfile.v0 + 16 months ago RUN /bin/sh -c set -eux; for src in idle3 p… 16.4kB buildkit.dockerfile.v0 + 16 months ago RUN /bin/sh -c set -eux; savedAptMark="$(a… 41.3MB buildkit.dockerfile.v0 + 16 months ago ENV PYTHON_SHA256=9cf9427bee9e2242e3877dd0f6… 0B buildkit.dockerfile.v0 + 16 months ago ENV PYTHON_VERSION=3.13.1 0B buildkit.dockerfile.v0 + 16 months ago ENV GPG_KEY=7169605F62C751356D054A26A821E680… 0B buildkit.dockerfile.v0 + 16 months ago RUN /bin/sh -c set -eux; apt-get update; a… 10.4MB buildkit.dockerfile.v0 + 16 months ago ENV PATH=/usr/local/bin:/usr/local/sbin:/usr… 0B buildkit.dockerfile.v0 + 16 months ago # debian.sh --arch 'amd64' out/ 'bookworm' '… 85.2MB debuerreotype 0.15 diff --git a/labs/lab18/evidence/31-nix-docker-history.txt b/labs/lab18/evidence/31-nix-docker-history.txt new file mode 100644 index 0000000000..8b0e0ab4b8 --- /dev/null +++ b/labs/lab18/evidence/31-nix-docker-history.txt @@ -0,0 +1,34 @@ +IMAGE CREATED CREATED BY SIZE COMMENT +d5f5ad80fafd N/A 28.7kB store paths: ['/nix/store/cf8dk5gzsxx6r9xkx1ckvbqz67w1z0gz-devops-info-service-nix-customisation-layer'] + N/A 53.2kB store paths: ['/nix/store/06c4hlvac4h4l4jl4zsn3cgywbsqpna6-devops-info-service-1.0.0'] + N/A 1.22MB store paths: ['/nix/store/wij96d7izljw38kksg0q384ygi8hfvkk-python3-3.12.8-env'] + N/A 856kB store paths: ['/nix/store/a4v5qavdkzip0bnwhmnq2a7m62hpnnpc-python3.12-prometheus-client-0.21.0'] + N/A 1.33MB store paths: ['/nix/store/ijc606v1g5vhqxrjk4qmj787kymp6sal-python3.12-flask-3.0.3'] + N/A 2.99MB store paths: ['/nix/store/jcf93x7sly7l76hyaqwramfyydnbvsf4-python3.12-werkzeug-3.0.6'] + N/A 2.06MB store paths: ['/nix/store/jvrzvrbnxdk6wp1qy9k1iyjhgbzw3jv2-python3.12-jinja2-3.1.5'] + N/A 246kB store paths: ['/nix/store/0vrargkskpn2m47ka7cfkc1bgq35achi-python3.12-itsdangerous-2.2.0'] + N/A 1.39MB store paths: ['/nix/store/k1qzmvyvarz0pgjaz06wx0y5vsgwbhbw-python3.12-click-8.1.7'] + N/A 172kB store paths: ['/nix/store/hb6zkd7hqgqsjsc2dswifgcx8ycc43ag-python3.12-blinker-1.8.2'] + N/A 176kB store paths: ['/nix/store/s03icbxsi62xvd9bigwbgqlgbr5zhkdp-python3.12-markupsafe-3.0.2'] + N/A 121MB store paths: ['/nix/store/dksjvr69ckglyw1k2ss1qgshhcix73p8-python3-3.12.8'] + N/A 1.1MB store paths: ['/nix/store/izpczxh0wcm3ra6z0073zf9j0mv2wfl4-xz-5.6.3'] + N/A 5.33MB store paths: ['/nix/store/v87awkhzf3nr7nc5i4gg77xzqv4bqjy3-tzdata-2025b'] + N/A 1.61MB store paths: ['/nix/store/v9smapvfv1z340qs3p7xbw6zb6zplfcf-sqlite-3.46.1'] + N/A 504kB store paths: ['/nix/store/vb3dx18nky7cq63br7x2mi86isli529w-readline-8.2p13'] + N/A 8.06MB store paths: ['/nix/store/qzn96phpnb6c56mlqa1424hfgf5hp67s-openssl-3.3.3'] + N/A 266kB store paths: ['/nix/store/c25k325zh2b9g8s68b7ixbjfh3a916cb-mpdecimal-4.0.0'] + N/A 164kB store paths: ['/nix/store/n4gd4rqkr0p2rkdhklvbx1rnx78m6dkj-mailcap-2.1.54'] + N/A 172kB store paths: ['/nix/store/iissy6zslzyb85rzjgq4waag9dixvv6s-libxcrypt-4.4.36'] + N/A 98.3kB store paths: ['/nix/store/fm7yigp87wq0p58x92iynwscdmspzkrb-libffi-3.4.6'] + N/A 643kB store paths: ['/nix/store/jn8gi3mbjm6b2khxcbm3vf2c1h5wpv17-gdbm-1.24-lib'] + N/A 328kB store paths: ['/nix/store/h08i7wrlqmd48lnaimaz28pny9i8vmr8-expat-2.7.1'] + N/A 106kB store paths: ['/nix/store/vrqss3954zk1c52mda3xf1rv7wc5ygba-bzip2-1.0.8'] + N/A 9.21MB store paths: ['/nix/store/hh698a2nnpqr47lh52n26wi8fiah3hid-gcc-13.3.0-lib'] + N/A 1.73MB store paths: ['/nix/store/mjhcjikhxps97mq5z54j4gjjfzgmsir5-bash-5.2p37'] + N/A 184kB store paths: ['/nix/store/mkhhjfg2isjbfx87dz191bzpnwx1bbr9-gcc-13.3.0-libgcc'] + N/A 164kB store paths: ['/nix/store/b6mjyiadysqlh7nps52faznnqmp32604-zlib-1.3.1'] + N/A 8.7MB store paths: ['/nix/store/cn67k729khgnd9i1j7gbyh6lpzz11ci5-ncurses-6.4.20221231'] + N/A 31.7MB store paths: ['/nix/store/5m9amsvvh2z8sl7jrnc87hzy21glw6k1-glibc-2.40-66'] + N/A 184kB store paths: ['/nix/store/y4d9iir0yqmrcswaqfi368d8m1rkv14s-xgcc-13.3.0-libgcc'] + N/A 639kB store paths: ['/nix/store/c47b963idja6h1d8n91pf28v2jcq96kp-libidn2-2.3.7'] + N/A 1.88MB store paths: ['/nix/store/2745pvn6cv32yn9gp2rlqiqhqgs01pb5-libunistring-1.2'] diff --git a/labs/lab18/render-submission18.sh b/labs/lab18/render-submission18.sh new file mode 100644 index 0000000000..3c46b25fe7 --- /dev/null +++ b/labs/lab18/render-submission18.sh @@ -0,0 +1,312 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +EVIDENCE_DIR="$ROOT_DIR/labs/lab18/evidence" +OUT="$ROOT_DIR/labs/submission18.md" + +read_evidence() { + local file="$1" + local path="$EVIDENCE_DIR/$file" + if [[ -s "$path" ]]; then + cat "$path" + else + echo "PENDING: run labs/lab18/run-task1.sh and labs/lab18/run-task2.sh, then regenerate this file." + fi +} + +cat > "$OUT" <-- +\`\`\` + +For example: + +\`\`\`text +/nix/store/-devops-info-service-1.0.0 +\`\`\` + +The hash is derived from build inputs. If the source code, dependency versions, or build instructions change, the hash changes and Nix creates a new output path. + +### Screenshot Evidence + +- [Nix app running](lab18/screenshots/01-nix-app-running.png) +- [Nix build reproducibility](lab18/screenshots/03-nix-build-reproducibility.png) + +### Reflection — How Nix Would Have Helped in Lab 1 + +Nix would have made the Lab 1 setup more reproducible from the beginning. Instead of relying on the local Python installation, virtual environment state, and pip behavior, the application could have been built with a declared Python interpreter and dependency set. New machines and CI runners would use the same build inputs and produce the same result, reducing “works on my machine” problems. + +--- + +## Task 2 — Reproducible Docker Images with Nix + +### Traditional Lab 2 Dockerfile + +\`\`\`dockerfile +$(cat "$ROOT_DIR/app_python/Dockerfile") +\`\`\` + +### Nix dockerTools Image Definition + +\`\`\`nix +$(cat "$ROOT_DIR/labs/lab18/app_python/docker.nix") +\`\`\` + +### Explanation of \`docker.nix\` + +| Field | Purpose | +|---|---| +| \`app = import ./default.nix\` | Reuses the reproducible app derivation from Task 1. | +| \`dockerTools.buildLayeredImage\` | Builds a Docker-compatible image from Nix store paths. | +| \`name\` and \`tag\` | Define the image name \`devops-info-service-nix:1.0.0\`. | +| \`contents = [ app ]\` | Adds the app and its closure to the image. | +| \`created = "1970-01-01T00:00:01Z"\` | Uses a fixed timestamp to avoid timestamp-based nondeterminism. | +| \`extraCommands\` | Creates a writable \`/tmp\` directory. | +| \`config.Cmd\` | Starts the Nix-built application. | +| \`ExposedPorts\` | Documents the app port \`5000/tcp\`. | + +### Traditional Dockerfile Reproducibility Test + +Traditional image hash from first \`docker save\`: + +\`\`\`text +$(read_evidence 19-lab2-save-hash-v1.txt) +\`\`\` + +Traditional image hash from second \`docker save\`: + +\`\`\`text +$(read_evidence 20-lab2-save-hash-v2.txt) +\`\`\` + +The traditional Docker image hashes are expected to differ because Docker image metadata and layer creation timestamps can vary between builds, even when the Dockerfile and source files are unchanged. + +### Nix Docker Image Reproducibility Test + +Nix dockerTools image hash from first build: + +\`\`\`text +$(read_evidence 22-nix-docker-hash-first.txt) +\`\`\` + +Nix dockerTools image hash from second build: + +\`\`\`text +$(read_evidence 25-nix-docker-hash-second.txt) +\`\`\` + +The Nix image hashes are expected to be identical because the image is built from content-addressed Nix store paths and uses a fixed creation timestamp. + +### Side-by-Side Runtime Test + +\`\`\`text +$(read_evidence 28-container-health-checks.txt) +\`\`\` + +The traditional Dockerfile-based container and the Nix dockerTools container both serve the same application and return healthy responses. + +### Image Size Comparison + +\`\`\`text +$(read_evidence 29-image-size-comparison.txt) +\`\`\` + +### Docker History — Traditional Image + +\`\`\`text +$(read_evidence 30-lab2-docker-history.txt) +\`\`\` + +### Docker History — Nix Image + +\`\`\`text +$(read_evidence 31-nix-docker-history.txt) +\`\`\` + +### Comparison Table — Lab 2 Dockerfile vs Lab 18 Nix dockerTools + +| Aspect | Lab 2 Dockerfile | Lab 18 Nix dockerTools | +|---|---|---| +| Base image | Uses \`python:3.13.1-slim\` | No mutable base image tag required | +| Dependency installation | Runs \`pip install\` during Docker build | Uses Nix-built Python environment | +| Timestamp behavior | Build metadata can change per build | Fixed \`created\` timestamp | +| Reproducibility | Approximate; image hashes can differ | Bit-for-bit reproducible image tarball | +| Caching | Docker layer cache | Nix content-addressed store and binary cache | +| Dependency closure | Implicit in image layers | Explicit Nix closure | +| Auditability | Depends on image and pip state | Store paths identify exact inputs | + +### Why Traditional Dockerfiles Cannot Guarantee Bit-for-Bit Reproducibility + +Traditional Dockerfiles often rely on mutable base image tags, build timestamps, package repositories, and install-time dependency resolution. Even with a pinned base image tag, the tag itself can be republished unless a digest is used. Commands like \`pip install\` and \`apt-get install\` also depend on external repository state unless every dependency and hash is pinned. + +Nix dockerTools avoids these issues by constructing the image from immutable Nix store paths and deterministic build inputs. Setting a fixed \`created\` timestamp removes another common source of nondeterminism. + +### Practical Scenarios Where Nix Reproducibility Matters + +- CI/CD systems where builds must be identical across runners. +- Security audits where the exact dependency closure must be known. +- Incident rollback where a previous version must be restored exactly. +- Long-lived projects where dependency repositories change over time. +- Multi-developer teams where local environments otherwise drift. + +### Screenshot Evidence + +- [Containers side-by-side](lab18/screenshots/02-containers-side-by-side.png) +- [Nix Docker hashes](lab18/screenshots/04-nix-docker-hashes.png) +- [Docker hash comparison](lab18/screenshots/05-docker-hash-comparison.png) + +### Reflection — Redoing Lab 2 with Nix + +If Lab 2 were implemented with Nix from the beginning, the Dockerfile would be replaced or supplemented by a Nix \`dockerTools\` expression. The application would first be built as a reproducible Nix derivation, then converted into a Docker-compatible image. This would provide stronger guarantees than a traditional Dockerfile while still allowing the final artifact to run with Docker. + +--- + +## Submission Checklist + +- [x] Task 1 Nix expression created: \`labs/lab18/app_python/default.nix\` +- [x] Task 2 Nix Docker expression created: \`labs/lab18/app_python/docker.nix\` +- [x] Bonus task intentionally skipped +- [ ] Evidence scripts executed locally +- [ ] Real command outputs copied into this submission +- [ ] Screenshots added to \`labs/lab18/screenshots/\` +- [ ] Branch \`feature/lab18\` committed +MD + +echo "Wrote $OUT" diff --git a/labs/lab18/run-task1.sh b/labs/lab18/run-task1.sh new file mode 100644 index 0000000000..16c3e142aa --- /dev/null +++ b/labs/lab18/run-task1.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +APP_DIR="$ROOT_DIR/labs/lab18/app_python" +EVIDENCE_DIR="$ROOT_DIR/labs/lab18/evidence" +mkdir -p "$EVIDENCE_DIR" + +cd "$APP_DIR" + +log() { + echo "[lab18-task1] $*" +} + +log "Recording Nix version" +nix --version | tee "$EVIDENCE_DIR/01-nix-version.txt" + +log "Running nixpkgs hello" +nix run nixpkgs#hello | tee "$EVIDENCE_DIR/02-nix-hello.txt" + +log "Building Python app with Nix" +rm -f result +nix-build | tee "$EVIDENCE_DIR/03-nix-build-first.txt" +readlink -f result | tee "$EVIDENCE_DIR/04-store-path-first.txt" +nix-hash --type sha256 result | tee "$EVIDENCE_DIR/05-nix-output-hash-first.txt" + +log "Building again to show identical store path" +rm -f result +nix-build | tee "$EVIDENCE_DIR/06-nix-build-second.txt" +readlink -f result | tee "$EVIDENCE_DIR/07-store-path-second.txt" +nix-hash --type sha256 result | tee "$EVIDENCE_DIR/08-nix-output-hash-second.txt" + +log "Forcing rebuild by deleting output store path" +STORE_PATH="$(readlink -f result)" +echo "$STORE_PATH" | tee "$EVIDENCE_DIR/09-store-path-before-delete.txt" +rm -f result +rm -f result labs/lab18/app_python/result +nix-store --delete "$STORE_PATH" | tee "$EVIDENCE_DIR/10-nix-store-delete.txt" || true +nix-build | tee "$EVIDENCE_DIR/11-nix-build-after-delete.txt" +readlink -f result | tee "$EVIDENCE_DIR/12-store-path-after-delete.txt" +nix-hash --type sha256 result | tee "$EVIDENCE_DIR/13-nix-output-hash-after-delete.txt" + +log "Comparing with unpinned pip workflow" +rm -rf venv1 venv2 freeze1.txt freeze2.txt requirements-unpinned.txt +printf 'flask\n' > requirements-unpinned.txt +python3 -m venv venv1 +# shellcheck disable=SC1091 +source venv1/bin/activate +python -m pip install --upgrade pip >/dev/null +pip install -r requirements-unpinned.txt >/dev/null +pip freeze | grep -i -E '^(Flask|Werkzeug|Jinja2|click|itsdangerous|MarkupSafe|blinker)==' | sort > freeze1.txt +deactivate +pip cache purge >/dev/null 2>&1 || rm -rf ~/.cache/pip +python3 -m venv venv2 +# shellcheck disable=SC1091 +source venv2/bin/activate +python -m pip install --upgrade pip >/dev/null +pip install -r requirements-unpinned.txt >/dev/null +pip freeze | grep -i -E '^(Flask|Werkzeug|Jinja2|click|itsdangerous|MarkupSafe|blinker)==' | sort > freeze2.txt +deactivate +{ + echo '--- freeze1.txt ---' + cat freeze1.txt + echo '--- freeze2.txt ---' + cat freeze2.txt + echo '--- diff freeze1 freeze2 ---' + diff -u freeze1.txt freeze2.txt || true +} | tee "$EVIDENCE_DIR/14-pip-comparison.txt" + +log "Task 1 evidence written to $EVIDENCE_DIR" diff --git a/labs/lab18/run-task2.sh b/labs/lab18/run-task2.sh new file mode 100644 index 0000000000..88bca56df0 --- /dev/null +++ b/labs/lab18/run-task2.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +APP_DIR="$ROOT_DIR/labs/lab18/app_python" +EVIDENCE_DIR="$ROOT_DIR/labs/lab18/evidence" +mkdir -p "$EVIDENCE_DIR" + +log() { + echo "[lab18-task2] $*" +} + +cd "$ROOT_DIR" + +log "Cleaning old lab18 containers" +docker stop lab2-container nix-container >/dev/null 2>&1 || true +docker rm lab2-container nix-container >/dev/null 2>&1 || true + +log "Building traditional Lab 2 Docker image twice" +docker build -t lab2-app:v1 ./app_python | tee "$EVIDENCE_DIR/15-lab2-docker-build-v1.txt" +docker inspect lab2-app:v1 --format '{{.Created}}' | tee "$EVIDENCE_DIR/16-lab2-created-v1.txt" +docker save lab2-app:v1 | sha256sum | tee "$EVIDENCE_DIR/19-lab2-save-hash-v1.txt" + +sleep 2 + +docker build -t lab2-app:v2 ./app_python | tee "$EVIDENCE_DIR/17-lab2-docker-build-v2.txt" +docker inspect lab2-app:v2 --format '{{.Created}}' | tee "$EVIDENCE_DIR/18-lab2-created-v2.txt" +docker save lab2-app:v2 | sha256sum | tee "$EVIDENCE_DIR/20-lab2-save-hash-v2.txt" + +log "Building Nix dockerTools image twice" +cd "$APP_DIR" +rm -f result +nix-build docker.nix | tee "$EVIDENCE_DIR/21-nix-docker-build-first.txt" +sha256sum result | tee "$EVIDENCE_DIR/22-nix-docker-hash-first.txt" + +docker load < result | tee "$EVIDENCE_DIR/23-nix-docker-load.txt" + +rm -f result +nix-build docker.nix | tee "$EVIDENCE_DIR/24-nix-docker-build-second.txt" +sha256sum result | tee "$EVIDENCE_DIR/25-nix-docker-hash-second.txt" + +log "Running traditional and Nix containers side-by-side" +docker stop lab2-container nix-container >/dev/null 2>&1 || true +docker rm lab2-container nix-container >/dev/null 2>&1 || true + +docker run -d -p 5000:5000 --name lab2-container lab2-app:v1 | tee "$EVIDENCE_DIR/26-lab2-container-id.txt" +docker run -d -p 5001:5000 --name nix-container devops-info-service-nix:1.0.0 | tee "$EVIDENCE_DIR/27-nix-container-id.txt" +sleep 5 + +{ + echo '--- docker ps ---' + docker ps --filter name=lab2-container --filter name=nix-container + echo '--- lab2 container health ---' + curl -fsS http://localhost:5000/health + echo + echo '--- nix container health ---' + curl -fsS http://localhost:5001/health + echo +} | tee "$EVIDENCE_DIR/28-container-health-checks.txt" + +log "Collecting image size and history comparisons" +docker images | grep -E 'lab2-app|devops-info-service-nix' | tee "$EVIDENCE_DIR/29-image-size-comparison.txt" +docker history lab2-app:v1 | tee "$EVIDENCE_DIR/30-lab2-docker-history.txt" +docker history devops-info-service-nix:1.0.0 | tee "$EVIDENCE_DIR/31-nix-docker-history.txt" + +log "Task 2 evidence written to $EVIDENCE_DIR" diff --git a/labs/lab18/screenshots/.gitkeep b/labs/lab18/screenshots/.gitkeep new file mode 100644 index 0000000000..e69de29bb2 diff --git a/labs/lab18/screenshots/01-nix-app-running.png b/labs/lab18/screenshots/01-nix-app-running.png new file mode 100644 index 0000000000..78e104a75c Binary files /dev/null and b/labs/lab18/screenshots/01-nix-app-running.png differ diff --git a/labs/lab18/screenshots/02-containers-side-by-side.png b/labs/lab18/screenshots/02-containers-side-by-side.png new file mode 100644 index 0000000000..0e419ae79d Binary files /dev/null and b/labs/lab18/screenshots/02-containers-side-by-side.png differ diff --git a/labs/lab18/screenshots/03-nix-build-reproducibility.png b/labs/lab18/screenshots/03-nix-build-reproducibility.png new file mode 100644 index 0000000000..80db00094f Binary files /dev/null and b/labs/lab18/screenshots/03-nix-build-reproducibility.png differ diff --git a/labs/lab18/screenshots/04-nix-docker-hashes.png b/labs/lab18/screenshots/04-nix-docker-hashes.png new file mode 100644 index 0000000000..799a85cfc3 Binary files /dev/null and b/labs/lab18/screenshots/04-nix-docker-hashes.png differ diff --git a/labs/lab18/screenshots/05-docker-hash-comparison.png b/labs/lab18/screenshots/05-docker-hash-comparison.png new file mode 100644 index 0000000000..07cbe3c17a Binary files /dev/null and b/labs/lab18/screenshots/05-docker-hash-comparison.png differ diff --git a/labs/submission18.md b/labs/submission18.md new file mode 100644 index 0000000000..921c9ec1f4 --- /dev/null +++ b/labs/submission18.md @@ -0,0 +1,470 @@ +# Lab 18 — Reproducible Builds with Nix + +## Scope + +This submission completes the required Lab 18 tasks only: + +- Task 1 — Build Reproducible Python App with Nix +- Task 2 — Reproducible Docker Images with Nix dockerTools + +The bonus Flakes task was intentionally not completed. + +--- + +## Environment and Installation Verification + +### Nix version + +```text +nix (Determinate Nix 3.20.0) 2.34.6 +``` + +### Basic Nix test + +Command used: + +```bash +nix run nixpkgs#hello +``` + +Output: + +```text +Hello, world! +``` + +--- + +## Task 1 — Reproducible Python App + +### Application Source + +The Lab 1/2 Flask application was copied to: + +```text +labs/lab18/app_python/ +``` + +The directory contains: + +- `app.py` — DevOps Info Service Flask application +- `requirements.txt` — original pip dependency file +- `Dockerfile` — traditional Lab 2 Dockerfile for comparison +- `default.nix` — Nix expression for reproducible app build +- `docker.nix` — Nix dockerTools image definition + +### `default.nix` + +```nix +{ pkgs ? import {} }: + +let + pythonEnv = pkgs.python3.withPackages (ps: with ps; [ + flask + prometheus-client + ]); +in +pkgs.stdenv.mkDerivation { + pname = "devops-info-service"; + version = "1.0.0"; + src = ./.; + + nativeBuildInputs = [ pkgs.makeWrapper ]; + + installPhase = '' + runHook preInstall + + mkdir -p $out/share/devops-info-service + cp app.py $out/share/devops-info-service/app.py + + mkdir -p $out/bin + makeWrapper ${pythonEnv}/bin/python $out/bin/devops-info-service \ + --add-flags $out/share/devops-info-service/app.py \ + --set HOST 0.0.0.0 \ + --set PORT 5000 \ + --set SERVICE_NAME devops-info-service \ + --set SERVICE_VERSION 1.0.0-nix \ + --set APP_ENV lab18-nix \ + --set VISITS_FILE /tmp/devops-info-service-visits + + runHook postInstall + ''; + + meta = with pkgs.lib; { + description = "DevOps Info Service built reproducibly with Nix"; + mainProgram = "devops-info-service"; + platforms = platforms.linux; + }; +} +``` + +### Explanation of `default.nix` + +| Field | Purpose | +|---|---| +| `pname` | Package name in the Nix store. | +| `version` | Application version included in the store path. | +| `src = ./.` | Uses the current directory as the source input. | +| `pythonEnv` | Creates a Python interpreter with exact dependencies from nixpkgs. | +| `flask` | Provides the Flask web framework dependency. | +| `prometheus-client` | Provides the Prometheus metrics dependency used by the app. | +| `makeWrapper` | Creates an executable wrapper for running `app.py` with the Nix-managed Python interpreter. | +| `VISITS_FILE=/tmp/devops-info-service-visits` | Makes the app runnable locally and inside minimal containers without relying on `/data`. | + +### Store Path Reproducibility + +First Nix build store path: + +```text +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 +``` + +Second Nix build store path: + +```text +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 +``` + +After deleting the output store path and rebuilding: + +```text +/nix/store/d0d1zw0my13vdjj06m2d3zhzzgm90060-devops-info-service-1.0.0 +``` + +The store paths are expected to be identical because Nix derives the output path from all build inputs, including the source, build instructions, Python interpreter, and dependencies. + +### Nix Output Hashes + +First output hash: + +```text +dd2bd2abba01a00297ed14ad60f29a24b67ce457b1211cdb52dc3b5c2eb0a0fd +``` + +Second output hash: + +```text +dd2bd2abba01a00297ed14ad60f29a24b67ce457b1211cdb52dc3b5c2eb0a0fd +``` + +After forced rebuild: + +```text +dd2bd2abba01a00297ed14ad60f29a24b67ce457b1211cdb52dc3b5c2eb0a0fd +``` + +These hashes demonstrate that the Nix-built application output is reproducible when inputs do not change. + +### Pip Comparison + +The following comparison was produced by creating two virtual environments from an unpinned `requirements-unpinned.txt` containing only `flask`: + +```text +--- freeze1.txt --- +Flask==3.1.3 +Jinja2==3.1.6 +MarkupSafe==3.0.3 +Werkzeug==3.1.8 +blinker==1.9.0 +click==8.3.3 +itsdangerous==2.2.0 +--- freeze2.txt --- +Flask==3.1.3 +Jinja2==3.1.6 +MarkupSafe==3.0.3 +Werkzeug==3.1.8 +blinker==1.9.0 +click==8.3.3 +itsdangerous==2.2.0 +--- diff freeze1 freeze2 --- +``` + +### Why `requirements.txt` Is Weaker Than Nix + +A traditional `requirements.txt` file can pin direct Python dependencies, but it does not fully describe the entire build environment. It usually depends on the system Python version, pip resolver behavior, platform-specific wheels, build tools, and transitive dependency resolution at install time. Even when direct dependencies are pinned, the environment is not as strongly isolated or content-addressed as a Nix build. + +Nix records the full dependency closure and builds in an isolated environment. The output path includes a hash derived from all relevant inputs. This makes the result reproducible and safely cacheable. + +### Nix Store Path Format + +A Nix output path has this general structure: + +```text +/nix/store/-- +``` + +For example: + +```text +/nix/store/-devops-info-service-1.0.0 +``` + +The hash is derived from build inputs. If the source code, dependency versions, or build instructions change, the hash changes and Nix creates a new output path. + +### Screenshot Evidence + +- [Nix app running](lab18/screenshots/01-nix-app-running.png) +- [Nix build reproducibility](lab18/screenshots/03-nix-build-reproducibility.png) + +### Reflection — How Nix Would Have Helped in Lab 1 + +Nix would have made the Lab 1 setup more reproducible from the beginning. Instead of relying on the local Python installation, virtual environment state, and pip behavior, the application could have been built with a declared Python interpreter and dependency set. New machines and CI runners would use the same build inputs and produce the same result, reducing “works on my machine” problems. + +--- + +## Task 2 — Reproducible Docker Images with Nix + +### Traditional Lab 2 Dockerfile + +```dockerfile +# Production-oriented image for a small Flask app. +# Pin a specific Python version for reproducible builds. +FROM python:3.13.1-slim + +# Python runtime defaults: no .pyc files, unbuffered logs (better for containers) +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 + +WORKDIR /app + +# Create a dedicated non-root user with numeric UID/GID. +RUN addgroup --system --gid 10001 app \ + && adduser --system --uid 10001 --ingroup app --no-create-home app \ + && mkdir -p /data /config \ + && chown -R 10001:10001 /app /data /config + +# Install dependencies first to leverage Docker layer caching. +COPY requirements.txt ./ +RUN pip install --no-cache-dir -r requirements.txt + +# Copy only the application code needed at runtime. +COPY app.py ./ + +# Drop privileges using a numeric user for Kubernetes runAsNonRoot validation. +USER 10001:10001 + +# Document the port (Flask defaults to 5000 in this repo) +EXPOSE 5000 + +# Start the application +CMD ["python", "app.py"] +``` + +### Nix dockerTools Image Definition + +```nix +{ pkgs ? import {} }: + +let + app = import ./default.nix { inherit pkgs; }; +in +pkgs.dockerTools.buildLayeredImage { + name = "devops-info-service-nix"; + tag = "1.0.0"; + + contents = [ app ]; + + # Fixed timestamp is required for reproducible image tarballs. + created = "1970-01-01T00:00:01Z"; + + extraCommands = '' + mkdir -p tmp + chmod 1777 tmp + ''; + + config = { + Cmd = [ "${app}/bin/devops-info-service" ]; + Env = [ + "HOST=0.0.0.0" + "PORT=5000" + "SERVICE_NAME=devops-info-service" + "SERVICE_VERSION=1.0.0-nix-docker" + "APP_ENV=lab18-nix-docker" + "VISITS_FILE=/tmp/devops-info-service-visits" + ]; + ExposedPorts = { + "5000/tcp" = {}; + }; + }; +} +``` + +### Explanation of `docker.nix` + +| Field | Purpose | +|---|---| +| `app = import ./default.nix` | Reuses the reproducible app derivation from Task 1. | +| `dockerTools.buildLayeredImage` | Builds a Docker-compatible image from Nix store paths. | +| `name` and `tag` | Define the image name `devops-info-service-nix:1.0.0`. | +| `contents = [ app ]` | Adds the app and its closure to the image. | +| `created = "1970-01-01T00:00:01Z"` | Uses a fixed timestamp to avoid timestamp-based nondeterminism. | +| `extraCommands` | Creates a writable `/tmp` directory. | +| `config.Cmd` | Starts the Nix-built application. | +| `ExposedPorts` | Documents the app port `5000/tcp`. | + +### Traditional Dockerfile Reproducibility Test + +Traditional image hash from first `docker save`: + +```text +5d0bd401bee017ef30287e8411a576af3bc1a5f0545cadae4f6f705916977ae6 - +``` + +Traditional image hash from second `docker save`: + +```text +81572d01b29834d3ae5db13a7b98c4f545b9ae0a4f7930d49b1372cd34485797 - +``` + +The traditional Docker image hashes are expected to differ because Docker image metadata and layer creation timestamps can vary between builds, even when the Dockerfile and source files are unchanged. + +### Nix Docker Image Reproducibility Test + +Nix dockerTools image hash from first build: + +```text +aa3638c60f34d8cfda3136c3e29a797f4edd5ee4b215b9603cfc3e1a62450eb5 result +``` + +Nix dockerTools image hash from second build: + +```text +aa3638c60f34d8cfda3136c3e29a797f4edd5ee4b215b9603cfc3e1a62450eb5 result +``` + +The Nix image hashes are expected to be identical because the image is built from content-addressed Nix store paths and uses a fixed creation timestamp. + +### Side-by-Side Runtime Test + +```text +--- docker ps --- +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +9a7ceb173dd2 devops-info-service-nix:1.0.0 "/nix/store/06c4hlva…" 9 seconds ago Up 8 seconds 0.0.0.0:5001->5000/tcp, [::]:5001->5000/tcp nix-container +892f8c447961 lab2-app:v1 "python app.py" 10 seconds ago Up 9 seconds 0.0.0.0:5000->5000/tcp, [::]:5000->5000/tcp lab2-container +--- lab2 container health --- +{"status":"healthy","timestamp":"2026-05-14T20:17:35.129Z","uptime_seconds":8,"variant":"primary"} + +--- nix container health --- +{"status":"healthy","timestamp":"2026-05-14T20:17:35.144Z","uptime_seconds":8,"variant":"primary"} +``` + +The traditional Dockerfile-based container and the Nix dockerTools container both serve the same application and return healthy responses. + +### Image Size Comparison + +```text +lab2-app v2 3d1599933b32 4 weeks ago 189MB +lab2-app v1 8fc4113f4485 4 weeks ago 189MB +devops-info-service-nix 1.0.0 d5f5ad80fafd 56 years ago 396MB +``` + +### Docker History — Traditional Image + +```text +IMAGE CREATED CREATED BY SIZE COMMENT +8fc4113f4485 4 weeks ago CMD ["python" "app.py"] 0B buildkit.dockerfile.v0 + 4 weeks ago EXPOSE &{[{{28 0} {28 0}}] 0xc001a908c0} 0B buildkit.dockerfile.v0 + 4 weeks ago USER 10001:10001 0B buildkit.dockerfile.v0 + 4 weeks ago COPY app.py ./ # buildkit 28.7kB buildkit.dockerfile.v0 + 4 weeks ago RUN /bin/sh -c pip install --no-cache-dir -r… 6.16MB buildkit.dockerfile.v0 + 4 weeks ago COPY requirements.txt ./ # buildkit 12.3kB buildkit.dockerfile.v0 + 4 weeks ago RUN /bin/sh -c addgroup --system --gid 10001… 57.3kB buildkit.dockerfile.v0 + 3 months ago WORKDIR /app 8.19kB buildkit.dockerfile.v0 + 3 months ago ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFER… 0B buildkit.dockerfile.v0 + 16 months ago CMD ["python3"] 0B buildkit.dockerfile.v0 + 16 months ago RUN /bin/sh -c set -eux; for src in idle3 p… 16.4kB buildkit.dockerfile.v0 + 16 months ago RUN /bin/sh -c set -eux; savedAptMark="$(a… 41.3MB buildkit.dockerfile.v0 + 16 months ago ENV PYTHON_SHA256=9cf9427bee9e2242e3877dd0f6… 0B buildkit.dockerfile.v0 + 16 months ago ENV PYTHON_VERSION=3.13.1 0B buildkit.dockerfile.v0 + 16 months ago ENV GPG_KEY=7169605F62C751356D054A26A821E680… 0B buildkit.dockerfile.v0 + 16 months ago RUN /bin/sh -c set -eux; apt-get update; a… 10.4MB buildkit.dockerfile.v0 + 16 months ago ENV PATH=/usr/local/bin:/usr/local/sbin:/usr… 0B buildkit.dockerfile.v0 + 16 months ago # debian.sh --arch 'amd64' out/ 'bookworm' '… 85.2MB debuerreotype 0.15 +``` + +### Docker History — Nix Image + +```text +IMAGE CREATED CREATED BY SIZE COMMENT +d5f5ad80fafd N/A 28.7kB store paths: ['/nix/store/cf8dk5gzsxx6r9xkx1ckvbqz67w1z0gz-devops-info-service-nix-customisation-layer'] + N/A 53.2kB store paths: ['/nix/store/06c4hlvac4h4l4jl4zsn3cgywbsqpna6-devops-info-service-1.0.0'] + N/A 1.22MB store paths: ['/nix/store/wij96d7izljw38kksg0q384ygi8hfvkk-python3-3.12.8-env'] + N/A 856kB store paths: ['/nix/store/a4v5qavdkzip0bnwhmnq2a7m62hpnnpc-python3.12-prometheus-client-0.21.0'] + N/A 1.33MB store paths: ['/nix/store/ijc606v1g5vhqxrjk4qmj787kymp6sal-python3.12-flask-3.0.3'] + N/A 2.99MB store paths: ['/nix/store/jcf93x7sly7l76hyaqwramfyydnbvsf4-python3.12-werkzeug-3.0.6'] + N/A 2.06MB store paths: ['/nix/store/jvrzvrbnxdk6wp1qy9k1iyjhgbzw3jv2-python3.12-jinja2-3.1.5'] + N/A 246kB store paths: ['/nix/store/0vrargkskpn2m47ka7cfkc1bgq35achi-python3.12-itsdangerous-2.2.0'] + N/A 1.39MB store paths: ['/nix/store/k1qzmvyvarz0pgjaz06wx0y5vsgwbhbw-python3.12-click-8.1.7'] + N/A 172kB store paths: ['/nix/store/hb6zkd7hqgqsjsc2dswifgcx8ycc43ag-python3.12-blinker-1.8.2'] + N/A 176kB store paths: ['/nix/store/s03icbxsi62xvd9bigwbgqlgbr5zhkdp-python3.12-markupsafe-3.0.2'] + N/A 121MB store paths: ['/nix/store/dksjvr69ckglyw1k2ss1qgshhcix73p8-python3-3.12.8'] + N/A 1.1MB store paths: ['/nix/store/izpczxh0wcm3ra6z0073zf9j0mv2wfl4-xz-5.6.3'] + N/A 5.33MB store paths: ['/nix/store/v87awkhzf3nr7nc5i4gg77xzqv4bqjy3-tzdata-2025b'] + N/A 1.61MB store paths: ['/nix/store/v9smapvfv1z340qs3p7xbw6zb6zplfcf-sqlite-3.46.1'] + N/A 504kB store paths: ['/nix/store/vb3dx18nky7cq63br7x2mi86isli529w-readline-8.2p13'] + N/A 8.06MB store paths: ['/nix/store/qzn96phpnb6c56mlqa1424hfgf5hp67s-openssl-3.3.3'] + N/A 266kB store paths: ['/nix/store/c25k325zh2b9g8s68b7ixbjfh3a916cb-mpdecimal-4.0.0'] + N/A 164kB store paths: ['/nix/store/n4gd4rqkr0p2rkdhklvbx1rnx78m6dkj-mailcap-2.1.54'] + N/A 172kB store paths: ['/nix/store/iissy6zslzyb85rzjgq4waag9dixvv6s-libxcrypt-4.4.36'] + N/A 98.3kB store paths: ['/nix/store/fm7yigp87wq0p58x92iynwscdmspzkrb-libffi-3.4.6'] + N/A 643kB store paths: ['/nix/store/jn8gi3mbjm6b2khxcbm3vf2c1h5wpv17-gdbm-1.24-lib'] + N/A 328kB store paths: ['/nix/store/h08i7wrlqmd48lnaimaz28pny9i8vmr8-expat-2.7.1'] + N/A 106kB store paths: ['/nix/store/vrqss3954zk1c52mda3xf1rv7wc5ygba-bzip2-1.0.8'] + N/A 9.21MB store paths: ['/nix/store/hh698a2nnpqr47lh52n26wi8fiah3hid-gcc-13.3.0-lib'] + N/A 1.73MB store paths: ['/nix/store/mjhcjikhxps97mq5z54j4gjjfzgmsir5-bash-5.2p37'] + N/A 184kB store paths: ['/nix/store/mkhhjfg2isjbfx87dz191bzpnwx1bbr9-gcc-13.3.0-libgcc'] + N/A 164kB store paths: ['/nix/store/b6mjyiadysqlh7nps52faznnqmp32604-zlib-1.3.1'] + N/A 8.7MB store paths: ['/nix/store/cn67k729khgnd9i1j7gbyh6lpzz11ci5-ncurses-6.4.20221231'] + N/A 31.7MB store paths: ['/nix/store/5m9amsvvh2z8sl7jrnc87hzy21glw6k1-glibc-2.40-66'] + N/A 184kB store paths: ['/nix/store/y4d9iir0yqmrcswaqfi368d8m1rkv14s-xgcc-13.3.0-libgcc'] + N/A 639kB store paths: ['/nix/store/c47b963idja6h1d8n91pf28v2jcq96kp-libidn2-2.3.7'] + N/A 1.88MB store paths: ['/nix/store/2745pvn6cv32yn9gp2rlqiqhqgs01pb5-libunistring-1.2'] +``` + +### Comparison Table — Lab 2 Dockerfile vs Lab 18 Nix dockerTools + +| Aspect | Lab 2 Dockerfile | Lab 18 Nix dockerTools | +|---|---|---| +| Base image | Uses `python:3.13.1-slim` | No mutable base image tag required | +| Dependency installation | Runs `pip install` during Docker build | Uses Nix-built Python environment | +| Timestamp behavior | Build metadata can change per build | Fixed `created` timestamp | +| Reproducibility | Approximate; image hashes can differ | Bit-for-bit reproducible image tarball | +| Caching | Docker layer cache | Nix content-addressed store and binary cache | +| Dependency closure | Implicit in image layers | Explicit Nix closure | +| Auditability | Depends on image and pip state | Store paths identify exact inputs | + +### Why Traditional Dockerfiles Cannot Guarantee Bit-for-Bit Reproducibility + +Traditional Dockerfiles often rely on mutable base image tags, build timestamps, package repositories, and install-time dependency resolution. Even with a pinned base image tag, the tag itself can be republished unless a digest is used. Commands like `pip install` and `apt-get install` also depend on external repository state unless every dependency and hash is pinned. + +Nix dockerTools avoids these issues by constructing the image from immutable Nix store paths and deterministic build inputs. Setting a fixed `created` timestamp removes another common source of nondeterminism. + +### Practical Scenarios Where Nix Reproducibility Matters + +- CI/CD systems where builds must be identical across runners. +- Security audits where the exact dependency closure must be known. +- Incident rollback where a previous version must be restored exactly. +- Long-lived projects where dependency repositories change over time. +- Multi-developer teams where local environments otherwise drift. + +### Screenshot Evidence + +- [Containers side-by-side](lab18/screenshots/02-containers-side-by-side.png) +- [Nix Docker hashes](lab18/screenshots/04-nix-docker-hashes.png) +- [Docker hash comparison](lab18/screenshots/05-docker-hash-comparison.png) + +### Reflection — Redoing Lab 2 with Nix + +If Lab 2 were implemented with Nix from the beginning, the Dockerfile would be replaced or supplemented by a Nix `dockerTools` expression. The application would first be built as a reproducible Nix derivation, then converted into a Docker-compatible image. This would provide stronger guarantees than a traditional Dockerfile while still allowing the final artifact to run with Docker. + +--- + +## Submission Checklist + +- [x] Task 1 Nix expression created: `labs/lab18/app_python/default.nix` +- [x] Task 2 Nix Docker expression created: `labs/lab18/app_python/docker.nix` +- [x] Bonus task intentionally skipped +- [x] Evidence scripts executed locally +- [x] Real command outputs copied into this submission +- [x] Screenshots added to `labs/lab18/screenshots/` +- [x] Branch `feature/lab18` committed