Add file content-hash dedup, expose file hash, harden upload

[martsokha]

Four related changes to the files domain and request observability.

Expose the file content hash

The File response now includes fileHash — the lowercase hex SHA-256 of the file's plaintext (the model already stored it; it was never surfaced). Appears on upload, read, and list responses.

Adds a self-validating FileHash newtype (handler/utility/): deserializes from a 64-character hex string, rejects bad hex or wrong length at the extractor boundary (a malformed value is a clean 400 before the handler runs), serializes back to hex, and carries its own OpenAPI schema (pattern: ^[0-9a-fA-F]{64}$).

Client-side dedup before upload

The file list endpoint gains a hash filter:

GET /workspaces/{workspaceSlug}/files/?hash=<sha256-hex>

It returns the workspace's live files with that exact content. A non-empty result means the file already exists, so the client can skip the upload (or reference the existing file). No new route — hash is a fifth facet alongside search/formats/modality, threaded through FileFilter and both list queries, hitting the existing (file_hash_sha256, file_size_bytes) partial index. Workspace- and document-kind-scoped, so it never leaks across tenants or exposes audit/artifact rows.

This is row-level dedup (find-existing). True storage-level dedup was investigated and rejected: file encryption uses a random per-stream nonce and per-workspace HKDF keys, so identical content never produces identical ciphertext — object dedup would require a convergent-encryption downgrade unacceptable for a PII product.

Harden the upload path against pool starvation

A stress test surfaced connection-pool exhaustion: upload_file held a pooled DB connection across the NATS streaming phase (which needs no DB), so concurrent uploads drained the 10-connection pool and later requests waited ~28s until the request timeout killed them with 500s.

upload_file now does the quick pre-flight DB work (auth, uploader, settings) under a connection, drops it, streams every file with no connection held, then re-acquires only for the final commit transaction.

Remove the RouteCategory metrics categorizer

Its hardcoded prefix list had drifted out of sync with the actual routes — /notifications/ (and others) logged category="unknown", while it still listed dead prefixes like /monitors///documents/. Since almost every route is workspace-scoped and collapses into a single workspaces category anyway, the categorization added noise, not signal. The request-metrics middleware keeps method, uri, status, duration, and body sizes.

Testing

Full gate green: cargo check/clippy --all-targets --all-features --workspace -D warnings/fmt --check, unit tests (197 passed), doc build. New tests cover FileHash parse/round-trip and rejection of bad hex/length.

Follow-ups (not in this PR)

• Pool config: POSTGRES_CONNECTION_TIMEOUT defaults to None (wait forever) — a finite value would fail fast instead of hanging to the request timeout; the default pool size (10) is low for concurrent uploads.
• Audit other handlers (connection-sync import, avatar upload) for the same "hold a connection across slow I/O" pattern.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added exact SHA-256 hash filtering when listing files.
  • File-list responses now include each file’s lowercase SHA-256 content hash.
  • Added validation for hash query parameters, including clear rejection of invalid values.
  • Improved file uploads by releasing database resources during multipart transfers.

• Bug Fixes

  • Duplicate-content filtering now applies consistently across paginated file listings.

• Refactor

  • Simplified request metrics by removing route-category tracking.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 31de5af4-d000-4cde-b27c-110090957f61

📥 Commits

Reviewing files that changed from the base of the PR and between 113f411 and cbd5e42.

📒 Files selected for processing (10)

• crates/nvisy-postgres/src/query/workspace_file.rs
• crates/nvisy-postgres/src/types/filtering/files.rs
• crates/nvisy-server/src/handler/files.rs
• crates/nvisy-server/src/handler/request/files.rs
• crates/nvisy-server/src/handler/response/files.rs
• crates/nvisy-server/src/handler/utility/file_hash.rs
• crates/nvisy-server/src/handler/utility/mod.rs
• crates/nvisy-server/src/middleware/mod.rs
• crates/nvisy-server/src/middleware/observability.rs
• crates/nvisy-server/src/middleware/route_category.rs

💤 Files with no reviewable changes (2)

• crates/nvisy-server/src/middleware/mod.rs
• crates/nvisy-server/src/middleware/route_category.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds SHA-256 file hash filtering and response fields, adjusts upload database connection scopes, and removes route-category tracking from request metrics.

Changes

File hash support

Layer / File(s)	Summary
Hash contract and request mapping crates/nvisy-server/src/handler/utility/file_hash.rs, crates/nvisy-server/src/handler/utility/mod.rs, crates/nvisy-server/src/handler/request/files.rs, crates/nvisy-postgres/src/types/filtering/files.rs	FileHash validates 64-character hexadecimal SHA-256 values. ListFiles maps the value to FileFilter.
Hash-aware file queries crates/nvisy-postgres/src/query/workspace_file.rs	Offset and cursor listing queries filter exact file hashes. Cursor count and fetch queries use the same hash constraint.
Hash response and API documentation crates/nvisy-server/src/handler/response/files.rs, crates/nvisy-server/src/handler/files.rs	File responses include lowercase hexadecimal hashes. File-list documentation describes hash filtering and the 400 response.

Upload connection lifetime

Layer / File(s)	Summary
Scoped upload preparation and persistence crates/nvisy-server/src/handler/files.rs	Upload authorization and settings lookup use a short-lived database connection. The handler releases it during streaming and reacquires it for final transactional persistence.

Request metrics simplification

Layer / File(s)	Summary
Uncategorized request metrics crates/nvisy-server/src/middleware/mod.rs, crates/nvisy-server/src/middleware/route_category.rs, crates/nvisy-server/src/middleware/observability.rs	The middleware removes RouteCategory and renames the metrics function to track_request_metrics. Metrics no longer include route categories.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: 🟡 Moderate · up to cbd5e

The upload flow now releases database connections during streaming, but a failure when reacquiring one for the final commit can leave encrypted objects without file records or cleanup coverage. Repeated authorized uploads under database pressure could consume storage and affect availability, so this should be addressed or explicitly accepted before merging.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.91% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 8 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the primary changes: content-hash deduplication, exposed file hashes, and hardened uploads. It is concise and specific.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/file-dedup-and-upload-hardening

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}