feat: add deep health check endpoint validating DB, Stellar RPC, Twilio, and Agent Loop

[AugistineCreates]

Closes #270

---

Summary

This PR implements a deep health check endpoint (GET /health/deep) to validate the reachability and status of the application's external dependencies (Database, Stellar RPC, Twilio, and the Agent Loop). It also updates the Kubernetes readiness probe to utilize this endpoint securely.

Changes

1. Backend Core & Routes

• src/routes/health.ts: Added the /health/deep endpoint.
  • Requires authentication using the X-Internal-Token header (or Bearer token) matched against the INTERNAL_SERVICE_TOKEN environment variable.
  • Executes dependency checks concurrently with a strict 3-second timeout per check.
  • Returns 503 Service Unavailable if any dependency is unhealthy.
  • Returns 200 OK with status degraded if any dependency is degraded but none are unhealthy.
• src/agent/loop.ts: Added tracking of lastTickAt timestamp. Exposed the last tick time in getAgentStatus() to allow checking if the background loop is actively ticking.

2. Kubernetes Configuration

• deploy/k8s/deployment.yaml: Updated the readinessProbe to use an exec command utilizing wget to query /health/deep with the INTERNAL_SERVICE_TOKEN passed in the request header.

3. Testing & Tooling

• tests/unit/routes/health.test.ts: Created unit tests mocking each dependency failure independently, verifying:
  • Authorization protection.
  • Successful health returns.
  • Database, Stellar RPC, and Twilio connectivity failures.
  • Agent loop failure (ticking ceased for > 2x interval) and degradation.
  • 3-second timeout enforcement.
• package.json: Added missing dependencies and type declarations to support compiling and running the test suite in the local workspace.

Verification

Executed the unit tests:

npx jest --preset ts-jest tests/unit/routes/health.test.ts

All 8 tests passed successfully.

[drips-wave]

@AugistineCreates Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits