perf: Implement Caching for Gemini API Route Handlers

[vivekducs]

Currently, our app/api/gemini/chat and app/api/gemini/insights endpoints make a live call to the Google Gemini API for every request. This leads to:

• Increased API costs (duplicate token usage)
• Higher latency for repeated queries
• Poor scalability under load

If multiple users send identical or similar requests (e.g., generating report cards), we unnecessarily recompute the same response.

---

Objective

Introduce a caching layer to:

• Reduce redundant API calls
• Improve response time
• Optimize cost efficiency

---

Cache Strategy

• Cache full Gemini API response
• Use deterministic cache key based on request payload
• Store responses with TTL (Time-To-Live)

---

Key Generation

• Generate cache key using SHA-256 hash

• Hash should include:

  • prompt
  • request body params (temperature, model, etc.)

• Ensure stable JSON serialization before hashing

Example:

hash(JSON.stringify(sortedRequestBody))

---

Tasks

• Integrate caching layer (Vercel KV / Redis / unstable_cache)

• Implement SHA-256 hashing for request payload

• Create cache lookup before Gemini API call

• Store successful API responses in cache

• Set TTL to 24 hours

• Skip caching for:

  • Failed responses
  • Streaming/incomplete responses

---

Edge Cases

• Avoid caching:

  • Errors (4xx / 5xx responses)
  • Partial or interrupted responses

• Ensure cache consistency across deployments

• Handle JSON ordering issues in request body

---

Invalidation Policy

• Automatic expiration via TTL (24 hours)

• Future scope:

  • Manual invalidation endpoint (admin use)

---

Acceptance Criteria

• Identical requests within 24 hours:

  • Do NOT trigger new Gemini API call

• Cached response latency:

  • < 100ms

• API call reduction:

  • ≥ 80% for repeated prompts

• No caching of failed or partial responses

---

Tech Notes

• Recommended tools:

  • @vercel/kv or Redis
  • crypto module for hashing

• Use middleware pattern or wrapper function for reuse

• Ensure compatibility with serverless environment

---

[adimadhubani]

Hi @vivekducs, I’d like to contribute to this issue as part of my open-source contributions. I have experience with Next.js, React, Node.js, and API development, and I can implement the Gemini API caching layer with SHA-256-based cache keys, TTL, and proper error handling.

Could you please assign this issue to me? I’d be happy to work on it. Thank you!

[vivekducs]

@adimadhubani Thanks for showing interest, I’ve assigned the issue to you
Feel free to start working on it. If you get stuck anywhere, just drop a comment.

[adimadhubani]

Hi @vivekducs, I have implemented the Gemini API caching layer with 24h TTL, SHA-256 key hashing, and fail-open handling. Created PR #3 for review!