Skip to content

Evaluation: rewrite Postman docs for Evaluation/STT/TTS#821

Closed
AkhileshNegi wants to merge 1 commit intomainfrom
enhancement/postman-evaluation-docs
Closed

Evaluation: rewrite Postman docs for Evaluation/STT/TTS#821
AkhileshNegi wants to merge 1 commit intomainfrom
enhancement/postman-evaluation-docs

Conversation

@AkhileshNegi
Copy link
Copy Markdown
Collaborator

Summary

Target issue is #PLEASE_TYPE_ISSUE_NUMBER

Bring the Evaluation, STT Evaluation, and TTS Evaluation folders of the Postman collection to a quality bar where a non-technical NGO operator can run the full evaluation lifecycle using only API calls.

This PR also tracks Kaapi API'S.postman_collection.json under git for the first time, so future updates can flow through reviewable diffs.

What changed

  • 5 collection bugs fixed (would have blocked first-time NGO users):
    • 2 URL path errors on EvaluationGet Dataset / Delete Dataset (/datasets/dataset_id={dataset_id}/datasets/:dataset_id)
    • 2 unusable Start STT/TTS bodies replaced "models": ["string"] with the actual supported model values
    • 1 form-data field on Upload audio file changed from type: "text" to type: "file" so multipart upload actually works
  • Every endpoint description rewritten across the 3 eval folders with: field-by-field schema, example payloads, response examples, error codes (400/404/409/413/422/500) with remediation, prerequisite endpoints, status enums, polling guidance
  • 3 new "Getting Started" walkthroughs (one per domain) with end-to-end cURL chains
  • 1 new top-level Errors & Troubleshooting reference with the global error matrix, common traps, capability gaps, and auth note
  • Silent invariants made explicit: 1 MB CSV cap, 200 MB audio cap, 5000-char TTS sample cap, 1-hour signed URL TTL, single supported model per provider, "TTS feedback only on status == SUCCESS"
  • Capability gaps surfaced: cancel/retry/export/bulk-feedback/dataset-DELETE/webhooks are not yet supported — now stated so users don't burn time looking

Out of scope

The Model Evaluation folder is intentionally untouched in this PR.

Companion document

docs/postman-evaluation-changelog.md lists the per-endpoint deltas alongside the import-into-fork instructions, so reviewers can scan changes without scrolling the 6500-line JSON.

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran fastapi run --reload app/main.py or docker compose up in the repository root and test. — N/A: docs-only change to the Postman JSON; no backend code touched.
  • If you've fixed a bug or added code that is tested and has test cases. — N/A for the same reason. The 5 bugs are in the Postman collection, validated locally with jq empty + structural assertions (folder counts, residual bug-pattern grep).

Notes

  • Recommended review path: read docs/postman-evaluation-changelog.md first, then spot-check 2–3 endpoint descriptions inside the JSON.
  • After merge, import the updated Kaapi API'S.postman_collection.json into the Postman fork ("Import → Replace" scoped to the fork) and open a Postman PR from the fork to the parent collection.
  • Cross-referenced source-of-truth files (routes / services / models / CRUD / constants) are listed at the bottom of the changelog.

🤖 Generated with Claude Code

Bring the Evaluation, STT Evaluation, and TTS Evaluation folders of the
Postman collection to a quality bar where a non-technical NGO operator
can run the full lifecycle using only API calls.

- Track Kaapi API'S.postman_collection.json under git for the first time.
- Fix 5 collection bugs: 2 URL path errors (Get/Delete Dataset),
  2 unusable Start STT/TTS bodies (`"models": ["string"]`), and 1
  wrong form-data field type on Upload audio file.
- Rewrite every endpoint description in the 3 eval folders with
  schemas, examples, errors, prerequisites, status enums, and
  polling guidance.
- Add 3 Getting Started walkthroughs (one per domain) and a single
  top-level Errors & Troubleshooting reference.
- Surface silent invariants (1 MB CSV cap, 200 MB audio cap,
  5000-char TTS sample cap, 1-hour signed URL TTL, single supported
  model per provider, TTS feedback only on status==SUCCESS).
- Surface capability gaps (no cancel/retry/export/bulk-feedback,
  no STT/TTS dataset DELETE, poll-only completion).

Model Evaluation folder is intentionally untouched.

Companion file: docs/postman-evaluation-changelog.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 11, 2026

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 15ca5d4f-368c-47f1-b3da-1b7127ff1b90

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch enhancement/postman-evaluation-docs

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant