updates

Author: nihalwashere

.github/workflows/ci.yml

@@ -0,0 +1,30 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  validate:
+    name: Validate docs
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: "20"
+
+      - name: Install Mintlify CLI
+        run: npm install -g mintlify
+
+      - name: Validate mint.json and MDX
+        run: mintlify broken-links || true
+        # Mintlify's link checker exits non-zero on external 4xx too; we
+        # report but don't fail the build on transient external failures.
+
+      - name: OpenAPI sanity check
+        run: |
+          node -e "JSON.parse(require('fs').readFileSync('openapi.json','utf8'))"

.gitignore

@@ -4,14 +4,21 @@
 # Dependencies
 node_modules/
 
+# Environment
+.env
+.env.*
+!.env.example
+
+# Logs
+*.log
+npm-debug.log*
+
 # OS
 .DS_Store
+Thumbs.db
 
 # IDE
 .idea/
 .vscode/
 *.swp
 *.swo
-
-# Claude
-.claude/

.nvmrc

@@ -0,0 +1 @@
+v22.12.0

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 vakra-dev
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,11 +1,57 @@
 # reader-docs
 
-Documentation for [Reader](https://github.com/vakra-dev/reader) - open-source web scraping for LLMs.
+Documentation for [Reader](https://reader.dev), the content extraction API for LLMs. Built with [Mintlify](https://mintlify.com).
 
-Built with [Mintlify](https://mintlify.com).
+## Structure
 
-## Development
+```
+reader-docs/
+├── mint.json                 # Navigation, theme, tabs, and API playground config
+├── openapi.json              # Generated from reader-api. DO NOT edit by hand
+├── home/
+│   ├── introduction.mdx
+│   ├── quickstart.mdx
+│   ├── concepts/             # 11 concept pages (read primitive, proxy modes, errors, …)
+│   └── guides/               # 31 guide pages (getting-started, async, production, …)
+├── api-reference/            # 10 MDX shells bound to openapi.json operations
+├── sdk/                      # JavaScript / Python SDK reference
+└── self-hosted/              # Separate section for the self-hosted flavor. The
+                              # ONLY place internal details like the scraping engine
+                              # and proxy tiers may appear. Cloud docs stay opaque.
+```
+
+## Local development
+
+```bash
+# One-time: install Mintlify CLI
+npm i -g mintlify
+
+# Start the dev server with live reload
+mintlify dev --port 6005
+```
+
+Open http://localhost:6005. Edits to MDX files hot-reload automatically.
+
+**Note on OpenAPI changes:** Mintlify's dev server caches `openapi.json` aggressively. After regenerating the spec (see below) you usually need to stop `mintlify dev` and restart it for the new operation bodies / examples to show up in the API playground.
+
+## Regenerating the OpenAPI spec
+
+`openapi.json` is generated from `reader-api/src/openapi/index.ts`, the source of truth for request/response shapes, error examples, and playground metadata. To regenerate after an API change:
 
 ```bash
-npx mintlify dev
+cd ../reader-api
+npm run docs:gen
 ```
+
+That writes a fresh `reader-docs/openapi.json`. Commit it alongside the reader-api changes so the docs preview matches.
+
+## Content rules
+
+- **Cloud docs are opaque.** Pages under `home/` and `api-reference/` must not mention internal implementation: no `datacenter`/`residential`, no engine internals, no proxy provider names, no "browser pool". The abstract vocabulary is `standard`/`stealth`/`auto` for proxy modes and plain English for everything else.
+- **Self-hosted docs are transparent.** Pages under `self-hosted/` exist specifically so operators can understand the infrastructure they're running. Use the real names there.
+- **API code samples must match the v0.2 SDK shape.** Results are discriminated unions: `if (result.kind === "scrape") result.data.markdown`. Jobs return `result.data.results`, not `result.job.data`. See existing pages under `sdk/` and `home/guides/` for reference patterns.
+- **Error codes are lowercase snake_case** (`rate_limited`, `insufficient_credits`), not `UPPER_SNAKE_CASE`.
+
+## Deploying
+
+Mintlify auto-deploys on push to the default branch. Check the Mintlify dashboard for deploy status and preview URLs.

api-reference/account/credits.mdx

@@ -0,0 +1,8 @@
+---
+title: "Get credit balance"
+openapi: "openapi.json GET /v1/usage/credits"
+---
+
+Returns your current workspace's balance, monthly limit, used, tier, and next reset time.
+
+See [Credits and billing](/home/concepts/credits-and-billing) for what each operation costs.

api-reference/account/history.mdx

@@ -0,0 +1,8 @@
+---
+title: "Usage history"
+openapi: "openapi.json GET /v1/usage/history"
+---
+
+Paginated log of recent requests for the workspace. Each entry records the URL, duration, status, cache hit, proxy mode (`standard` / `stealth`), credits charged, and timestamp.
+
+Use this endpoint to audit spend, find spike culprits, or build a stats dashboard. The dashboard's Activity view is a thin wrapper on top of this feed.

api-reference/authentication.mdx

@@ -0,0 +1,85 @@
+---
+title: "Authentication"
+description: "Every Reader API request is authenticated with an API key passed in the x-api-key header."
+---
+
+Reader uses API key authentication. Every request to `https://api.reader.dev/v1/*` must include a valid key in the `x-api-key` header.
+
+## Get an API key
+
+<Card title="Open Dashboard →" href="https://app.reader.dev" horizontal>
+  Sign up free at app.reader.dev. You get **1,000 credits every month** on the free tier - no credit card required.
+</Card>
+
+Inside the dashboard:
+
+1. Click **API Keys** in the sidebar
+2. Click **Create API Key** and give it a name
+3. Copy the key - it starts with `rdr_` and is shown **only once**
+
+<Warning>
+API keys are shown once at creation time. Store them securely (environment variables, a secrets manager) - never commit them to version control or expose them in client-side code.
+</Warning>
+
+## Using your key
+
+Pass the key in the `x-api-key` header on every request:
+
+<CodeGroup>
+
+```bash curl
+curl -X POST https://api.reader.dev/v1/read \
+  -H "Content-Type: application/json" \
+  -H "x-api-key: rdr_your_api_key" \
+  -d '{"url": "https://example.com"}'
+```
+
+```javascript JavaScript
+import { ReaderClient } from "@vakra-dev/reader-js";
+
+// The SDK handles the header for you
+const reader = new ReaderClient({
+  apiKey: process.env.READER_API_KEY,
+});
+```
+
+```python Python
+from reader_py import ReaderClient
+
+# The SDK handles the header for you
+reader = ReaderClient(api_key=os.environ["READER_API_KEY"])
+```
+
+</CodeGroup>
+
+## Managing keys
+
+All key management happens in the dashboard - not through the API. This is intentional: rotating, revoking, and creating keys is a privileged operation that shouldn't be scriptable from a compromised key.
+
+- **List keys** - API Keys page in the dashboard
+- **Create a new key** - Click "Create API Key"
+- **Revoke a key** - Click the trash icon next to any key
+- **Rotate a key** - Create a new one, update your app, then revoke the old one
+
+You can have up to **10 active keys per workspace**. Use separate keys for development, staging, and production so you can rotate them independently.
+
+## Error responses
+
+| Status | Meaning |
+|---|---|
+| `401 Unauthorized` | Missing, invalid, or revoked `x-api-key` header |
+| `402 Payment Required` | Key is valid but the workspace has run out of credits |
+| `403 Forbidden` | Key doesn't have permission for the requested resource |
+
+See the full [Errors](/home/concepts/errors) reference for status code details and retry guidance.
+
+## Where to go next
+
+<CardGroup cols={2}>
+  <Card title="Make your first request" icon="bolt" href="/home/quickstart">
+    60-second walkthrough from key creation to first scrape.
+  </Card>
+  <Card title="POST /v1/read" icon="code" href="/api-reference/read">
+    Full reference for the unified scrape/crawl/batch endpoint.
+  </Card>
+</CardGroup>