diff --git a/README.md b/README.md index b0256c6..d42c1d5 100644 --- a/README.md +++ b/README.md @@ -144,10 +144,10 @@ modapi init --domain acme.com \ --brief "Stop contact details shared outside the platform, scams and harassment" ``` -`init` does not change your code. To add the moderation calls, let your coding agent do it with the setup skill: +`init` does not change your code. To add the moderation calls, let your coding agent do it with the [setup skill](https://github.com/moderation-api/skills): ```bash -npx skills add moderation-api/cli +npx skills add moderation-api/skills ``` Then ask your agent to "set up Moderation API". The skill finds where users create content, adds the moderation call, handles the result and sets up webhooks, asking you before each change. diff --git a/internal/custom/init.go b/internal/custom/init.go index 37701a9..d1552cb 100644 --- a/internal/custom/init.go +++ b/internal/custom/init.go @@ -405,7 +405,7 @@ func nextSteps(s *stack, r *initResult) []string { steps = append(steps, s.Note) } steps = append(steps, - "Moderate content where users create it: call content.submit and act on recommendation.action (allow, review or reject). The moderation-api-setup agent skill does this for you: npx skills add moderation-api/cli", + "Moderate content where users create it: call content.submit and act on recommendation.action (allow, review or reject). The moderation-api-setup agent skill does this for you: npx skills add moderation-api/skills", "Restart your dev server so it reads the new env values.", ) if r.DashboardURL != "" { diff --git a/skills/moderation-api-setup/SKILL.md b/skills/moderation-api-setup/SKILL.md deleted file mode 100644 index 9260b77..0000000 --- a/skills/moderation-api-setup/SKILL.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -name: moderation-api-setup -description: Set up Moderation API (moderationapi.com) content moderation in an existing codebase - create the project with the modapi CLI, find where users create content, add the moderation call, act on the result, and verify webhooks. Use when the user asks to add content moderation, set up or integrate Moderation API, moderate user-generated content (messages, posts, comments, profiles, uploads), or block spam, scams or abuse in their app. ---- - -# Set up Moderation API - -You add Moderation API to the user's existing app. The `modapi` CLI does the -account work (sign-in, project, keys). You do the code: find where users create -content, moderate it on the server, and act on the result. - -Work in this order. Ask before each change to the user's code or account. - -## 1. Check the starting point - -- Run `modapi --version`. If it is missing, install it: - `curl -fsSL https://raw.githubusercontent.com/moderation-api/cli/main/scripts/install.sh | bash` - (or `go install github.com/moderation-api/cli/cmd/modapi@latest`). -- Run `modapi doctor -o json` in the project root. It tells you if a project - key exists, if it works, if the SDK is installed, and if the env file is in - `.gitignore`. Each check that is not `ok` has a `remedy`. -- Search the code for an existing integration (`@moderation-api/sdk`, - `moderation_api`, `moderation-api/sdk-php`, `moderationapi.com`). If one - exists, do not set up a second one: fill the gaps doctor reports. -- If the app already uses another moderation service, stop and ask the user - whether to replace it or run both. - -## 2. Plan and get approval - -Read the codebase and write a short plan for the user: - -- **The platform**: what it is, who uses it, whether children use it. You pass - this to `init` so the project suggests the right policies. -- **The content entry points**: every server-side place where user content is - created or changed, for example chat messages, posts, comments, reviews, - profile names and bios, listing titles, image uploads. Give file and function. -- **For each entry point**: what to do on `reject` and `review` (see step 5). -- **Webhooks**: whether the app needs to act when a moderator decides later. - -Show the plan and wait for a "yes". Moderation policy is a product decision: -propose, do not decide alone. - -## 3. Create the project - -```bash -modapi init --name "" --domain \ - --context "" \ - --brief "" \ - [--used-by-minors] -o json -``` - -- `init` writes `MODAPI_SECRET_KEY` and `MODAPI_WEBHOOK_SECRET` to `.env.local` - (Next.js) or `.env`, and adds that file to `.gitignore`. It never prints them. -- If it fails with `authentication_error`, the user must sign in. Ask them to - run `modapi login` (it opens their browser; `modapi login --device` over SSH). - Do not ask for keys in the chat. -- If `status` is `already_set_up`, a key exists: keep it and continue. -- Run `--dry-run -o json` first if you want to see what init will do. -- Add `--install` to install the SDK, or run the `install_command` from the - result. There is no Go SDK: in Go, call the REST API. - -## 4. Add the moderation call - -Make one small helper in the app's server code that calls `content.submit`, -and call it from each approved entry point. Snippets for each SDK are in -[references/sdk-snippets.md](references/sdk-snippets.md). - -Send with every call: - -- `content`: `{type: "text", text}`, `{type: "image", url}`, or - `{type: "object", data: {...}}` for content with several fields (a listing - with title, description and photos). -- `contentId`: the ID of the record in the app's database. Webhooks send it back. -- `authorId`: the user's ID. It turns on author trust levels and blocking. -- `conversationId`: the chat, thread or room, when there is one. It gives the - AI the context of the conversation. -- `metadata`: optional small facts that help reviewers, for example the surface. - -## 5. Act on the result - -Read `recommendation.action`: - -| Action | Default handling | -|----------|------------------| -| `allow` | Save and publish as normal. | -| `review` | Save it, and hide it or publish it, as agreed in the plan. It is in the review queue for a moderator. | -| `reject` | Do not publish. Show the user a short, neutral message. | - -Also read `author.block` and `author.trust_level`: a blocked author's content -can be refused without a call. - -Rules: - -- Call from the server only. The key must never reach the browser or the app bundle. -- **Fail open.** If the API is down or times out, log it and allow the content, - so moderation never stops users from posting. Use a short timeout. -- Moderate on create and on edit. -- Do not store the full API response on every record. `contentId` is enough - to find it in the dashboard. - -## 6. Webhooks (when the plan needs them) - -Moderators decide on `review` items later in the dashboard. A webhook tells the -app the decision. - -1. Add a POST route, for example `/webhooks/moderation`. Verify the signature - against the **raw** request body with the SDK (`validateWebhook` in - TypeScript, `construct_event` in Python and Ruby, `constructEvent` in PHP). - They read `MODAPI_WEBHOOK_SECRET`. See the snippets file. -2. Handle `event.type`. The event's `data.object.item.id` is the `contentId` - you sent, and `item.author_id` is your user ID: - - `queue_item.rejected`: hide or delete the content. - - `queue_item.allowed`: publish content you held for review. - - `author.blocked` / `author.suspended` / `author.unblocked`: update the user. - - `queue_item.action`: a custom action; `data.object.key` names it. -3. Register the URL once it is public: - `modapi webhooks create --name "App" --url https:///webhooks/moderation -e QUEUE_ITEM_REJECTED -e QUEUE_ITEM_ALLOWED -e AUTHOR_BLOCKED` - For local testing, use a tunnel (for example `ngrok http 3000`). - -## 7. Verify - -1. Run `modapi doctor`. Fix every `fail`. -2. Send one test: `modapi content submit --content '{"type":"text","text":"Buy followers now, DM me on WhatsApp +1 555 0100"}' --jq '.recommendation.action'` -3. Start the app and create content through one real entry point. Confirm the - request in the dashboard (the link is in the `init` result). -4. Tell the user what you changed, file by file, and give them the link to - review the suggested policies. - -## Never - -- Print, read aloud or commit `.env` values. Refer to them by name. -- Ask the user to paste an API key into the chat. Use `modapi login`, or ask - them to put the key in the env file themselves. -- Call the API from client-side code. -- Create test users, test content or a second project in an existing app - without asking. diff --git a/skills/moderation-api-setup/references/sdk-snippets.md b/skills/moderation-api-setup/references/sdk-snippets.md deleted file mode 100644 index 4cdaa56..0000000 --- a/skills/moderation-api-setup/references/sdk-snippets.md +++ /dev/null @@ -1,229 +0,0 @@ -# SDK snippets - -A moderation helper and a webhook route for each official SDK. Adapt names -and file paths to the codebase; keep the behavior: server-side only, short -timeout, fail open, raw body for webhooks. - -All SDKs read the key from `MODAPI_SECRET_KEY` and the webhook secret from -`MODAPI_WEBHOOK_SECRET` (written by `modapi init`). - -## TypeScript / JavaScript (`@moderation-api/sdk`) - -```typescript -// lib/moderation.ts (server only) -import { ModerationApi } from "@moderation-api/sdk"; - -const moderationApi = new ModerationApi({ - secretKey: process.env.MODAPI_SECRET_KEY ?? "", - timeoutMs: 5_000, -}); - -export type ModerationDecision = "allow" | "review" | "reject"; - -export async function moderateText(input: { - text: string; - contentId: string; - authorId?: string; - conversationId?: string; -}): Promise { - try { - const result = await moderationApi.content.submit({ - content: { type: "text", text: input.text }, - contentId: input.contentId, - authorId: input.authorId, - conversationId: input.conversationId, - }); - return result.recommendation.action; - } catch (error) { - // Fail open: never block users because moderation is unreachable. - console.error("Moderation API call failed", error); - return "allow"; - } -} -``` - -Images: `content: { type: "image", url }`. Several fields at once: -`content: { type: "object", data: { title: { type: "text", text: title }, photo: { type: "image", url } } }`. - -Webhook route (Fetch API: Next.js App Router, Hono, Bun, Workers): - -```typescript -// app/api/webhooks/moderation/route.ts -import { validateWebhook } from "@moderation-api/sdk"; - -export async function POST(request: Request) { - let event; - try { - event = await validateWebhook({ request }); // reads MODAPI_WEBHOOK_SECRET - } catch { - return new Response("Invalid signature", { status: 400 }); - } - const item = event.data?.object?.item; - switch (event.type) { - case "queue_item.rejected": - // hide the record with id item.id - break; - case "queue_item.allowed": - // publish the record with id item.id - break; - } - return new Response(null, { status: 200 }); -} -``` - -Express: mount the route with `express.raw({ type: "application/json" })` so -`req.body` is the raw body, then `await validateWebhook({ request: req })`. -Next.js Pages Router: `export const config = { api: { bodyParser: false } }`. - -## Python (`moderation_api`) - -```python -# moderation.py (server only) -import logging -import os - -from moderation_api import ModerationAPI - -client = ModerationAPI(secret_key=os.environ.get("MODAPI_SECRET_KEY", ""), timeout_ms=5_000) -log = logging.getLogger(__name__) - - -def moderate_text(text: str, content_id: str, author_id: str | None = None, - conversation_id: str | None = None) -> str: - """Return "allow", "review" or "reject".""" - try: - res = client.content.submit( - content={"type": "text", "text": text}, - content_id=content_id, - author_id=author_id, - conversation_id=conversation_id, - ) - return res.recommendation.action - except Exception: # fail open - log.exception("Moderation API call failed") - return "allow" -``` - -Webhook (Flask; in Django use `request.body`, in FastAPI `await request.body()`): - -```python -from flask import request -from moderation_api.webhook_signature import WebhookSignatureError, construct_event - -@app.post("/webhooks/moderation") -def moderation_webhook(): - try: - event = construct_event(request.get_data(), request.headers.get("modapi-signature", "")) - except WebhookSignatureError: - return "Invalid signature", 400 - item = event["data"]["object"].get("item", {}) - if event["type"] == "queue_item.rejected": - ... # hide the record with id item["id"] - return "", 200 -``` - -## Ruby (`moderation_api`) - -```ruby -# app/services/moderation.rb -require "moderation_api" - -module Moderation - Ops = ::ModerationApi::Models::Operations - CLIENT = ::ModerationApi::ModerationApi.new( - secret_key: ENV.fetch("MODAPI_SECRET_KEY", ""), - timeout_ms: 5_000 - ) - - # Returns "allow", "review" or "reject". - def self.moderate_text(text:, content_id:, author_id: nil, conversation_id: nil) - res = CLIENT.content.submit( - body: Ops::SubmitContentRequestBody.new( - content: Ops::TextRequest.new(type: "text", text: text), - content_id: content_id, - author_id: author_id, - conversation_id: conversation_id - ) - ) - res.object&.recommendation&.action || "allow" - rescue StandardError => e # fail open - Rails.logger.error("Moderation API call failed: #{e.message}") - "allow" - end -end -``` - -Webhook (Rails controller; skip CSRF protection for this action): - -```ruby -require "moderation_api/webhook_signature" - -def moderation_webhook - event = ModerationApi::WebhookSignature.construct_event( - request.raw_post, request.headers["modapi-signature"] - ) - item = event.dig("data", "object", "item") || {} - # case event["type"] when "queue_item.rejected" then hide item["id"] - head :ok -rescue ModerationApi::WebhookSignature::VerificationError - head :bad_request -end -``` - -## PHP (`moderation-api/sdk-php`) - -```php -use ModerationApi\ModerationApi; -use ModerationApi\Models\Operations; - -function moderateText(string $text, string $contentId, ?string $authorId = null): string -{ - static $sdk = null; - $sdk ??= ModerationApi::builder() - ->setSecurity(getenv('MODAPI_SECRET_KEY') ?: '') - ->setClient(new \GuzzleHttp\Client(['timeout' => 5])) - ->build(); - - try { - $response = $sdk->content->submit(body: new Operations\SubmitContentRequestBody( - content: new Operations\TextRequest(text: $text), - contentId: $contentId, - authorId: $authorId, - )); - return $response->object?->recommendation?->action ?? 'allow'; - } catch (\Throwable $e) { // fail open - error_log('Moderation API call failed: ' . $e->getMessage()); - return 'allow'; - } -} -``` - -Webhook: - -```php -use ModerationApi\WebhookSignature; -use ModerationApi\WebhookSignatureException; - -$payload = file_get_contents('php://input'); // raw body -try { - $event = WebhookSignature::constructEvent($payload, $_SERVER['HTTP_MODAPI_SIGNATURE'] ?? null); -} catch (WebhookSignatureException $e) { - http_response_code(400); - exit; -} -$item = $event['data']['object']['item'] ?? []; -// if ($event['type'] === 'queue_item.rejected') hide $item['id'] -http_response_code(200); -``` - -In Laravel, read the raw body with `$request->getContent()` and exclude the -route from CSRF verification. - -## Go (no SDK) - -POST `https://moderationapi.com/api/v1/moderate` with -`Authorization: Bearer $MODAPI_SECRET_KEY` and the same JSON body -(`content`, `contentId`, `authorId`, `conversationId`). Use an -`http.Client{Timeout: 5 * time.Second}` and fail open. Verify webhooks with -HMAC-SHA256 over the raw body, hex-encoded, compared in constant time with the -`modapi-signature` header.