feat: Add prompt-attack-detection-agent#206
Open
Ujjwal5705 wants to merge 8 commits into
Open
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughChangesA new Prompt Attack Detection Agent kit defines prompt-threat analysis behavior, connects it to a configured Lamatic LLM flow, and provides a Next.js interface for submitting prompts and displaying structured results. Prompt Attack Detection Agent
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
:robot_face: AgentKit Structural ValidationNew Contributions Detected
Check Results
|
Contributor
|
Failure recorded at 2026-07-10T06:00:15Z UTC. If this PR is not fixed within 4 weeks it will be automatically closed. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/prompt-attack-detection-agent/constitutions/default.md`:
- Around line 6-9: Update the Safety section in the default constitution so
prompt-injection and jailbreak text supplied as detector input is treated as
untrusted data: do not follow, execute, or refuse because of those embedded
instructions, and instead classify the content and return the required security
assessment. Preserve refusal behavior only for requests that are outside the
detector’s classification task.
In `@kits/prompt-attack-detection-agent/flows/prompt-attack-detection-agent.ts`:
- Around line 77-118: The response currently stringifies the LLM analysis and
uses an unstructured Generate Text node. Update LLMNode_190 to a schema-backed
structured-output node, configure its output schema for the documented analysis
object, and change graphqlResponseNode’s outputMapping so
{{LLMNode_190.output.generatedResponse}} is mapped without quotes.
In
`@kits/prompt-attack-detection-agent/prompts/prompt-attack-detection-agent_llmnode-190_system_0.md`:
- Around line 19-23: The recommendation values are inconsistent across the
prompt, agent.md, and README.md. Choose a single enum set—matching the
documented contract—and update the recommendation list and all related examples
or parsing guidance consistently across these files.
In
`@kits/prompt-attack-detection-agent/prompts/prompt-attack-detection-agent_llmnode-190_user_1.md`:
- Around line 1-2: Update the detection prompt containing
`{{triggerNode_1.output.prompt}}` to clearly delimit the injected payload and
explicitly instruct the model to treat it only as data, ignoring any
instructions within it. Preserve the request to analyze the payload for security
threats.
In `@kits/prompt-attack-detection-agent/README.md`:
- Around line 80-84: Expand the README’s “Built With” section with a
human-readable setup guide covering prerequisites, Groq credential
configuration, kit deployment, and API invocation. Include the required
environment variables or secrets and a representative sample request showing how
to call the deployed agent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4c21d776-e60c-4ff0-87f9-34b295b886c7
📒 Files selected for processing (8)
kits/prompt-attack-detection-agent/README.mdkits/prompt-attack-detection-agent/agent.mdkits/prompt-attack-detection-agent/constitutions/default.mdkits/prompt-attack-detection-agent/flows/prompt-attack-detection-agent.tskits/prompt-attack-detection-agent/lamatic.config.tskits/prompt-attack-detection-agent/model-configs/prompt-attack-detection-agent_llmnode-190_generative-model-name.tskits/prompt-attack-detection-agent/prompts/prompt-attack-detection-agent_llmnode-190_system_0.mdkits/prompt-attack-detection-agent/prompts/prompt-attack-detection-agent_llmnode-190_user_1.md
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/prompt-attack-detection-agent/apps/actions/orchestrate.ts`:
- Around line 1-36: Import the parent kit’s lamatic.config in analyzePrompt and
use its declared flow or step definitions when invoking executePromptAnalysis.
Update executePromptAnalysis and its callers as needed so execution is driven by
the imported kit configuration rather than relying solely on LAMATIC_FLOW_ID.
In `@kits/prompt-attack-detection-agent/apps/app/globals.css`:
- Around line 5-16: Replace hardcoded colors in the global html/body styles with
semantic CSS custom properties declared in :root, then map those variables to
semantic Tailwind color tokens in tailwind.config.ts. Update affected
components, including AnalysisResult, RiskBadge, LoadingSpinner, and
ErrorMessage, to use tokens such as background, foreground, border, and
destructive instead of raw palette classes.
In `@kits/prompt-attack-detection-agent/apps/components.json`:
- Around line 1-9: Update the shadcn/ui configuration object to include "rsc":
true and an "aliases" block defining the components, utils, and ui import paths,
using the project’s established directory aliases so npx shadcn add resolves and
places files correctly.
In `@kits/prompt-attack-detection-agent/apps/components/AnalysisResult.tsx`:
- Around line 51-60: Update the attack_types mapping in AnalysisResult to use a
composite React key combining each type with its map index, preventing
collisions when duplicate attack type strings are returned.
In `@kits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsx`:
- Around line 15-33: Wrap the textarea and button in a form element within
PromptAnalysisForm, handle submission through the form’s onSubmit callback,
prevent the default submit behavior, and invoke onSubmit with the current
prompt. Set the button type to submit and preserve the existing styling and
controls.
In `@kits/prompt-attack-detection-agent/apps/components/RiskBadge.tsx`:
- Around line 9-14: Update the severity-to-color logic in RiskBadge so
"critical" receives a red or otherwise highest-risk style, "high" and "medium"
retain their appropriate styles, and "low" is the only severity using green.
Normalize severity once and handle these values explicitly, with a safe fallback
that does not imply safety.
In `@kits/prompt-attack-detection-agent/apps/lib/lamatic-client.ts`:
- Around line 31-40: Update the Lamatic call in the function containing
executeFlow to enforce a bounded timeout with cancellation or an SDK-supported
timeout option, and ensure timeout failures are handled consistently. Replace
the raw error-object logging in the catch block with a sanitized, generic
message that excludes URLs, project identifiers, credentials, and SDK response
details while preserving the original error for propagation.
- Line 36: The executePromptAnalysis function should validate res.result at
runtime before casting it to PromptAnalysisOutput, ensuring the nested
data.analysis.risk_score shape is present and valid. Add a lightweight schema or
equivalent guard, handle invalid payloads explicitly, and only return the cast
value after validation.
In `@kits/prompt-attack-detection-agent/apps/package.json`:
- Around line 11-18: The standard form stack is missing from the dependencies
while PromptAnalysisForm.tsx uses local state. Add react-hook-form, zod, and the
appropriate resolver dependency, then refactor PromptAnalysisForm to use useForm
with a zod schema and zodResolver for validation and submission.
- Line 25: Upgrade the tailwindcss dependency in package.json to v4 or later,
replace the v3 Tailwind PostCSS plugin in postcss.config.mjs with
`@tailwindcss/postcss` and add that dependency, and migrate styling configuration
from tailwind.config.ts into CSS using Tailwind v4 directives/theme variables;
remove the obsolete JavaScript configuration file and update any affected
imports or build scripts.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: 714de3ef-3710-4b2b-aa5c-324ed436dff4
⛔ Files ignored due to path filters (1)
kits/prompt-attack-detection-agent/apps/package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (21)
kits/prompt-attack-detection-agent/apps/.env.examplekits/prompt-attack-detection-agent/apps/.gitignorekits/prompt-attack-detection-agent/apps/actions/orchestrate.tskits/prompt-attack-detection-agent/apps/app/globals.csskits/prompt-attack-detection-agent/apps/app/layout.tsxkits/prompt-attack-detection-agent/apps/app/page.tsxkits/prompt-attack-detection-agent/apps/components.jsonkits/prompt-attack-detection-agent/apps/components/AnalysisResult.tsxkits/prompt-attack-detection-agent/apps/components/ErrorMessage.tsxkits/prompt-attack-detection-agent/apps/components/LoadingSpinner.tsxkits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsxkits/prompt-attack-detection-agent/apps/components/RiskBadge.tsxkits/prompt-attack-detection-agent/apps/lib/lamatic-client.tskits/prompt-attack-detection-agent/apps/next-env.d.tskits/prompt-attack-detection-agent/apps/next.config.mjskits/prompt-attack-detection-agent/apps/package.jsonkits/prompt-attack-detection-agent/apps/postcss.config.mjskits/prompt-attack-detection-agent/apps/tailwind.config.tskits/prompt-attack-detection-agent/apps/tsconfig.jsonkits/prompt-attack-detection-agent/apps/types/index.tskits/prompt-attack-detection-agent/lamatic.config.ts
Ujjwal5705
changed the title
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/prompt-attack-detection-agent/apps/app/globals.css`:
- Around line 5-24: Remove the backticks from all theme token values in :root,
including --background, --foreground, and the remaining CSS variables, so each
stores a plain hex color. Preserve the existing var(--background) usage and do
not add another background declaration.
In `@kits/prompt-attack-detection-agent/apps/components.json`:
- Around line 5-9: The shadcn aliases in the configuration contain literal
backticks, producing invalid import paths. Update the components, utils, and ui
alias values to plain path strings without backticks, preserving their existing
path targets.
In `@kits/prompt-attack-detection-agent/apps/lib/lamatic-client.ts`:
- Around line 29-39: Update the request logic in the Lamatic client function
using an AbortController or the SDK’s cancellation mechanism, passing its signal
to lamaticClient.executeFlow and triggering abort when the 30-second timeout
fires. Ensure the timeout is cleared in a finally block so both success and
failure clean up the timer and no background request continues.
In `@kits/prompt-attack-detection-agent/apps/package.json`:
- Around line 25-26: The dependency key for Tailwind’s PostCSS plugin contains
literal backticks and is invalid. In the dependencies object of package.json,
rename the key "`@tailwindcss/postcss`" to the actual package name
"`@tailwindcss/postcss`", preserving the "^4" version.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: e9d4497a-091a-452b-8711-b6223c54b410
📒 Files selected for processing (7)
kits/prompt-attack-detection-agent/apps/actions/orchestrate.tskits/prompt-attack-detection-agent/apps/app/globals.csskits/prompt-attack-detection-agent/apps/components.jsonkits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsxkits/prompt-attack-detection-agent/apps/components/RiskBadge.tsxkits/prompt-attack-detection-agent/apps/lib/lamatic-client.tskits/prompt-attack-detection-agent/apps/package.json
Contributor
|
Hi @Ujjwal5705! 👋 Before this PR can be reviewed by maintainers, please resolve all comments and requested changes from the CodeRabbit automated review. Steps to follow:
This helps keep the review process efficient for everyone. Thank you! 🙏 |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
kits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsx (1)
9-38: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winForm should use
react-hook-form+zodper coding guidelines.The
<form>wrapping fix is confirmed and looks good. However, the coding guideline for kit Next.js apps explicitly requiresreact-hook-form + zodfor forms. This component uses rawuseStatewith no schema validation. For a security-focused kit that accepts user prompts, zod validation (e.g., min/max length, required) would also strengthen the input contract server-side.As per coding guidelines: "For kit Next.js apps: use... react-hook-form + zod for forms."
♻️ Proposed refactor using react-hook-form + zod
+"use client"; + +import { useForm } from "react-hook-form"; +import { zodResolver } from "`@hookform/resolvers/zod`"; +import { z } from "zod"; + +const schema = z.object({ + prompt: z.string().min(1, "Prompt is required").max(10000, "Prompt too long"), +}); + +type FormValues = z.infer<typeof schema>; + interface Props { onSubmit: (prompt: string) => void; } export default function PromptAnalysisForm({ onSubmit }: Props) { - const [prompt, setPrompt] = useState(""); + const { register, handleSubmit, formState: { errors } } = useForm<FormValues>({ + resolver: zodResolver(schema), + }); return ( <form - onSubmit={(e) => { - e.preventDefault(); - onSubmit(prompt); - }} + onSubmit={handleSubmit((values) => onSubmit(values.prompt))} className="space-y-6" > <textarea - value={prompt} - onChange={(e) => setPrompt(e.target.value)} + {...register("prompt")} placeholder="Paste a prompt to analyze..." className="h-72 w-full rounded-xl border border-gray-300 bg-white p-4 font-mono text-gray-900 placeholder:text-gray-500 focus:border-blue-500 focus:outline-none focus:ring-2 focus:ring-blue-500" /> + {errors.prompt && ( + <p className="text-sm text-red-600">{errors.prompt.message}</p> + )} <button type="submit" className="rounded-xl bg-blue-600 px-6 py-3 text-white hover:bg-blue-700" > Analyze Prompt </button> </form> ); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsx` around lines 9 - 38, Refactor PromptAnalysisForm to use react-hook-form with a zod schema instead of raw useState. Define required prompt validation with appropriate minimum and maximum length constraints, wire the schema through zodResolver, register the textarea, display validation errors, and submit validated data through onSubmit while preserving the existing form styling and behavior.Source: Coding guidelines
♻️ Duplicate comments (1)
kits/prompt-attack-detection-agent/apps/components/AnalysisResult.tsx (1)
54-63: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winDuplicate React key risk still present.
Line 57 still uses
key={type}without an index fallback. LLM-generatedattack_typescan contain duplicate strings, causing React key collisions and rendering warnings. This was flagged previously and remains unresolved.♻️ Proposed fix
{a.attack_types.map((type, index) => ( <span - key={type} + key={`${type}-${index}`} className="rounded-full bg-blue-100 px-3 py-1 text-sm" >🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/prompt-attack-detection-agent/apps/components/AnalysisResult.tsx` around lines 54 - 63, Update the attack_types mapping in AnalysisResult to guarantee unique React keys when duplicate type strings occur: include the map index in the key, preferably alongside type, instead of using key={type} alone.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/prompt-attack-detection-agent/apps/lib/lamatic-client.ts`:
- Around line 25-30: Update the Prompt Attack Detection flow execution method
documented above to pass an AbortController signal to executeFlow, abort it when
the timeout wins, and clear the timeout afterward. Before casting or returning
the response, validate that res.result exists and matches the expected
PromptAnalysisOutput shape, throwing a descriptive error for missing or invalid
results.
In `@kits/prompt-attack-detection-agent/README.md`:
- Around line 1-48: The README lacks the required human-readable setup guide and
markdown formatting. Expand README.md with prerequisites, environment-variable
configuration, installation and deployment steps, and a sample request; add a
top-level H1 heading, insert a blank line after the “Folder Structure” heading,
and ensure the file ends with a newline.
---
Outside diff comments:
In `@kits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsx`:
- Around line 9-38: Refactor PromptAnalysisForm to use react-hook-form with a
zod schema instead of raw useState. Define required prompt validation with
appropriate minimum and maximum length constraints, wire the schema through
zodResolver, register the textarea, display validation errors, and submit
validated data through onSubmit while preserving the existing form styling and
behavior.
---
Duplicate comments:
In `@kits/prompt-attack-detection-agent/apps/components/AnalysisResult.tsx`:
- Around line 54-63: Update the attack_types mapping in AnalysisResult to
guarantee unique React keys when duplicate type strings occur: include the map
index in the key, preferably alongside type, instead of using key={type} alone.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: 76047be0-26e6-4d07-b59c-b40f58a59771
📒 Files selected for processing (8)
kits/prompt-attack-detection-agent/README.mdkits/prompt-attack-detection-agent/apps/actions/orchestrate.tskits/prompt-attack-detection-agent/apps/components/AnalysisResult.tsxkits/prompt-attack-detection-agent/apps/components/ErrorMessage.tsxkits/prompt-attack-detection-agent/apps/components/PromptAnalysisForm.tsxkits/prompt-attack-detection-agent/apps/components/RiskBadge.tsxkits/prompt-attack-detection-agent/apps/lib/lamatic-client.tskits/prompt-attack-detection-agent/lamatic.config.ts
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/prompt-attack-detection-agent/README.md`:
- Around line 49-69: Add language identifiers to the four fenced examples in the
README’s prompt examples section, using an appropriate identifier such as text
for each block, and ensure the file ends with exactly one newline.
- Around line 14-15: Update the README’s recommendation values to match the
contract defined in prompt-attack-detection-agent_llmnode-190_system_0.md:
Allow, Allow After Sanitization, and Reject. Ensure the documented values are
exact and consistently ordered across the README and prompt.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5f077902-3900-4d2e-8670-d35c886ea0a2
📒 Files selected for processing (1)
kits/prompt-attack-detection-agent/README.md
Author
|
Hi! All CodeRabbit comments have been addressed. Could someone please take a look when you have time? Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution Type
Summary
This PR adds a new AgentKit named Prompt Attack Detection Agent.
The kit analyzes user prompts for:
The project includes:
Validation Checklist
Folder Structure
kits/prompt-attack-detection-agent/
├── apps/
├── flows/
├── prompts/
├── model-configs/
├── constitutions/
├── README.md
├── agent.md
└── lamatic.config.ts