docs(samples): add AML alert triage coded agent sample - #1110
Open
1aifanatic wants to merge 2 commits into
Open
1aifanatic wants to merge 2 commits into
1aifanatic wants to merge 2 commits into
Conversation
Adds samples/aml-alert-triage-agent: a LangGraph coded agent that triages anti-money-laundering alerts. The UiPath LLM Gateway extracts facts and writes the rationale; an external decision model (TypeSafe Jev) makes every typed, calibrated decision (5 red flags, a risk level, a disposition) in one call. Includes a 12-alert synthetic eval set with ground truth, evaluate.py for schema/accuracy/evidence-grounding checks, and a `decider` input that swaps the gateway LLM into the decision seat for a like-for-like comparison. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G2Vpmcrr6qxtrpnwynoJJk
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Critical data-handling and validation issues, along with evaluation and dependency fixes, remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 3
Open (5)
What changed in this PR
Adds an AML alert-triage LangGraph coded-agent sample integrating UiPath LLM Gateway with TypeSafe Jev.
Changes:
- Adds extraction, decision, and explanation graph nodes with switchable deciders.
- Adds AML rubric, synthetic evaluations, and comparison tooling.
- Adds deployment configuration, examples, diagrams, and documentation.
| File | Description |
|---|---|
samples/README.md |
Adds the sample index entry. |
samples/aml-alert-triage-agent/uipath.json |
Configures packaging and runtime. |
samples/aml-alert-triage-agent/rubric.py |
Defines the AML decision rubric. |
samples/aml-alert-triage-agent/README.md |
Documents usage, architecture, and results. |
samples/aml-alert-triage-agent/pyproject.toml |
Defines project dependencies. |
samples/aml-alert-triage-agent/main.py |
Implements the agent graph and integrations. |
samples/aml-alert-triage-agent/langgraph.json |
Registers the graph entry point. |
samples/aml-alert-triage-agent/evaluate.py |
Runs evaluation and comparison reports. |
samples/aml-alert-triage-agent/evals/sample-llm.json |
Provides a runnable LLM example. |
samples/aml-alert-triage-agent/evals/sample-escalate.json |
Provides a runnable escalation example. |
samples/aml-alert-triage-agent/evals/sample-close.json |
Provides a runnable closure example. |
samples/aml-alert-triage-agent/evals/sample-ambiguous.json |
Provides a runnable ambiguous-alert example. |
samples/aml-alert-triage-agent/evals/alerts.json |
Provides synthetic alerts and expected outcomes. |
samples/aml-alert-triage-agent/docs/test-examples.md |
Documents testing examples. |
samples/aml-alert-triage-agent/docs/diagrams.md |
Documents system flows. |
samples/aml-alert-triage-agent/docs/architecture.svg |
Provides the architecture illustration. |
samples/aml-alert-triage-agent/bindings.json |
Defines deployment resources. |
samples/aml-alert-triage-agent/agent.mermaid |
Records the graph topology. |
samples/aml-alert-triage-agent/.gitignore |
Ignores generated evaluation results. |
samples/aml-alert-triage-agent/.env.example |
Documents required environment variables. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Send alert text in its own message inside <alert> tags; every system prompt says to treat the tag contents as data, not instructions. - Parse extraction output into ExtractedFacts, so unknown keys are dropped and the LLM cannot overwrite the authoritative input fields. - Stop sending the raw narrative to the external Jev API. The digest now holds only input fields and validated facts, plus extracted jurisdictions. - Parse the LLM decider's answer into LLMDecision and bound probabilities and confidence in Output; malformed answers fail instead of passing as valid. - evaluate.py --compare now requires both deciders to pass the schema gate. - README: document untrusted-input handling and mark measured results as pending re-measurement against the new digest. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RruNaZj2sZwu39KNyBracV
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Closes #1109
Summary
Adds
samples/aml-alert-triage-agent: a LangGraph coded agent that triages anti-money-laundering alerts. The UiPath LLM Gateway does the language work (extraction and rationale), and an external decision model (TypeSafe Jev) makes every typed, calibrated decision in a single call.What's in the sample
main.pyextract->decide->explain, with a switchabledeciderinput (jev/llm)rubric.pyRUBRIC_VERSIONevaluate.py--comparehead-to-headevals/sample-*.jsoninputsdocs/The layout matches the other samples:
langgraph.json,uipath.json,bindings.json,.env.example, and auv-stylepyproject.tomlwith auipath-devdev group.entry-points.jsonis not committed, per the repo's.gitignore.samples/README.mdgets a new entry.Notes for reviewers
typesafe-sdk) and needs aJEV_API_KEY. Locally the key comes from.env; in serverless runs it is read from an Orchestrator Asset namedJevApiKey. Settingdecider: "llm"runs the whole agent on the LLM Gateway alone.uipath run, and was published and run on Orchestrator asPythonCodedAgent. After the review fixes (902ec88), the raw narrative is no longer sent to Jev, so the README's measured results are marked pending re-measurement untilpython evaluate.py --compareis re-run. The review fixes are covered by an offline check that runs the real graph with both models faked: prompt-injection containment, input-field protection, output bounds, and the compare exit code.<alert>-tagged message, and all model output is validated against Pydantic schemas (ExtractedFacts,LLMDecision, boundedOutput).testcases/*integration jobs and SonarCloud fail on this fork PR because repository secrets (CLIENT_ID/CLIENT_SECRET/BASE_URL,SONAR_TOKEN) are not exposed to forks. They fail atuipath authand don't exercise this sample. Lint, unit tests, commit-lint and Socket Security pass.🤖 Generated with Claude Code
https://claude.ai/code/session_01G2Vpmcrr6qxtrpnwynoJJk