Skip to content

docs(samples): add AML alert triage coded agent sample - #1110

Open
1aifanatic wants to merge 2 commits into
UiPath:mainfrom
1aifanatic:feat/aml-alert-triage-sample
Open

1aifanatic wants to merge 2 commits into
UiPath:mainfrom
1aifanatic:feat/aml-alert-triage-sample

Conversation

@1aifanatic

@1aifanatic 1aifanatic commented Sep 23, 2026

Copy link
Copy Markdown

Closes #1109

Summary

Adds samples/aml-alert-triage-agent: a LangGraph coded agent that triages anti-money-laundering alerts. The UiPath LLM Gateway does the language work (extraction and rationale), and an external decision model (TypeSafe Jev) makes every typed, calibrated decision in a single call.

extract (LLM Gateway) -> decide (Jev: 5x red flag, risk score, disposition) -> explain (LLM Gateway)

What's in the sample

File Purpose
main.py The graph: extract -> decide -> explain, with a switchable decider input (jev / llm)
rubric.py The policy: red flags, risk levels, and dispositions, versioned via RUBRIC_VERSION
evaluate.py Schema gate, accuracy, evidence grounding, and a --compare head-to-head
evals/ 12 synthetic alerts with ground truth, plus ready-to-run sample-*.json inputs
docs/ Architecture SVG, Mermaid diagrams, copy-paste test inputs

The layout matches the other samples: langgraph.json, uipath.json, bindings.json, .env.example, and a uv-style pyproject.toml with a uipath-dev dev group. entry-points.json is not committed, per the repo's .gitignore. samples/README.md gets a new entry.

Notes for reviewers

  • Third-party dependency: the default decider calls the TypeSafe API (typesafe-sdk) and needs a JEV_API_KEY. Locally the key comes from .env; in serverless runs it is read from an Orchestrator Asset named JevApiKey. Setting decider: "llm" runs the whole agent on the LLM Gateway alone.
  • Tested: it runs locally with uipath run, and was published and run on Orchestrator as PythonCodedAgent. After the review fixes (902ec88), the raw narrative is no longer sent to Jev, so the README's measured results are marked pending re-measurement until python evaluate.py --compare is re-run. The review fixes are covered by an offline check that runs the real graph with both models faked: prompt-injection containment, input-field protection, output bounds, and the compare exit code.
  • Untrusted input: alert text goes to every LLM call in its own <alert>-tagged message, and all model output is validated against Pydantic schemas (ExtractedFacts, LLMDecision, bounded Output).
  • CI: the testcases/* integration jobs and SonarCloud fail on this fork PR because repository secrets (CLIENT_ID/CLIENT_SECRET/BASE_URL, SONAR_TOKEN) are not exposed to forks. They fail at uipath auth and don't exercise this sample. Lint, unit tests, commit-lint and Socket Security pass.
  • All alert data is synthetic.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G2Vpmcrr6qxtrpnwynoJJk

Adds samples/aml-alert-triage-agent: a LangGraph coded agent that triages
anti-money-laundering alerts. The UiPath LLM Gateway extracts facts and writes
the rationale; an external decision model (TypeSafe Jev) makes every typed,
calibrated decision (5 red flags, a risk level, a disposition) in one call.

Includes a 12-alert synthetic eval set with ground truth, evaluate.py for
schema/accuracy/evidence-grounding checks, and a `decider` input that swaps
the gateway LLM into the decision seat for a like-for-like comparison.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G2Vpmcrr6qxtrpnwynoJJk
Copilot AI lite review requested due to automatic review settings September 23, 2026 03:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical data-handling and validation issues, along with evaluation and dependency fixes, remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 3 High severity · 2 Medium severity

Open (5)
What changed in this PR

Adds an AML alert-triage LangGraph coded-agent sample integrating UiPath LLM Gateway with TypeSafe Jev.

Changes:

  • Adds extraction, decision, and explanation graph nodes with switchable deciders.
  • Adds AML rubric, synthetic evaluations, and comparison tooling.
  • Adds deployment configuration, examples, diagrams, and documentation.
File Description
samples/​README.md Adds the sample index entry.
samples/​aml-alert-triage-agent/​uipath.json Configures packaging and runtime.
samples/​aml-alert-triage-agent/​rubric.py Defines the AML decision rubric.
samples/​aml-alert-triage-agent/​README.md Documents usage, architecture, and results.
samples/​aml-alert-triage-agent/​pyproject.toml Defines project dependencies.
samples/​aml-alert-triage-agent/​main.py Implements the agent graph and integrations.
samples/​aml-alert-triage-agent/​langgraph.json Registers the graph entry point.
samples/​aml-alert-triage-agent/​evaluate.py Runs evaluation and comparison reports.
samples/​aml-alert-triage-agent/​evals/​sample-llm.json Provides a runnable LLM example.
samples/​aml-alert-triage-agent/​evals/​sample-escalate.json Provides a runnable escalation example.
samples/​aml-alert-triage-agent/​evals/​sample-close.json Provides a runnable closure example.
samples/​aml-alert-triage-agent/​evals/​sample-ambiguous.json Provides a runnable ambiguous-alert example.
samples/​aml-alert-triage-agent/​evals/​alerts.json Provides synthetic alerts and expected outcomes.
samples/​aml-alert-triage-agent/​docs/​test-examples.md Documents testing examples.
samples/​aml-alert-triage-agent/​docs/​diagrams.md Documents system flows.
samples/​aml-alert-triage-agent/​docs/​architecture.svg Provides the architecture illustration.
samples/​aml-alert-triage-agent/​bindings.json Defines deployment resources.
samples/​aml-alert-triage-agent/​agent.mermaid Records the graph topology.
samples/​aml-alert-triage-agent/​.gitignore Ignores generated evaluation results.
samples/​aml-alert-triage-agent/​.env.example Documents required environment variables.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread samples/aml-alert-triage-agent/main.py Outdated
Comment thread samples/aml-alert-triage-agent/main.py Outdated
Comment thread samples/aml-alert-triage-agent/main.py Outdated
Comment thread samples/aml-alert-triage-agent/evaluate.py Outdated
Comment thread samples/aml-alert-triage-agent/main.py Outdated
- Send alert text in its own message inside <alert> tags; every system prompt
  says to treat the tag contents as data, not instructions.
- Parse extraction output into ExtractedFacts, so unknown keys are dropped and
  the LLM cannot overwrite the authoritative input fields.
- Stop sending the raw narrative to the external Jev API. The digest now holds
  only input fields and validated facts, plus extracted jurisdictions.
- Parse the LLM decider's answer into LLMDecision and bound probabilities and
  confidence in Output; malformed answers fail instead of passing as valid.
- evaluate.py --compare now requires both deciders to pass the schema gate.
- README: document untrusted-input handling and mark measured results as
  pending re-measurement against the new digest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RruNaZj2sZwu39KNyBracV

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sample proposal: AML alert triage coded agent (LLM extracts, decision model decides)

2 participants