diff --git a/docs/decisions/0042-decision-model-inference.md b/docs/decisions/0042-decision-model-inference.md new file mode 100644 index 00000000000..2944c089d98 --- /dev/null +++ b/docs/decisions/0042-decision-model-inference.md @@ -0,0 +1,595 @@ +--- +status: proposed +contact: joslat +date: 2026-09-20 +deciders: westey-m, SergeyMenshykh, rogerbarreto +consulted: +informed: +--- + +# Decision-model inference in .NET Agent Framework: `IDecisionClient`, `DecisionLoopEvaluator`, and the TypeSafe provider + +> **PR note:** This ADR ships in the same pull request as the initial .NET implementation: the experimental +> `IDecisionClient` contract in `Microsoft.Agents.AI.Abstractions`, `DecisionLoopEvaluator` in `Microsoft.Agents.AI`, +> the `Microsoft.Agents.AI.TypeSafe` provider package, their tests, and the `Harness_Step06_DecisionLoop` sample. The +> listed deciders are the .NET owners of the Harness/Loop code (proposed by the contributor; maintainers may adjust +> per `docs/decisions/README.md`). The repository's highest ADR number at the time of writing is `0041`; renumber if +> another ADR lands first. + +## Context and Problem Statement + +Microsoft Agent Framework (MAF) contains several places where the runtime must make a **bounded decision** rather than +generate open-ended text: should an agent loop continue, has the task been completed, which already-authorized tools or +Skills are relevant for this turn, which participant should act next, should a result be escalated. + +Today these decisions are made with application code, deterministic rules, or a generative `IChatClient`. Those remain +valid. A new category of model is emerging whose primary operation is not text generation: it evaluates application +state against one or more **bounded, typed questions** and returns structured probabilistic decisions that application +code consumes directly. TypeSafe AI's **Jev** is the motivating implementation; it exposes three primitives: + +- binary yes/no probability (`noul`); +- categorical choice with a probability distribution (`choice`); +- ordered score with a distribution (`score`). + +The provider is not the architectural decision. The question is: + +> **How should .NET Agent Framework integrate decision-oriented inference without treating it as chat generation, +> without permanently owning a provider-neutral inference abstraction that belongs in `Microsoft.Extensions.AI`, +> and without coupling MAF Core to Jev/TypeSafe, while still being implementable and usable today?** + +Related proposals and requests: + +- `dotnet/extensions#7764` — provider-neutral decision-model abstraction for `Microsoft.Extensions.AI` (MEAI). + **Status at the time of writing: opened 2026-09-19 by a community member, `untriaged`, no maintainer response.** + MEAI 10.10.0, the version this repository references, contains no decision types. +- `microsoft/agent-framework#8545` — .NET: integrate MEAI decision-model inference into agent decision points. +- `microsoft/agent-framework#8556` — Python: first-class support for TypeSafe AI's Jev. + +The intended long-term ownership split is unchanged by the MEAI timing: + +```text +Microsoft.Extensions.AI long-term owner of the provider-neutral capability (IDecisionClient) +Microsoft Agent Framework hosts an experimental reference copy of that contract until MEAI ships it, + and consumes it at orchestration seams +Provider packages own transports (Microsoft.Agents.AI.TypeSafe for Jev; others later) +``` + +## Terminology + +**Generative inference**: `messages -> IChatClient -> generated tokens -> optional structured parsing`. Appropriate for +reasoning, prose/code, tool arguments, free-form critique. + +**Decision inference**: `state + bounded typed questions -> decision model -> probabilities / choices / scores`. +Appropriate for "Is the task complete?", "Which candidate applies?", "Where does this fall on an ordered rubric?". + +**Decision policy**: the model returns a probability; the framework or application decides what it means +(`continue / stop / shortlist / escalate`). Thresholds and authorization are policy, not model semantics. + +**Model-reported probability**: the value a decision model returns. It is not an empirically calibrated accuracy +unless a specific model/provider/workload establishes that property. + +### Why this is not just structured `IChatClient` output + +An `IChatClient` can be prompted to emit `{"answered": true}`; MAF already supports structured output (ADR-0036). +That provides *syntactic* structure. A decision contract standardizes what `IChatClient` intentionally does not: one +shared state, multiple heterogeneous questions, closed output domains declared before inference, explicit probability +and distributions, batching, and uncertainty that code can reason over. `P(task_complete) = 0.94` is not semantically +equivalent to `LLM-generated JSON -> parse bool -> true`. The distinction is analogous to MEAI having embedding +abstractions rather than modeling embeddings as "ask a chat model to output an array of numbers". This ADR does not +supersede ADR-0036. + +## Existing architecture this decision builds on + +### `LoopEvaluator` is the judgment seam + +```csharp +public abstract class LoopEvaluator +{ + public abstract ValueTask EvaluateAsync(LoopContext context, CancellationToken cancellationToken = default); +} +``` + +Implementations: `AIJudgeLoopEvaluator`, `DelegateLoopEvaluator`, `CompletionMarkerLoopEvaluator`, +`TodoCompletionLoopEvaluator`, `BackgroundTaskCompletionLoopEvaluator`. `AIJudgeLoopEvaluator` sends the original +request and the latest response to an `IChatClient`, obtains a structured `JudgeVerdict` (`answered`, `gapAnalysis`), +and continues the loop with the gap analysis as feedback. A decision model is a natural second implementation of the +same seam that returns `P(task_complete)` instead of a verdict plus prose. + +### `LoopAgent` ordering already provides a cascade + +`LoopAgent` evaluates its evaluators **in order**. The first evaluator returning `Continue(...)` wins and drives the +next iteration; remaining evaluators are not consulted. The loop stops only when **every** evaluator returns `Stop()`. +Therefore `Stop()` means "this evaluator does not request another iteration", not a veto. (Verified in +`LoopAgent.EvaluateAndBuildNextAsync`.) + +Ordering `[DecisionLoopEvaluator, AIJudgeLoopEvaluator]` yields an asymmetric cheap-then-strong cascade without any new +abstraction: a cheap "incomplete" skips the expensive judge and re-runs the agent; a cheap "complete" is verified by the +strong judge. A cheap false negative costs one iteration; a cheap false positive is still caught. + +`LoopAgent` also shows why continuation feedback matters: in the default reused-session mode, `Continue()` with no +feedback re-invokes the agent with **no new input messages** (`BuildNextMessages`). + +## Decision Drivers + +- **Provider neutrality**: MAF Core must not depend on TypeSafe, Jev, or any decision-model vendor. +- **Correct abstraction ownership**: the reusable inference capability belongs in MEAI long term. Anything MAF + defines meanwhile must mirror the upstream proposal so it can be deleted, not migrated. +- **Implementable and usable now**: the first integration must not be blocked on an untriaged upstream proposal, and + a consumer such as AgentEval must be able to reference one shared contract instead of inventing its own. +- **Semantic clarity**: decision inference is not chat generation. +- **Compatibility**: optional; no change to existing `IChatClient`, agent, tool, Skill, approval, or workflow behavior. +- **Leverage existing seams**: `LoopEvaluator`, `AIContextProvider`, `GroupChatManager`, `AgentSkillsProvider`. +- **Minimal v1 surface**: solve loop completion; no universal selection/routing framework. +- **Safe orchestration**: probabilistic inference is never an authorization or approval mechanism. +- **Data minimization**: a second inference provider is a second external data boundary. +- **Explicit uncertainty and failure**: `P = 0.51` is not truth; provider failure is not "incomplete". +- **Concurrency and AOT**: stateless shareable components; explicit serialized state at the provider boundary. + +## Considered Options + +1. Represent decision models as `IChatClient` and use structured output. +2. Define a MAF-specific public `IDecisionClient` hierarchy. +3. Add Jev/TypeSafe directly as a MAF Core provider. +4. Keep decision inference entirely application-owned; no first-class MAF integration. +5. Gate MAF entirely on the MEAI abstraction (`dotnet/extensions#7764`) and implement nothing until it ships. +6. MEAI owns the future abstraction; MAF consumes decisions at existing seams through a minimal, evaluator-scoped + callback now, and adds an `IDecisionClient` overload when MEAI ships it. +7. **MAF hosts an experimental reference implementation of the proposed MEAI contract in + `Microsoft.Agents.AI.Abstractions`, mirroring `dotnet/extensions#7764`; consumers in Core take `IDecisionClient`; + the Jev transport lives in a separate `Microsoft.Agents.AI.TypeSafe` provider package.** + +## Decision Outcome + +Chosen option: **7**, because it is implementable and usable today, keeps MAF Core provider-neutral, gives every +consumer (the loop evaluator, future selectors, and external frameworks such as AgentEval) one contract, and, by +mirroring the upstream proposal verbatim under `[Experimental]`, makes the eventual move to `Microsoft.Extensions.AI` +a namespace change rather than a redesign. Option 6 was the first iteration of this branch; it was superseded once +it became clear that a callback per consumer does not scale beyond the loop and leaves Choice and Score unusable. + +### Decision 1: `Microsoft.Agents.AI.Abstractions` hosts an experimental reference copy of the MEAI proposal + +The `Decisions/` folder of `Microsoft.Agents.AI.Abstractions` (namespace `Microsoft.Agents.AI`, all types +`[Experimental(MAAI001)]`) defines the contract proposed in `dotnet/extensions#7764`, name for name: + +```text +IDecisionClient : IDisposable GetResponseAsync(DecisionRequest, DecisionOptions?, ct); GetService(Type, key) +DecisionRequest JsonElement State; IList Questions (ids unique) +DecisionOptions ModelId, AdditionalProperties, RawRepresentationFactory, Clone() +DecisionQuestion (abstract) Id, Instructions, AdditionalProperties + BinaryDecisionQuestion Criteria { TrueDescription, FalseDescription } + ChoiceDecisionQuestion IList Choices (unique names, at least two) + ScoreDecisionQuestion IList Levels (ordered, at least two) +DecisionResponse Answers keyed by question id, ResponseId, ModelId, UsageDetails Usage, RawRepresentation +DecisionAnswer (abstract) RawRepresentation, AdditionalProperties + BinaryDecisionAnswer TrueProbability (validated 0..1; no confidence) + ChoiceDecisionAnswer SelectedChoice, Probabilities[name], Confidence? + ScoreDecisionAnswer Score, Probabilities[levelIndex], Confidence? +DecisionClientMetadata ProviderName, ProviderUri, DefaultModelId (via GetService) +DecisionClientExtensions GetResponseAsync(state, JsonTypeInfo, questions); GetService() +``` + +Two additions go beyond the proposal and are candidates to contribute upstream: `DecisionClientException` with a +`DecisionFailureKind` (`Authentication`, `InvalidRequest`, `RateLimited`, `Overloaded`, `ProviderUnavailable`, +`InvalidResponse`, `Unknown`) and `IsTransient`, so callers can distinguish a retryable condition from a permanent +one; and construction-time validation on the answer types (a `TrueProbability`, `Confidence`, or distribution value +supplied at construction or through a setter must be finite and within 0..1). The `Probabilities` collections stay +mutable to match the proposed MEAI shape, so a producer that mutates them afterwards owns their invariants. Both +additions were adopted from AgentEval's `AgentEval.Decisions` contract (ADR-033 there), which is kept isomorphic to +this one. + +Provider limits (for example Jev's 255 choices or 10 levels) are not part of the contract; providers enforce them. + +### Decision 2: the contract is a staging ground, not a fork + +When `Microsoft.Extensions.AI` ships an equivalent abstraction, the MAF types are removed (the experimental +attribute is the licence to do so) and consumers retarget to the MEAI namespace. Mirroring the proposal verbatim is +what keeps that a mechanical change. MAF does not evolve the contract independently: shape changes go to +`dotnet/extensions#7764` first. If MEAI rejects the concept, this ADR is revisited and the types either stay as +MAF's own or move to a satellite package. + +### Decision 3: `DecisionLoopEvaluator` is a `LoopEvaluator` over `IDecisionClient` + +Takes an `IDecisionClient` and asks one binary question, `"Has the agent fully addressed the user's original request?"` (`QuestionId = +"task_completed"`), and maps the probability through `CompletionThreshold` (default 0.90): + +```text +P >= threshold -> LoopEvaluation.Stop() (no continuation requested; later evaluators still run) +P < threshold -> LoopEvaluation.Continue(ContinueFeedbackMessage) +``` + +### Decision 4: `AIJudgeLoopEvaluator` is unchanged + +They solve different problems. The judge produces a verdict **and** free-form gap analysis; the decision evaluator +produces a bounded probability. A decision model is not forced to fabricate prose. + +### Decision 5: continuation carries deterministic, configurable feedback + +Default: `"The original request is not yet fully addressed. Continue working on it."` This is application text, not +model output. Setting it to `null`/whitespace continues without feedback (documented consequence: no new input +messages in reused-session mode). + +### Decision 6: operational failure is not "incomplete" + +A `DecisionClientException` (or any other exception from the client), or a response without a +`BinaryDecisionAnswer` for the completion question, is handled by `DecisionLoopFailureBehavior`. A failure the client +classifies as transient (`DecisionClientException.IsTransient`: rate limit, overload, provider outage) follows the +separate `TransientFailureBehavior` when one is configured, so a loop can survive a hiccup while a rejected key or an +invalid request still surfaces. This adopts the concern raised in `microsoft/agent-framework#8545` (a loop should not +die on a momentary outage) without making "continue" the blanket default. The general policy: + +```text +Throw (default) propagate +Continue request another iteration with the feedback message +DeferToNextEvaluator return Stop() so later evaluators decide; ends the loop if none remain +``` + +Caller cancellation always propagates. No failure mode is selected implicitly based on provider identity. When a policy +other than throwing is applied, the evaluator logs a warning carrying the exception, its kind, and its transience. + +### Decision 7: default state projection is minimal, text-only, and fails loudly + +```json +{ "originalRequest": [ { "role": "user", "text": "..." } ], "latestResponse": "...", "iteration": 2 } +``` + +Serialized through the source-generated `LoopJsonContext`. The evaluator never serializes `AgentSession`, tool +registries, memory, or provider objects. If an original request message contains content other than `TextContent`, +the default projection **throws** with a message pointing at `StateFactory`, instead of silently judging a lossy +representation. `StateFactory` (`Func`) replaces the projection; the application then owns +the disclosed data. This deliberately differs from `AIJudgeLoopEvaluator`, whose `IChatClient` path can preserve +non-text `AIContent`. (Jev, for reference, accepts text-only state and limits a request to 64k tokens.) + +Like `AIJudgeLoopEvaluator`, the default projection judges the **latest** response against the original request. Agents +that build an answer incrementally across turns need a `StateFactory` that accumulates responses (the sample shows one +that keeps its per-run list on `LoopContext.AdditionalProperties`). This was confirmed on the first live run: an agent +answering one sub-question per turn kept scoring `P ≈ 0.04` until it produced a self-contained recap. + +### Decision 8: probability and policy stay separate + +The evaluator applies `CompletionThreshold`; the callback must not. Documentation uses the term **model-reported +probability**. A false stop returns an incomplete result while a false continue costs one iteration, so a high +threshold is the safer default. + +### Decision 9: decision models may shortlist but never authorize (future integrations) + +```text +available candidates -> deterministic authorization / trust -> authorized candidates -> decision-model relevance -> subset +``` + +A decision model may reduce `A, B, C` to `A, C`; it may never introduce `D`. Tool approval (ADR-0006) and hook +enforcement (ADR-0035) are unchanged. + +### Decision 10: use existing seams; no generic selector yet + +Future tool/Skill shortlisting, group-chat routing, and workflow branching use `AIContextProvider`, +`AgentSkillsProvider`, `GroupChatManager.SelectNextAgentAsync`, and workflow executors. `IAICandidateSelector` is +not added until at least two concrete consumers demonstrate the same stable shape. + +### Decision 11: decision inference is not an evaluation framework + +`IEvaluator`/`IAgentEvaluator`, Foundry evaluations, and third-party frameworks such as AgentEval are evaluation +operations; they may consume decision inference the same way they consume `IChatClient`. + +### Decision 12: Jev lives in the `Microsoft.Agents.AI.TypeSafe` provider package + +MAF Core and Abstractions contain no `Jev*`, `Noul*`, or `SystemOne*` types. `TypeSafeDecisionClient` implements +`IDecisionClient` over `POST https://api.typesafe.ai/v1/systemone` (or OpenRouter's relay of the same protocol), +mapping `BinaryDecisionQuestion` to `noul`, `ChoiceDecisionQuestion` to `choice`, and `ScoreDecisionQuestion` to +`score`. It parses strictly (every question answered with the matching kind, every probability in range, the +selected choice among the requested ones, score levels 0-indexed), classifies HTTP failures into +`DecisionFailureKind`, never retries silently, and never puts the API key into an exception. This follows the +repository's provider-package precedent (`.OpenAI`, `.Foundry`, `.CopilotStudio`, ADR-0021) and allocates feature +index 75 (`typesafe`) in the .NET feature-usage registry. + +### Decision 13: pin model versions where reproducibility matters + +Applications may use the moving alias `jev-latest`. Tests, calibration runs, and benchmarks pin a versioned id such as +`jev-1.13.0` (the TypeSafe id; `typesafe/jev-1.13` was an OpenRouter slug). + +### Decision 14: decision inference is an additional external data boundary + +Type docs and the sample state what is sent (projected original request + latest response, every iteration) and that +the default excludes session store, memories, tool catalogs, and `AdditionalProperties`. + +### Decision 15: telemetry stays with the inference component; the evaluator logs orchestration facts only + +The provider owns model-call telemetry. `DecisionLoopEvaluator` accepts an optional `ILoggerFactory` and logs one +debug line per evaluation (iteration, model-reported probability, threshold, outcome) and a warning when a failure +policy is applied. It never logs the projected state. This is the orchestration-level correlation ADR-0003 allows and +`microsoft/agent-framework#8545` asks for, and it doubles as the capture point for calibration and shadow-mode +comparisons without any wrapper client. + +## Implemented .NET API + +The contract types are listed under Decision 1. The loop consumer: + +```csharp +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionLoopEvaluator : LoopEvaluator +{ + public const double DefaultCompletionThreshold = 0.90; + public const string DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?"; + public const string DefaultCompletedDescription = "..."; + public const string DefaultIncompleteDescription = "..."; + public const string DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it."; + public const string CompletionQuestionId = "task_completed"; + + public DecisionLoopEvaluator(IDecisionClient decisionClient, DecisionLoopEvaluatorOptions? options = null, ILoggerFactory? loggerFactory = null); + public override ValueTask EvaluateAsync(LoopContext context, CancellationToken cancellationToken = default); +} + +public sealed class DecisionLoopEvaluatorOptions +{ + public double CompletionThreshold { get; set; } = 0.90; // 0..1, validated at construction + public string? CompletionQuestion { get; set; } // null -> default; blank -> ArgumentException + public string? CompletedDescription { get; set; } // null -> omitted + public string? IncompleteDescription { get; set; } + public string? ContinueFeedbackMessage { get; set; } // null/blank -> continue without feedback + public Func? StateFactory { get; set; } + public DecisionLoopFailureBehavior FailureBehavior { get; set; } = DecisionLoopFailureBehavior.Throw; + public DecisionLoopFailureBehavior? TransientFailureBehavior { get; set; } // null -> FailureBehavior +} + +public enum DecisionLoopFailureBehavior { Throw, Continue, DeferToNextEvaluator } +``` + +The provider: + +```csharp +namespace Microsoft.Agents.AI.TypeSafe; + +public sealed class TypeSafeDecisionClient : IDecisionClient +{ + public TypeSafeDecisionClient(string apiKey, TypeSafeDecisionClientOptions? options = null, HttpClient? httpClient = null); + public Uri Endpoint { get; } + public string ModelId { get; } +} + +public sealed class TypeSafeDecisionClientOptions +{ + public const string DefaultEndpoint = "https://api.typesafe.ai/v1/systemone"; + public const string OpenRouterEndpoint = "https://openrouter.ai/api/v1/systemone"; + public const string DefaultModelId = "jev-latest"; + public Uri Endpoint { get; set; } + public string ModelId { get; set; } + public TimeSpan Timeout { get; set; } +} +``` + +Files: `dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/*.cs` (contract); +`dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluator.cs`, `DecisionLoopEvaluatorOptions.cs`, +`DecisionLoopFailureBehavior.cs`, `DecisionLoopState.cs` (internal projection), plus one `[JsonSerializable]` line in +`LoopJsonContext.cs` (consumer); `dotnet/src/Microsoft.Agents.AI.TypeSafe/*.cs` (provider). + +## Composition patterns + +### Decision model only + +```csharp +var loop = new LoopAgent( + innerAgent, + new DecisionLoopEvaluator(jev, new() { CompletionThreshold = 0.95 }), + new LoopAgentOptions { MaxIterations = 8 }); +``` + +### Cheap decision + strong generative verifier (recommended) + +```csharp +var loop = new LoopAgent( + innerAgent, + [ + new DecisionLoopEvaluator(jev, new() + { + CompletionThreshold = 0.95, + TransientFailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator, + }), + new AIJudgeLoopEvaluator(strongJudgeChatClient), + ], + new LoopAgentOptions { MaxIterations = 8 }); +``` + +```text +decision: incomplete -> Continue -> strong judge not called -> agent runs again +decision: complete -> Stop -> strong judge runs -> Continue + gap analysis | Stop -> loop ends +decision: transient failure -> Stop (defer) -> strong judge runs; permanent failure -> throws +``` + +### Provider and direct use (from the sample) + +```csharp +using IDecisionClient jev = new TypeSafeDecisionClient(apiKey, new() { ModelId = "jev-1.13.0" }); + +DecisionResponse response = await jev.GetResponseAsync(new DecisionRequest(state, +[ + new BinaryDecisionQuestion("refund_requested", "Does the customer request a refund?"), + new ChoiceDecisionQuestion("request_type", "What is the main request?", [new("refund"), new("rebooking"), new("information")]), + new ScoreDecisionQuestion("frustration", "How frustrated is the customer?", [new("calm"), new("concerned"), new("angry")]), +])); + +var refund = (BinaryDecisionAnswer)response.Answers["refund_requested"]; // refund.TrueProbability +``` + +The evaluator applies the completion threshold; a client never does. + +## Security model + +- **No authority**: identity, authorization, resource access, tool permission, approval, secrets, and privilege remain + deterministic policy. A probability is a signal after those boundaries are established. +- **No approval bypass**: ADR-0006 unchanged. A selected or judged tool still follows approval semantics. +- **Deterministic policy wins where provable**: do not replace `if forbidden -> block` with "ask a decision model". +- **Untrusted state**: decision models consume user- and tool-provided content and can be manipulated like generative + models. Authorization precedes selection; outputs are not proof; data is minimized; security-critical uncertainty + escalates or falls back to deterministic policy. Complements ADR-0024 and ADR-0035. +- **Manipulated probabilities** can end a loop early (incomplete result) or prolong it (cost); `MaxIterations` bounds + the latter, and the strong-judge cascade mitigates the former. + +## Privacy and data boundary + +A decision source may be a different provider from the primary `IChatClient`, creating a second data processor. Type +docs state what is sent. The default projection excludes persisted history beyond the original request, unrelated +memory, secrets, tool catalogs, provider-internal objects, and ambient service state. `StateFactory` users own what +they add. + +## Failure and uncertainty semantics + +| Situation | Meaning | Behavior | +| --- | --- | --- | +| auth failure, invalid request, malformed response, invalid probability | permanent operational failure | `FailureBehavior` (default `Throw`) | +| rate limit, overload, provider outage (`IsTransient`) | transient operational failure | `TransientFailureBehavior`, else `FailureBehavior` | +| Non-text original request with default projection | configuration error | always throws (points at `StateFactory`) | +| `P < threshold` | valid semantic result | `Continue(feedback)` | +| `P >= threshold` | valid semantic result | `Stop()` (no continuation requested) | +| caller cancellation | cancellation | propagates | + +A future API may add an explicit uncertainty band (`confident incomplete / uncertain / confident complete`) with two +thresholds. Not required in v1: the ordered-evaluator chain already provides conservative cheap-then-strong +verification. + +## Validation + +The implementation is compliant with this ADR when: + +1. The decision contract in `Microsoft.Agents.AI.Abstractions` mirrors `dotnet/extensions#7764` name for name, is + `[Experimental]`, and adds only the exception model and construction-time validation of answer values. ✔ +2. MAF Core and Abstractions have no dependency on Jev or TypeSafe; the transport is in `Microsoft.Agents.AI.TypeSafe`. ✔ +3. `DecisionLoopEvaluator` is a `LoopEvaluator`. ✔ +4. `AIJudgeLoopEvaluator` behavior is unchanged. ✔ (existing tests pass) +5. Provider failure is never interpreted as a semantic probability. ✔ (evaluator tests for all three policies and a + missing/mismatched answer; provider tests for every HTTP class, unusable bodies, and out-of-range values) +6. The default projection never silently discards material non-text input. ✔ (throws; `StateFactory` path tested) +7. Tool authorization/approval semantics are unchanged. ✔ (no changes outside `Harness/Loop`) +8. The evaluator composes with ordered loop evaluators as described. ✔ (LoopAgent composition tests) +9. Model-call telemetry remains with the provider; the evaluator logs orchestration facts only and never the state. ✔ (tested) +10. Existing agents behave exactly as before unless the evaluator is configured. ✔ + +Tests: `dotnet/tests/Microsoft.Agents.AI.UnitTests/Harness/Loop/DecisionLoopEvaluatorTests.cs` (evaluator: construction +validation, threshold boundaries, feedback, request contents and projection, criteria omission, non-text rejection, +`StateFactory`, all failure policies, transient versus permanent failures, missing/mismatched answer, cancellation, +provider-side `OperationCanceledException`, logging content and redaction, concurrency, and `LoopAgent` cascade behavior) and +`dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/*.cs` (provider: request serialization for all three kinds, +provider limits, duplicate ids, strict parsing of a mixed batch, OpenRouter extras, every unusable-body case, +HTTP failure classification, bearer/endpoint/model override, redaction, network failure, cancellation, `GetService`, +feature-usage marking). The feature-registry validation test covers the new index. + +## Consequences + +### Positive + +- MAF gains a genuinely different inference capability at an existing seam without coupling Core to a vendor. +- One contract for every consumer today (loop evaluator, AgentEval, future selectors); deletion rather than + migration when MEAI ships, because the shape is the upstream proposal's. +- `AIJudgeLoopEvaluator` and `DecisionLoopEvaluator` stay semantically distinct. +- Existing evaluator ordering yields a cost-reducing cascade with no new orchestration engine. +- Probabilities remain available to policy instead of being flattened into generated JSON. +- Evaluation frameworks can reuse the same pattern (a probability source behind a callback) without MAF internals. + +### Negative / trade-offs + +- Another concept for developers to understand next to `IChatClient`. +- A separate decision provider is another external data boundary. +- Model-reported probabilities require workload-specific calibration and thresholds. +- Not all inputs have a lossless JSON representation; non-text requests need a `StateFactory`. +- Until MEAI ships, MAF carries a contract it does not intend to own; the experimental attribute and the verbatim + mirroring bound that cost, but a divergent MEAI review outcome would still force a rename for early adopters. +- Neutral: some workloads remain better served by deterministic code or an `IChatClient` judge. + +## Pros and Cons of the Options + +### Option 1: `IChatClient` + structured output + +- Good: broad existing integration; `AIJudgeLoopEvaluator` reusable unchanged. +- Bad: falsely advertises chat semantics; loses probability/distribution semantics; conflates syntactic structure with + a distinct capability; encourages prose expectations a decision model does not satisfy. + +### Option 2: MAF-specific public `IDecisionClient` + +- Good: no waiting on MEAI; MAF controls the shape. +- Bad: decision inference is not agent-specific; duplicates a lower-layer abstraction; incompatible contracts across + frameworks; avoidable public-API migration cost later. + +### Option 3: Jev/TypeSafe directly in MAF Core + +- Good: fast and ergonomic for one provider. +- Bad: vendor coupling; `Noul`/`System One` vocabulary leaks into the framework; parallel concepts for every provider. + +### Option 4: application-owned only + +- Good: no new Core types; existing extension points suffice for prototypes. +- Bad: every application rebuilds projection, question, threshold, failure policy, security docs; inconsistent seams; + users may wrap decision providers as chat clients for convenience. + +### Option 5: fully gated on MEAI + +- Good: purest layering. +- Bad: not implementable today; `dotnet/extensions#7764` is untriaged with no maintainer signal; blocks samples, + tests, calibration work, and the AgentEval integration indefinitely. + +### Option 6: MEAI owns the future abstraction; evaluator-scoped callback now (superseded) + +- Good: implementable today; no abstraction at all in MAF; follows the `DelegateLoopEvaluator` precedent. +- Bad: one callback per consumer does not scale to tool/Skill selection, routing, or workflows; Choice and Score are + unusable; external consumers such as AgentEval get nothing to reference and invent their own contract. + +### Option 7: experimental reference copy of the MEAI proposal in Abstractions; provider package for Jev (chosen) + +- Good: implementable and usable today; one contract for all consumers; verbatim mirroring makes the MEAI transition a + deletion; provider stays out of Core; follows the repository's provider-package precedent; gives `dotnet/extensions#7764` + a working implementation, a provider, and consumers as evidence. +- Neutral: MAF temporarily carries a contract it does not intend to own. +- Bad: if MEAI's review lands a materially different shape, early adopters face a rename; developers must understand + generation vs. decision inference; a second provider adds operational complexity. + +## Cross-SDK considerations + +The conceptual rule is cross-SDK: `decision model != chat model`, `provider transport != orchestration policy`, +`authorization != probabilistic selection`. Python may expose an analogous decision client through its provider +ecosystem (`microsoft/agent-framework#8556`) without mirroring the .NET types. + +## Related ADRs + +| ADR | Relationship | +| --- | --- | +| [ADR-0002 Agent Tools](0002-agent-tools.md) | Future tool shortlisting operates only over already-available/authorized tools. | +| [ADR-0003 Agent OpenTelemetry Instrumentation](0003-agent-opentelemetry-instrumentation.md) | Inference telemetry stays with the inference component; MAF adds orchestration-level facts only. | +| [ADR-0006 User Approval](0006-userapproval.md) | Decision inference never grants or bypasses approval. | +| [ADR-0014 Feature Collections](0014-feature-collections.md) / [ADR-0017 Additional Properties](0017-agent-additional-properties.md) | Future per-run decision policy should reuse existing extension mechanisms. | +| [ADR-0015 AgentRunContext](0015-agent-run-context.md) | Reuse run context rather than a parallel context abstraction. | +| [ADR-0021 Provider-Leading Client Design](0021-provider-leading-clients.md) | Provider transports stay out of core abstractions. | +| [ADR-0023 Evaluation Architecture](0023-foundry-evals-integration.md) | Evaluators may consume decision inference; it does not replace evaluation abstractions. | +| [ADR-0024 Prompt Injection Defense](0024-prompt-injection-defense.md) / [ADR-0035 Agent-Hooks Enforcement](0035-dotnet-agent-hooks-enforcement.md) | Decision models are signals; deterministic enforcement remains authoritative. | +| [ADR-0036 Structured Output](0036-structured-output.md) | Most important semantic comparison; not superseded. | +| [ADR-0037 Agent Skills](0037-agent-skills-design.md) | Future Skill shortlisting selects among trusted Skills only. | +| [ADR-0038 CodeAct Integration](0038-codeact-integration.md) | Precedent for optional dependencies outside core and reuse of context-provider seams. | + +## Related issues and external references + +- [microsoft/agent-framework#8545](https://github.com/microsoft/agent-framework/issues/8545) — .NET decision-model integration request. +- [microsoft/agent-framework#8556](https://github.com/microsoft/agent-framework/issues/8556) — Python Jev request. +- [microsoft/agent-framework#8562](https://github.com/microsoft/agent-framework/issues/8562) — this proposal: .NET `IDecisionClient` reference implementation, `DecisionLoopEvaluator`, and the TypeSafe provider package. +- [dotnet/extensions#7764](https://github.com/dotnet/extensions/issues/7764) — MEAI provider-neutral decision abstraction proposal. +- [TypeSafe documentation](https://docs.typesafe.ai/) — API: `POST /v1/systemone`; primitives `noul`/`choice`/`score`; models `jev-1.13.0`, `jev-latest`, `jev-preview`. + +## Implementation sequence + +1. **This PR**: ADR, the `IDecisionClient` contract in Abstractions, `DecisionLoopEvaluator` + options + failure + behavior, the `Microsoft.Agents.AI.TypeSafe` provider, unit tests for all three, PublicAPI baselines, feature + index 75, `Harness_Step06_DecisionLoop` sample (loop, cascade, direct mixed batch), Harness README updates. +2. **Upstream**: comment on `dotnet/extensions#7764` with this implementation as evidence; when MEAI lands the + abstraction, delete the Abstractions copy and retarget consumers and the provider. +3. **Calibration**: use the sample's probability output (and AgentEval) to compare `DecisionLoopEvaluator` vs. + `AIJudgeLoopEvaluator` vs. human labels on real loops; track false-stop rate separately. +4. **Tool-selection experiment** via `AIContextProvider`; **Skill-selection experiment** via `AgentSkillsProvider`. +5. **Generic selector** only if (4) demonstrates a stable shared shape; separate ADR. + +## Revisit criteria + +1. MEAI accepts, changes, or rejects a provider-neutral decision abstraction. +2. Multiple providers show incompatible semantics that make a shared shape misleading. +3. Selection use cases need a generic abstraction not expressible through existing seams. +4. Calibration shows model-reported probabilities are not useful enough for runtime policy. +5. Cross-SDK experience shows unacceptable conceptual divergence. + +## One-sentence design principle + +> **Let generative models generate, let deterministic policy authorize, and let decision models make bounded +> probabilistic judgments where a full generative call is unnecessary.** diff --git a/docs/specs/feature-usage-bit-registry.md b/docs/specs/feature-usage-bit-registry.md index 4f7ee4cd937..fc8fb0e3b47 100644 --- a/docs/specs/feature-usage-bit-registry.md +++ b/docs/specs/feature-usage-bit-registry.md @@ -248,7 +248,8 @@ only to approved first-party endpoints. | 72 | `local_codeact` | Local Python CodeAct provider | `Microsoft.Agents.AI.LocalCodeAct.LocalCodeActProvider` | | 73 | `hosting.a2a` | A2A hosting endpoints | `Microsoft.AspNetCore.Builder.A2AEndpointRouteBuilderExtensions.MapA2AJsonRpc` | | 74 | `hosting.openai` | OpenAI-compatible hosting endpoints | `Microsoft.AspNetCore.Builder.MicrosoftAgentAIHostingOpenAIEndpointRouteBuilderExtensions.MapOpenAIResponses` | -| 75–127 | _reserved_ | future packages | — | +| 75 | `typesafe` | TypeSafe System One decision client | `Microsoft.Agents.AI.TypeSafe.TypeSafeDecisionClient` | +| 76–127 | _reserved_ | future packages | — | ## Opt-out diff --git a/dotnet/agent-framework-dotnet.slnx b/dotnet/agent-framework-dotnet.slnx index 638d6096225..2097906a991 100644 --- a/dotnet/agent-framework-dotnet.slnx +++ b/dotnet/agent-framework-dotnet.slnx @@ -123,6 +123,7 @@ + @@ -632,6 +633,7 @@ + @@ -664,6 +666,7 @@ + diff --git a/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/ConsoleObservingDecisionClient.cs b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/ConsoleObservingDecisionClient.cs new file mode 100644 index 00000000000..d73c88c1c6f --- /dev/null +++ b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/ConsoleObservingDecisionClient.cs @@ -0,0 +1,37 @@ +// Copyright (c) Microsoft. All rights reserved. + +#pragma warning disable MAAI001 // Suppress experimental API warnings for Agents AI experiments. + +using System.Text.Json; +using Microsoft.Agents.AI; + +namespace Harness_Step06_DecisionLoop; + +/// +/// A delegating that prints every binary answer it sees, so the loop's decisions are +/// visible on the console. It is the decision-model counterpart of a logging chat client: the same shape works for +/// shadow-mode recording, calibration capture, or metrics. +/// +internal sealed class ConsoleObservingDecisionClient(IDecisionClient inner) : IDecisionClient +{ + public async Task GetResponseAsync(DecisionRequest request, DecisionOptions? options = null, CancellationToken cancellationToken = default) + { + DecisionResponse response = await inner.GetResponseAsync(request, options, cancellationToken); + + int iteration = request.State.ValueKind == JsonValueKind.Object && request.State.TryGetProperty("iteration", out JsonElement it) ? it.GetInt32() : 0; + foreach (DecisionQuestion question in request.Questions) + { + if (response.Answers.TryGetValue(question.Id, out DecisionAnswer? answer) && answer is BinaryDecisionAnswer binary && iteration > 0) + { + Console.WriteLine($"\n [Jev] iteration {iteration}: P({question.Id}) = {binary.TrueProbability:F3} (model {response.ModelId}, {response.Usage?.InputTokenCount} input tokens)"); + } + } + + return response; + } + + public object? GetService(Type serviceType, object? serviceKey = null) => + serviceKey is null && serviceType.IsInstanceOfType(this) ? this : inner.GetService(serviceType, serviceKey); + + public void Dispose() => inner.Dispose(); +} diff --git a/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/Harness_Step06_DecisionLoop.csproj b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/Harness_Step06_DecisionLoop.csproj new file mode 100644 index 00000000000..4335853b905 --- /dev/null +++ b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/Harness_Step06_DecisionLoop.csproj @@ -0,0 +1,20 @@ + + + + Exe + net10.0 + + enable + enable + + + + + + + + + + + + diff --git a/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/Program.cs new file mode 100644 index 00000000000..29cb1f352cb --- /dev/null +++ b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/Program.cs @@ -0,0 +1,256 @@ +// Copyright (c) Microsoft. All rights reserved. + +// This sample demonstrates the DecisionLoopEvaluator: a LoopEvaluator that asks a decision-oriented model, through the +// provider-neutral IDecisionClient, for the probability that the original request has been fully addressed, instead of +// asking a generative chat model for a verdict plus prose. TypeSafe's Jev (System One) is the decision model, reached +// through the Microsoft.Agents.AI.TypeSafe provider package; the core framework has no dependency on it. +// +// 1. Decision-only loop — Jev judges completion after every iteration; the loop stops once the model-reported +// probability reaches the completion threshold (DecisionLoopEvaluator). Because this agent answers one +// sub-question per turn, the demo supplies a StateFactory that sends Jev every response so far instead of the +// default latest-response-only projection. +// 2. Cheap-then-strong cascade — Jev runs first; while it says "incomplete" the expensive AI judge is skipped and +// the agent runs again; once Jev says "complete" the AI judge verifies (and can still send the loop back with +// a gap analysis). This uses existing LoopAgent ordering semantics: the first evaluator that asks to continue +// wins, and the loop stops only when every evaluator stops (DecisionLoopEvaluator + AIJudgeLoopEvaluator). +// 3. Direct decisions — one IDecisionClient call with a mixed batch of binary, choice, and score questions, to +// show the three primitives outside of a loop. +// +// The agent is deliberately instructed to answer only one sub-question per response so that the loop visibly needs +// several iterations to fully address a multi-part request. + +#pragma warning disable MAAI001 // Suppress experimental API warnings for Agents AI experiments. + +using System.ClientModel; +using System.Text.Json; +using Harness_Step06_DecisionLoop; +using Microsoft.Agents.AI; +using Microsoft.Agents.AI.TypeSafe; +using Microsoft.Extensions.AI; +using OpenAI; + +// Primary agent + generative judge: OpenAI, or any OpenAI-compatible chat-completions endpoint (for example Bitdeer +// with zai-org/GLM-5.3-Flash) when OPENAI_COMPATIBLE_ENDPOINT is set. +// The key is tied to the endpoint: a compatible endpoint uses only OPENAI_COMPATIBLE_API_KEY, and OpenAI uses only +// OPENAI_API_KEY, so a credential for one service is never sent to the other. +var chatEndpoint = Environment.GetEnvironmentVariable("OPENAI_COMPATIBLE_ENDPOINT"); +var chatApiKey = chatEndpoint is null + ? Environment.GetEnvironmentVariable("OPENAI_API_KEY") ?? throw new InvalidOperationException("OPENAI_API_KEY is not set (or set OPENAI_COMPATIBLE_ENDPOINT and OPENAI_COMPATIBLE_API_KEY).") + : Environment.GetEnvironmentVariable("OPENAI_COMPATIBLE_API_KEY") ?? throw new InvalidOperationException("OPENAI_COMPATIBLE_ENDPOINT is set but OPENAI_COMPATIBLE_API_KEY is not."); +var chatModel = Environment.GetEnvironmentVariable("OPENAI_COMPATIBLE_MODEL") + ?? Environment.GetEnvironmentVariable("OPENAI_CHAT_MODEL_NAME") + ?? "gpt-5.4-mini"; + +// Decision model: TypeSafe System One (Jev). TYPESAFE_API_KEY is required; TYPESAFE_MODEL optionally pins a version. +var typeSafeApiKey = Environment.GetEnvironmentVariable("TYPESAFE_API_KEY") ?? throw new InvalidOperationException("TYPESAFE_API_KEY is not set."); +var typeSafeModel = Environment.GetEnvironmentVariable("TYPESAFE_MODEL") ?? TypeSafeDecisionClientOptions.DefaultModelId; + +// The HarnessAgent pre-configures function invocation, per-service-call chat history persistence, and +// context-window compaction. These bounds size the in-loop compaction window. +const int MaxContextWindowTokens = 400_000; +const int MaxOutputTokens = 16_000; + +var openAIClient = chatEndpoint is null + ? new OpenAIClient(chatApiKey) + : new OpenAIClient(new ApiKeyCredential(chatApiKey), new OpenAIClientOptions { Endpoint = new Uri(chatEndpoint) }); +IChatClient CreateChatClient() => openAIClient.GetChatClient(chatModel).AsIChatClient(); + +// The provider client implements IDecisionClient. It is wrapped in a small delegating client (defined at the bottom) +// that prints every binary answer, so the loop's decisions are visible on the console. +using var jev = new ConsoleObservingDecisionClient(new TypeSafeDecisionClient(typeSafeApiKey, new() { ModelId = typeSafeModel })); +Console.WriteLine($"Primary model: {chatModel} ({chatEndpoint ?? "OpenAI"}) · Decision model: {typeSafeModel} (TypeSafe System One)"); + +const string Task = "Explain why the sky is blue, why sunsets are red, and why clouds are white."; + +await DecisionOnlyLoopAsync(); +await CascadeLoopAsync(); +await DirectDecisionsAsync(); + +// Pattern 1: Jev alone decides whether the request has been fully addressed. +async Task DecisionOnlyLoopAsync() +{ + Console.WriteLine("\n=== 1. Decision-only loop — Jev judges completion over all responses so far (threshold 0.90, max 5) ==="); + + AIAgent loopAgent = new LoopAgent( + CreateAnswerer(), + new DecisionLoopEvaluator( + jev, + new DecisionLoopEvaluatorOptions + { + // A false stop returns an incomplete answer while a false continue only costs one more iteration, + // so the threshold is deliberately high. It is application policy, not a property of the model. + CompletionThreshold = 0.90, + + // The default projection judges only the latest response (like AIJudgeLoopEvaluator). This agent + // spreads its answer across turns, so judge the accumulated responses instead. The factory keeps its + // per-run list on the LoopContext, which is what keeps the evaluator itself stateless and shareable. + StateFactory = CreateCumulativeState, + }), + new LoopAgentOptions { MaxIterations = 5 }); + + AgentResponse response = await StreamLoopAsync(loopAgent, Task); + Console.WriteLine($"\nFinal response:\n{response.Text}"); +} + +// Pattern 2: Jev first, strong generative judge only when Jev believes the work is complete. +async Task CascadeLoopAsync() +{ + Console.WriteLine("\n=== 2. Cheap-then-strong cascade — Jev, then AI judge verifies (max 6) ==="); + + int judgeCalls = 0; + var strongJudge = new AIJudgeLoopEvaluator(CreateChatClient()); + var countingJudge = new DelegateLoopEvaluator(async (context, cancellationToken) => + { + judgeCalls++; + Console.WriteLine(" [AI judge] Jev believes the work is complete; verifying with the generative judge..."); + LoopEvaluation verdict = await strongJudge.EvaluateAsync(context, cancellationToken); + Console.WriteLine(verdict.ShouldReinvoke ? " [AI judge] Not complete — sending gap analysis back to the agent." : " [AI judge] Confirmed complete."); + return verdict; + }); + + AIAgent loopAgent = new LoopAgent( + CreateAnswerer(), + [ + // Order matters: Jev goes first. When it says "incomplete" the loop continues immediately and the AI judge + // is never consulted. When it says "complete" it returns Stop, which only means "no continuation requested", + // so the AI judge runs next. A transient Jev failure (rate limit, overload, outage) defers to the AI judge; + // a permanent one (rejected key, invalid request) still surfaces, because hiding it would only cost money. + new DecisionLoopEvaluator( + jev, + new DecisionLoopEvaluatorOptions + { + CompletionThreshold = 0.90, + TransientFailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator, + }), + countingJudge, + ], + new LoopAgentOptions { MaxIterations = 6 }); + + AgentResponse response = await StreamLoopAsync(loopAgent, Task); + Console.WriteLine($"\nFinal response:\n{response.Text}"); + Console.WriteLine($"\nGenerative judge calls: {judgeCalls} (a plain AI-judge loop would have called it after every iteration)."); +} + +// Pattern 3: the three decision primitives in one batched call, outside of any loop. +async Task DirectDecisionsAsync() +{ + Console.WriteLine("\n=== 3. Direct decisions — one call, three primitives ==="); + + JsonElement state = JsonSerializer.SerializeToElement(new + { + ticket = "My flight was cancelled and nobody told me. I'm really frustrated. Can I get a refund?", + policy = "Cancelled flights are eligible for a full refund.", + }); + + var request = new DecisionRequest( + state, + [ + new BinaryDecisionQuestion("refund_requested", "Does the customer request a refund?"), + new ChoiceDecisionQuestion("request_type", "What is the customer's main request?", + [ + new("refund", "The customer wants money returned."), + new("rebooking", "The customer wants another flight."), + new("information", "The customer only wants information."), + ]), + new ScoreDecisionQuestion("frustration", "How frustrated does the customer appear?", + [ + new("Calm and neutral."), + new("Concerned but civil."), + new("Very angry or using strong language."), + ]), + ]); + + DecisionResponse response = await jev.GetResponseAsync(request); + + var refund = (BinaryDecisionAnswer)response.Answers["refund_requested"]; + var requestType = (ChoiceDecisionAnswer)response.Answers["request_type"]; + var frustration = (ScoreDecisionAnswer)response.Answers["frustration"]; + + Console.WriteLine($" model: {response.ModelId}, input tokens: {response.Usage?.InputTokenCount}"); + Console.WriteLine($" refund_requested: P(true) = {refund.TrueProbability:F3}"); + Console.WriteLine($" request_type: {requestType.SelectedChoice} (confidence {requestType.Confidence:F2}) — " + + string.Join(", ", requestType.Probabilities.Select(p => $"{p.Key} {p.Value:F2}"))); + Console.WriteLine($" frustration: score {frustration.Score:F2} on 0..2 (confidence {frustration.Confidence:F2}) — " + + string.Join(", ", frustration.Probabilities.OrderBy(p => p.Key).Select(p => $"level {p.Key} {p.Value:F2}"))); +} + +// Builds the state Jev judges in pattern 1: the original request plus every response produced so far in this run. +// Per-run state lives in LoopContext.AdditionalProperties (owned by the run), never on the evaluator. Whatever the +// factory returns is sent to the decision provider, so keep it to what the question needs. +static JsonElement CreateCumulativeState(LoopContext context) +{ + const string ResponsesKey = "Harness_Step06_DecisionLoop.ResponsesSoFar"; + if (context.AdditionalProperties.TryGetValue(ResponsesKey, out object? existing) && existing is List responses) + { + responses.Add(context.LastResponse.Text); + } + else + { + responses = [context.LastResponse.Text]; + context.AdditionalProperties[ResponsesKey] = responses; + } + + return JsonSerializer.SerializeToElement(new + { + originalRequest = context.InitialMessages.Select(m => new { role = m.Role.Value, text = m.Text }), + responsesSoFar = responses, + iteration = context.Iteration, + }); +} + +// Creates a lean HarnessAgent that answers one sub-question per turn so the loop has visible work to do. +AIAgent CreateAnswerer() => + CreateChatClient().AsHarnessAgent(new HarnessAgentOptions + { + Name = "answerer", + MaxContextWindowTokens = MaxContextWindowTokens, + MaxOutputTokens = MaxOutputTokens, + DisableAgentModeProvider = true, + DisableTodoProvider = true, + DisableFileMemory = true, + DisableWebSearch = true, + ChatOptions = new ChatOptions + { + Instructions = + """ + You are a helpful science explainer. The user may ask several questions at once. Answer exactly ONE + of the not-yet-answered questions per response, in two or three sentences, and then stop. When you + are asked to continue, answer the next unanswered question. Never answer more than one per response. + """, + MaxOutputTokens = MaxOutputTokens, + }, + }); + +// Streams a loop run to the console, marking each new inner run with a "--- run N ---" header so you can see when +// the LoopAgent re-invokes the inner agent. Each message is prefixed with "User:" or "Agent:" based on its role, so +// the loop's on-behalf-of feedback (User) is visually distinct from the agent's responses (Agent). +static async Task StreamLoopAsync(AIAgent loopAgent, string input) +{ + string? currentResponseId = null; + ChatRole? currentRole = null; + var runCount = 0; + var updates = new List(); + + await foreach (var update in loopAgent.RunStreamingAsync(input)) + { + if (update.ResponseId is { } responseId && responseId != currentResponseId) + { + currentResponseId = responseId; + currentRole = null; + Console.WriteLine($"\n--- run {++runCount} ---"); + } + + if (update.Role is { } role && role != currentRole) + { + currentRole = role; + var prefix = role == ChatRole.User ? "User" : role == ChatRole.Assistant ? "Agent" : role.Value; + Console.Write($"\n{prefix}: "); + } + + Console.Write(update.Text); + updates.Add(update); + } + + Console.WriteLine(); + return updates.ToAgentResponse(); +} diff --git a/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/README.md b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/README.md new file mode 100644 index 00000000000..d65f78cdc6c --- /dev/null +++ b/dotnet/samples/02-agents/Harness/Harness_Step06_DecisionLoop/README.md @@ -0,0 +1,117 @@ +# What this sample demonstrates + +This sample demonstrates **`DecisionLoopEvaluator`**, a `LoopEvaluator` for `LoopAgent` that judges loop completion with a +*decision-oriented* model instead of a generative one. After each iteration it asks a decision source a single bounded +yes/no question ("Has the agent fully addressed the user's original request?") and receives a **model-reported +probability**. The evaluator compares that probability with a configured completion threshold: at or above it the +evaluator stops requesting iterations, below it the loop continues with a deterministic feedback message. + +TypeSafe's **Jev** ("System One" model) is the decision model used here. Jev does not generate text: it evaluates a +piece of state against typed questions and returns probabilities directly, at a fraction of the latency and cost of a +generative judge. It is reached through the provider-neutral **`IDecisionClient`** abstraction +(`Microsoft.Agents.AI.Abstractions`, experimental, mirroring the shape proposed for `Microsoft.Extensions.AI` in +[dotnet/extensions#7764](https://github.com/dotnet/extensions/issues/7764)) and its TypeSafe implementation, +`TypeSafeDecisionClient` in the **`Microsoft.Agents.AI.TypeSafe`** package. The core framework has **no dependency** on +Jev or TypeSafe: `DecisionLoopEvaluator` only sees an `IDecisionClient`. The sample wraps the client in a small delegating +[`ConsoleObservingDecisionClient`](./ConsoleObservingDecisionClient.cs) that prints every probability. + +## Looping patterns showcased + +| # | Pattern | Evaluators | Notes | +| --- | --- | --- | --- | +| 1 | Decision-only loop | `DecisionLoopEvaluator` + `StateFactory` | Jev judges completion after every iteration. The loop stops once `P(complete) >= 0.90`. Because the agent spreads its answer across turns, a `StateFactory` sends Jev the original request plus **every response so far** (kept per run in `LoopContext.AdditionalProperties`). The console prints the probability, model id, and input tokens for each iteration. | +| 2 | Cheap-then-strong cascade | `DecisionLoopEvaluator` → `AIJudgeLoopEvaluator` | Uses the default latest-response projection for both evaluators, so they judge the same evidence. Jev runs first. While it says "incomplete" the loop continues immediately and the expensive generative judge is skipped. Once Jev says "complete", the AI judge verifies and can still send the loop back with a gap analysis. No new cascade abstraction is needed: this is existing `LoopAgent` ordering (first evaluator to request continuation wins; the loop stops only when every evaluator stops). | +| 3 | Direct decisions | `IDecisionClient` | One batched call with a binary, a choice, and a score question against the same state, printing the probability, the selected choice with its distribution, and the weighted score with its level distribution. | + +The agent is deliberately instructed to answer only **one** of the three sub-questions per response, so each loop +visibly needs several iterations. `MaxIterations` caps every loop so it always terminates. + +### How `DecisionLoopEvaluator` differs from `AIJudgeLoopEvaluator` + +| | `AIJudgeLoopEvaluator` | `DecisionLoopEvaluator` | +| --- | --- | --- | +| Backed by | generative `IChatClient` | any `IDecisionClient` (decision model, or a test double) | +| Returns | verdict **and** free-form gap analysis | probability only | +| Feedback on continue | judge's gap analysis | deterministic `ContinueFeedbackMessage` | +| Threshold | none (boolean verdict) | `CompletionThreshold`, application policy | +| On provider failure | throws | `FailureBehavior`: `Throw` (default), `Continue`, or `DeferToNextEvaluator`; `TransientFailureBehavior` applies separately to rate limits, overload, and outages (`DecisionClientException.IsTransient`) | + +Neither replaces the other; pattern 2 shows why they compose well. + +## Prerequisites + +1. An OpenAI API key, **or** an API key for any OpenAI-compatible chat-completions endpoint (for example Bitdeer with + `zai-org/GLM-5.3-Flash`), for the primary agent and the generative judge. +2. A TypeSafe API key for Jev (https://typesafe.ai). + +## Environment Variables + +```bash +# Primary agent + AI judge — option A: OpenAI +export OPENAI_API_KEY="sk-..." +# Optional: chat model (defaults to gpt-5.4-mini) +export OPENAI_CHAT_MODEL_NAME="gpt-5.4-mini" + +# Primary agent + AI judge — option B: any OpenAI-compatible endpoint (example: Bitdeer + GLM-5.3-Flash) +export OPENAI_COMPATIBLE_ENDPOINT="https://api-inference.bitdeer.ai/v1" +export OPENAI_COMPATIBLE_API_KEY="..." +export OPENAI_COMPATIBLE_MODEL="zai-org/GLM-5.3-Flash" + +# Required: decision model (TypeSafe System One / Jev) +export TYPESAFE_API_KEY="ts-..." +# Optional: Jev model id (defaults to jev-latest; pin e.g. jev-1.13.0 for reproducible runs) +export TYPESAFE_MODEL="jev-latest" +``` + +## Running the Sample + +```bash +cd dotnet +dotnet run --project samples/02-agents/Harness/Harness_Step06_DecisionLoop +``` + +> OpenAI-compatible does not mean behaviorally identical: structured output, tool calling, and streaming details vary +> by provider. `AIJudgeLoopEvaluator` falls back to text verdict markers when a provider ignores structured output. + +## What to Expect + +Each loop is executed with `RunStreamingAsync`, so output is printed live and every re-invocation of the inner agent is +marked with a `--- run N ---` header. After each run a `[Jev] iteration N: P(complete) = 0.xxx` line shows the +model-reported probability. In pattern 1 the first runs score low (one or two of three questions answered) and the loop +stops once the accumulated responses cover the whole request, typically after the third run. In pattern 2 the agent +only stops once a single response stands on its own as complete (usually a recap on the fourth run); you additionally +see `[AI judge] ...` lines only when Jev believed the work was complete, and a final count of generative judge calls, +which is smaller than the number of iterations. + +Observed on a reference run (Jev `jev-1.13.0`, Bitdeer `zai-org/GLM-5.3-Flash`): pattern 1 produced 0.04, 0.07, 0.97 and +stopped after three runs; pattern 2 produced 0.04, 0.05, 0.05, 0.95 across four runs and called the generative judge +once, which confirmed completion; pattern 3 returned `refund_requested` P=0.99, `request_type` = refund (1.00), and a +frustration score of 1.09 on 0..2 with P(level 1) = 0.91. + +## Design notes + +- **Thresholds are policy, not model semantics.** A false stop returns an incomplete answer while a false continue only + costs an extra iteration, so the sample uses a deliberately high threshold (0.90). Calibrate it for your workload; + `P = 0.90` is a model-reported value, not an empirical accuracy guarantee. +- **Minimal state.** By default the evaluator sends only the text of the original request messages, the latest response + text, and the iteration number. Requests containing non-text content (images, data) are rejected rather than silently + flattened; supply `DecisionLoopEvaluatorOptions.StateFactory` to judge those. Jev accepts text-only state and limits a + request to 64k tokens (32k for state). +- **Latest response vs. accumulated responses.** The default projection, like `AIJudgeLoopEvaluator`, judges the latest + response against the original request. That is right for agents that produce a complete answer each turn. For agents + that build the answer incrementally, use a `StateFactory` that accumulates (pattern 1); keep the accumulated list on + `LoopContext.AdditionalProperties` so the evaluator stays stateless and shareable, and mind the provider's state limit. +- **Failure is not "incomplete".** A provider error is handled by `FailureBehavior`, never converted into a low + probability. Pattern 2 sets `TransientFailureBehavior = DeferToNextEvaluator` so the AI judge decides when Jev is + momentarily unavailable, while a permanent failure (rejected key, invalid request) still surfaces. Pass an + `ILoggerFactory` to the evaluator to get one debug line per decision (iteration, probability, threshold, outcome) and a + warning whenever a failure policy is applied; the projected state is never logged. + +## Security Considerations + +`DecisionLoopEvaluator` is an explicit opt-in to sending the projected original request and the agent's latest response +to an additional external inference boundary on every iteration. A compromised decision endpoint could exfiltrate that +data or return manipulated probabilities that prematurely end (or needlessly prolong) the loop. Only configure a decision +source you trust as much as the primary model, keep the state projection minimal, and never use the probability as an +authorization or approval signal. Pattern 2 additionally uses `AIJudgeLoopEvaluator`; see +[Harness_Step05_Loop](../Harness_Step05_Loop/README.md#security-considerations) for its considerations. diff --git a/dotnet/samples/02-agents/Harness/README.md b/dotnet/samples/02-agents/Harness/README.md index 9417ba42952..e41437e40e8 100644 --- a/dotnet/samples/02-agents/Harness/README.md +++ b/dotnet/samples/02-agents/Harness/README.md @@ -10,6 +10,7 @@ Samples demonstrating the [Harness AIContextProviders](../../../src/Microsoft.Ag | [Harness_Step02_Research_WithBackgroundAgents](./Harness_Step02_Research_WithBackgroundAgents/README.md) | Using BackgroundAgentsProvider to delegate stock price lookups to a web-search background agent concurrently | | [Harness_Step03_DataProcessing](./Harness_Step03_DataProcessing/README.md) | Using FileAccessProvider to give an agent access to CSV data files for reading, analysis, and output generation | | [Harness_Step05_Loop](./Harness_Step05_Loop/README.md) | Wrapping a HarnessAgent with the LoopAgent decorator to re-invoke it until a configured LoopEvaluator (completion marker, predicate, AI judge, or approval-aware loop) decides to stop | +| [Harness_Step06_DecisionLoop](./Harness_Step06_DecisionLoop/README.md) | Using DecisionLoopEvaluator with an IDecisionClient (TypeSafe Jev via the Microsoft.Agents.AI.TypeSafe package) to judge loop completion from a model-reported probability, alone and as a cheap-then-strong cascade in front of AIJudgeLoopEvaluator, plus a direct mixed batch of binary, choice, and score decisions | ## Build your own claw blog series @@ -31,6 +32,8 @@ corresponding types) before enabling them in production: [Harness_Step02_Research_WithBackgroundAgents](./Harness_Step02_Research_WithBackgroundAgents/README.md)). - **`AIJudgeLoopEvaluator`** (used by `LoopAgent`) — sends conversation content to a second, external judge chat client (see [Harness_Step05_Loop](./Harness_Step05_Loop/README.md)). +- **`DecisionLoopEvaluator`** (used by `LoopAgent`) — sends the projected request and latest response to an + external `IDecisionClient` on every iteration (see [Harness_Step06_DecisionLoop](./Harness_Step06_DecisionLoop/README.md)). - **`AgentSkillsProvider`** with external skill sources (e.g. `UseMcpSkills`) — loads skill content, and potentially scripts, from a remote source (see [AgentSkills samples](../AgentSkills/Agent_Step06_McpBasedSkills/README.md)). diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionAnswer.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionAnswer.cs new file mode 100644 index 00000000000..7f63cde613e --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionAnswer.cs @@ -0,0 +1,42 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// The answer to a : the model-reported probability that the proposition is true. +/// +/// +/// The probability is the whole answer. There is no separate confidence: P(false) = 1 - P(true). A value near 1 +/// supports true, a value near 0 supports false, and a value near 0.5 is uncertain. It is a model-reported probability, +/// not an empirically calibrated accuracy. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class BinaryDecisionAnswer : DecisionAnswer +{ + private double _trueProbability; + + /// Initializes a new instance of the class. + public BinaryDecisionAnswer() + { + } + + /// Initializes a new instance of the class. + /// The probability that the proposition is true. + /// is not finite or is outside 0 to 1. + public BinaryDecisionAnswer(double trueProbability) + { + this.TrueProbability = trueProbability; + } + + /// Gets or sets the probability, from 0 to 1, that the proposition is true. + /// The value is not finite or is outside 0 to 1. + public double TrueProbability + { + get => this._trueProbability; + set => this._trueProbability = EnsureProbability(value, nameof(value)); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionCriteria.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionCriteria.cs new file mode 100644 index 00000000000..9c1cbe75356 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionCriteria.cs @@ -0,0 +1,20 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Describes what a true and a false answer to a mean, to sharpen the semantic +/// boundary between them. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class BinaryDecisionCriteria +{ + /// Gets or sets a description of what a true answer means. + public string? TrueDescription { get; set; } + + /// Gets or sets a description of what a false answer means. + public string? FalseDescription { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionQuestion.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionQuestion.cs new file mode 100644 index 00000000000..c2bad53ec6f --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/BinaryDecisionQuestion.cs @@ -0,0 +1,32 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// A question whose answer is the probability that a proposition is true for the state, reported as a +/// . +/// +/// +/// Phrase the proposition so that a high probability means "yes". The answer has no separate confidence value: the +/// probability already describes the two-outcome distribution (P(false) = 1 - P(true)). Use +/// to sharpen the boundary between the two outcomes when it is subtle. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class BinaryDecisionQuestion : DecisionQuestion +{ + /// + /// Initializes a new instance of the class. + /// + /// The identifier of the question, unique within a request. + /// The proposition to evaluate against the state. + public BinaryDecisionQuestion(string id, string instructions) + : base(id, instructions) + { + } + + /// Gets or sets optional descriptions of what a true and a false answer mean. + public BinaryDecisionCriteria? Criteria { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ChoiceDecisionAnswer.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ChoiceDecisionAnswer.cs new file mode 100644 index 00000000000..553f376a956 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ChoiceDecisionAnswer.cs @@ -0,0 +1,59 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// The answer to a : the selected choice and the probability distribution over +/// every choice. +/// +/// +/// is the fundamental interoperable value: it contains an entry for every requested choice, +/// each between 0 and 1, summing to approximately 1. is one of the requested choices. +/// is an optional provider-derived summary of the distribution and is not comparable across +/// providers unless documented. The distribution supplied at construction is validated (non-blank keys, values within +/// 0 to 1); the collection itself stays mutable, matching the proposed Microsoft.Extensions.AI shape, so +/// producers that mutate it afterwards own its invariants. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class ChoiceDecisionAnswer : DecisionAnswer +{ + private double? _confidence; + private string _selectedChoice; + + /// Initializes a new instance of the class. + /// The name of the selected choice. + /// The probability of each choice, keyed by choice name. + /// or is . + /// is empty or whitespace. + public ChoiceDecisionAnswer(string selectedChoice, IDictionary probabilities) + { + this._selectedChoice = Throw.IfNullOrWhitespace(selectedChoice); + this.Probabilities = EnsureDistribution(probabilities, nameof(probabilities)); + } + + /// Gets or sets the name of the selected choice. + /// The value is . + /// The value is empty or whitespace. + public string SelectedChoice + { + get => this._selectedChoice; + set => this._selectedChoice = Throw.IfNullOrWhitespace(value); + } + + /// Gets the probability of each choice, keyed by . + public IDictionary Probabilities { get; } + + /// Gets or sets an optional provider-derived confidence in , from 0 to 1. + /// The value is not finite or is outside 0 to 1. + public double? Confidence + { + get => this._confidence; + set => this._confidence = EnsureOptionalProbability(value, nameof(value)); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ChoiceDecisionQuestion.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ChoiceDecisionQuestion.cs new file mode 100644 index 00000000000..b77e92ea02b --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ChoiceDecisionQuestion.cs @@ -0,0 +1,59 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// A question that selects exactly one member of a caller-defined, unordered set of choices, reported as a +/// carrying the selection and a probability distribution over every choice. +/// +/// +/// Choice names must be unique within the question. Their descriptions provide semantic criteria and impose no +/// ordering; for an ordered scale use . A choice needs at least two members. +/// Providers may impose an upper limit; that limit is not part of this contract. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class ChoiceDecisionQuestion : DecisionQuestion +{ + /// + /// Initializes a new instance of the class. + /// + /// The identifier of the question, unique within a request. + /// The classification question to evaluate against the state. + /// The choices to select from. + /// is or contains a element. + /// has fewer than two members or contains duplicate names. + public ChoiceDecisionQuestion(string id, string instructions, IEnumerable choices) + : base(id, instructions) + { + _ = Throw.IfNull(choices); + + var list = new List(); + var names = new HashSet(StringComparer.Ordinal); + foreach (DecisionChoice choice in choices) + { + _ = Throw.IfNull(choice, nameof(choices)); + if (!names.Add(choice.Name)) + { + throw new ArgumentException($"Choice names must be unique within a question; '{choice.Name}' appears more than once.", nameof(choices)); + } + + list.Add(choice); + } + + if (list.Count < 2) + { + throw new ArgumentException("A choice question needs at least two choices.", nameof(choices)); + } + + this.Choices = list; + } + + /// Gets the choices to select from. + public IList Choices { get; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionAnswer.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionAnswer.cs new file mode 100644 index 00000000000..4f729dbd1b8 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionAnswer.cs @@ -0,0 +1,66 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Extensions.AI; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// Represents the typed answer to a . The concrete kinds are +/// , , and , each +/// matching the kind of question that produced it. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public abstract class DecisionAnswer +{ + /// Initializes a new instance of the class. + protected DecisionAnswer() + { + } + + /// Gets or sets the raw, provider-specific representation of the answer, when available. + public object? RawRepresentation { get; set; } + + /// Gets or sets any additional, provider-specific properties associated with the answer. + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } + + /// Validates that a probability-like value is finite and within 0 to 1 inclusive. + /// The value to validate. + /// The name of the member being set. + /// . + /// is not finite or is outside 0 to 1. + private protected static double EnsureProbability(double value, string paramName) + { + if (double.IsNaN(value) || double.IsInfinity(value) || value < 0 || value > 1) + { + throw new ArgumentOutOfRangeException(paramName, value, "The value must be a finite number between 0 and 1 inclusive."); + } + + return value; + } + + /// Validates that an optional confidence value, when present, is finite and within 0 to 1 inclusive. + private protected static double? EnsureOptionalProbability(double? value, string paramName) => + value is null ? null : EnsureProbability(value.Value, paramName); + + /// Validates every entry of a distribution supplied at construction: non-blank keys and unit probabilities. + private protected static IDictionary EnsureDistribution(IDictionary probabilities, string paramName) + { + _ = Throw.IfNull(probabilities, paramName); + foreach (KeyValuePair pair in probabilities) + { + if (pair.Key is null || (pair.Key is string key && string.IsNullOrWhiteSpace(key))) + { + throw new ArgumentException("Distribution keys must not be null or blank.", paramName); + } + + _ = EnsureProbability(pair.Value, paramName); + } + + return probabilities; + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionChoice.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionChoice.cs new file mode 100644 index 00000000000..ba10ba9fc50 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionChoice.cs @@ -0,0 +1,34 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// One member of a 's choice set. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionChoice +{ + /// + /// Initializes a new instance of the class. + /// + /// The name of the choice. It is the key of the choice in . + /// An optional description of what the choice means. + /// is . + /// is empty or whitespace. + public DecisionChoice(string name, string? description = null) + { + this.Name = Throw.IfNullOrWhitespace(name); + this.Description = description; + } + + /// Gets the name of the choice. + public string Name { get; } + + /// Gets or sets an optional description of what the choice means. + public string? Description { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientException.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientException.cs new file mode 100644 index 00000000000..0bda65f695c --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientException.cs @@ -0,0 +1,67 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// The exception thrown by an when a call did not yield usable answers. +/// +/// +/// An operational failure is not a semantic answer. Implementations throw this exception instead of returning a +/// fabricated probability, and never include credentials in . +/// tells the caller whether a retry with the same request could reasonably succeed; any retry should remain observable +/// (for example in latency and cost accounting) rather than hidden inside the client. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public class DecisionClientException : Exception +{ + /// Initializes a new instance of the class. + public DecisionClientException() + : this(DecisionFailureKind.Unknown, "The decision client call failed.") + { + } + + /// Initializes a new instance of the class. + /// The message that describes the error. + public DecisionClientException(string message) + : this(DecisionFailureKind.Unknown, message) + { + } + + /// Initializes a new instance of the class. + /// The message that describes the error. + /// The exception that is the cause of the current exception. + public DecisionClientException(string message, Exception? innerException) + : this(DecisionFailureKind.Unknown, message, statusCode: null, innerException) + { + } + + /// Initializes a new instance of the class. + /// The failure classification. + /// The message that describes the error. It must not contain credentials. + /// The HTTP status the provider returned, when one was. + /// The exception that is the cause of the current exception, if any. + public DecisionClientException(DecisionFailureKind kind, string message, int? statusCode = null, Exception? innerException = null) + : base(message, innerException) + { + this.Kind = kind; + this.StatusCode = statusCode; + } + + /// Gets the failure classification. + public DecisionFailureKind Kind { get; } + + /// Gets the HTTP status the provider returned, or when the failure happened before a response. + public int? StatusCode { get; } + + /// + /// Gets a value indicating whether a retry with the same request could reasonably succeed: for + /// rate limiting, overload, and provider-side failures; for authentication, request + /// validation, and unusable responses. + /// + public bool IsTransient => + this.Kind is DecisionFailureKind.RateLimited or DecisionFailureKind.Overloaded or DecisionFailureKind.ProviderUnavailable; +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientExtensions.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientExtensions.cs new file mode 100644 index 00000000000..cd12a0ab138 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientExtensions.cs @@ -0,0 +1,62 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.Text.Json; +using System.Text.Json.Serialization.Metadata; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// Provides extension methods for . +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public static class DecisionClientExtensions +{ + /// + /// Evaluates against strongly typed , serializing the state with + /// the supplied so that the client's boundary stays + /// trimming and Native AOT friendly. + /// + /// The type of the state. + /// The decision client. + /// The state to evaluate. + /// The serialization metadata for . + /// The questions to ask about the state. + /// Optional per-call options. + /// The to monitor for cancellation requests. + /// One answer per question, keyed by . + /// , , or is . + public static Task GetResponseAsync( + this IDecisionClient client, + TState state, + JsonTypeInfo stateTypeInfo, + IEnumerable questions, + DecisionOptions? options = null, + CancellationToken cancellationToken = default) + { + _ = Throw.IfNull(client); + _ = Throw.IfNull(stateTypeInfo); + _ = Throw.IfNull(questions); + + JsonElement element = JsonSerializer.SerializeToElement(state, stateTypeInfo); + return client.GetResponseAsync(new DecisionRequest(element, questions), options, cancellationToken); + } + + /// Asks the client for an object of the specified type. + /// The type of the object to be retrieved. + /// The decision client. + /// An optional key that can be used to help identify the target service. + /// The found object, otherwise . + /// is . + public static TService? GetService(this IDecisionClient client, object? serviceKey = null) + { + _ = Throw.IfNull(client); + return client.GetService(typeof(TService), serviceKey) is TService service ? service : default; + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientMetadata.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientMetadata.cs new file mode 100644 index 00000000000..c79683e1acd --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionClientMetadata.cs @@ -0,0 +1,36 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Provides metadata about an , retrievable through . +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public class DecisionClientMetadata +{ + /// + /// Initializes a new instance of the class. + /// + /// The name of the decision provider, if applicable. + /// The URL for accessing the decision provider, if applicable. + /// The identifier of the model used by default, if applicable. + public DecisionClientMetadata(string? providerName = null, Uri? providerUri = null, string? defaultModelId = null) + { + this.ProviderName = providerName; + this.ProviderUri = providerUri; + this.DefaultModelId = defaultModelId; + } + + /// Gets the name of the decision provider. + public string? ProviderName { get; } + + /// Gets the URL for accessing the decision provider. + public Uri? ProviderUri { get; } + + /// Gets the identifier of the model used by default when a request does not specify one. + public string? DefaultModelId { get; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionFailureKind.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionFailureKind.cs new file mode 100644 index 00000000000..d8ac11d3a26 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionFailureKind.cs @@ -0,0 +1,38 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Classifies why an call failed, so a caller can tell a retryable condition from a +/// permanent one. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public enum DecisionFailureKind +{ + /// The failure could not be classified, or happened locally before a response (for example a network error). + Unknown = 0, + + /// The credentials were missing or rejected. Not transient. + Authentication = 1, + + /// The request failed the provider's validation. Not transient: the same request will fail again. + InvalidRequest = 2, + + /// The provider rate-limited the caller. Transient; back off before retrying. + RateLimited = 3, + + /// The provider is temporarily overloaded. Transient. + Overloaded = 4, + + /// The provider failed on its side or was unreachable. Transient. + ProviderUnavailable = 5, + + /// + /// The provider answered successfully but the body was unusable: a missing answer, an answer of the wrong kind, a + /// probability outside 0 to 1, or unparseable content. Not transient, and never converted into an answer. + /// + InvalidResponse = 6, +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionOptions.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionOptions.cs new file mode 100644 index 00000000000..e5d5ffee288 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionOptions.cs @@ -0,0 +1,50 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Extensions.AI; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Represents per-call options for an . +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public class DecisionOptions +{ + /// Gets or sets the model identifier to use for this call, overriding the client's default. + /// + /// Pin a versioned identifier when reproducibility matters (tests, calibration, benchmarks); a moving alias can + /// change what answers between runs. + /// + public string? ModelId { get; set; } + + /// Gets or sets any additional, provider-specific properties associated with the options. + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } + + /// + /// Gets or sets a callback responsible for creating the raw, provider-specific representation of these options. + /// + /// + /// The underlying implementation may have its own representation of options. When + /// is invoked with a , that implementation + /// may convert the provided options into its own representation in order to use it while performing the operation. + /// For situations where a consumer knows which concrete is being used and how it + /// represents options, a new instance of that provider-specific type may be returned by this callback for the + /// to use instead of creating a new instance. The + /// implementation should then apply any relevant settings from the on top of it. + /// + public Func? RawRepresentationFactory { get; set; } + + /// Produces a clone of the current instance. + /// A clone of the current instance. + /// The clone is shallow: the dictionary is copied, its values are not. + public virtual DecisionOptions Clone() => + new() + { + ModelId = this.ModelId, + AdditionalProperties = this.AdditionalProperties?.Clone(), + RawRepresentationFactory = this.RawRepresentationFactory, + }; +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionQuestion.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionQuestion.cs new file mode 100644 index 00000000000..71357a7d843 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionQuestion.cs @@ -0,0 +1,40 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics.CodeAnalysis; +using Microsoft.Extensions.AI; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// Represents a bounded, typed question that an evaluates against a +/// . The concrete kinds are , +/// , and . +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public abstract class DecisionQuestion +{ + /// + /// Initializes a new instance of the class. + /// + /// The identifier of the question, unique within a request. Answers are keyed by it. + /// What is being asked about the state. + /// or is . + /// or is empty or whitespace. + protected DecisionQuestion(string id, string instructions) + { + this.Id = Throw.IfNullOrWhitespace(id); + this.Instructions = Throw.IfNullOrWhitespace(instructions); + } + + /// Gets the identifier of the question. Answers in a are keyed by it. + /// Identifiers are application correlation keys; a provider must not attach model semantics to them. + public string Id { get; } + + /// Gets or sets what is being asked about the state. + public string Instructions { get; set; } + + /// Gets or sets any additional, provider-specific properties associated with the question. + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionRequest.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionRequest.cs new file mode 100644 index 00000000000..87b5d07d836 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionRequest.cs @@ -0,0 +1,69 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.Text.Json; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// Represents a request to an : a piece of application state and the typed questions to +/// evaluate against it. +/// +/// +/// +/// is a rather than an arbitrary object so that the serialization boundary +/// is explicit and trimming/Native AOT friendly. It may be a JSON object, an array, a string, or a scalar. Use +/// to supply strongly typed state through a +/// . +/// +/// +/// Every question is evaluated against the same state. Question identifiers must be unique within a request; they are +/// application correlation keys and carry no model semantics. +/// +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionRequest +{ + /// + /// Initializes a new instance of the class. + /// + /// The state to evaluate. + /// The questions to ask about the state. + /// is or contains a element. + /// contains two questions with the same . + public DecisionRequest(JsonElement state, IEnumerable questions) + { + _ = Throw.IfNull(questions); + + this.State = state; + + var list = new List(); + var ids = new HashSet(StringComparer.Ordinal); + foreach (DecisionQuestion question in questions) + { + _ = Throw.IfNull(question, nameof(questions)); + if (!ids.Add(question.Id)) + { + throw new ArgumentException($"Question ids must be unique within a request; '{question.Id}' appears more than once.", nameof(questions)); + } + + list.Add(question); + } + + this.Questions = list; + } + + /// Gets or sets the state that every question is evaluated against. + public JsonElement State { get; set; } + + /// Gets the questions to ask about . + /// + /// The list is mutable so callers can add questions after construction. Providers validate that identifiers are + /// unique and non-empty at call time. + /// + public IList Questions { get; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionResponse.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionResponse.cs new file mode 100644 index 00000000000..95f08c7fbea --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionResponse.cs @@ -0,0 +1,61 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Extensions.AI; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// Represents the response of an : one per question, keyed by +/// , plus what answered and what it cost. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionResponse +{ + /// + /// Initializes a new instance of the class with an empty answer set. + /// + public DecisionResponse() + { + this.Answers = new Dictionary(); + } + + /// + /// Initializes a new instance of the class. + /// + /// The answers, keyed by question identifier. + /// is . + public DecisionResponse(IDictionary answers) + { + this.Answers = Throw.IfNull(answers); + } + + /// Gets the answers, keyed by the of the question each answers. + /// + /// A provider returns an answer of the matching kind for every question in the request. A missing or mismatched + /// answer is a protocol error the provider surfaces as a rather than an absent key. + /// + public IDictionary Answers { get; } + + /// Gets or sets the identifier of the response, when the provider supplies one. + public string? ResponseId { get; set; } + + /// Gets or sets the identifier of the model that produced the answers. + /// + /// This is the resolved identifier the provider reports (for example the versioned build behind a moving alias), which + /// is what provenance should record, not the identifier that was requested. + /// + public string? ModelId { get; set; } + + /// Gets or sets usage details for the call, when the provider reports them. + public UsageDetails? Usage { get; set; } + + /// Gets or sets the raw, provider-specific representation of the response, when available. + public object? RawRepresentation { get; set; } + + /// Gets or sets any additional, provider-specific properties associated with the response. + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionScoreLevel.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionScoreLevel.cs new file mode 100644 index 00000000000..f41af6675f4 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/DecisionScoreLevel.cs @@ -0,0 +1,29 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// One level of a 's ordered scale. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionScoreLevel +{ + /// + /// Initializes a new instance of the class. + /// + /// A description of what the level means. + /// is . + /// is empty or whitespace. + public DecisionScoreLevel(string description) + { + this.Description = Throw.IfNullOrWhitespace(description); + } + + /// Gets or sets a description of what the level means. + public string Description { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/IDecisionClient.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/IDecisionClient.cs new file mode 100644 index 00000000000..420e4770d7b --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/IDecisionClient.cs @@ -0,0 +1,57 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Represents a decision-oriented inference client: a model or service that evaluates a piece of application state +/// against one or more bounded, typed questions and returns structured probabilistic answers. +/// +/// +/// +/// This is a different capability from generative inference. An +/// produces content; an produces, for each question, a probability +/// (), a selection with a probability distribution over caller-defined choices +/// (), or a position on a caller-defined ordered scale with a distribution over its +/// levels (). The output domain of every question is declared before inference, so +/// application code consumes the answers directly instead of parsing generated text. +/// +/// +/// All questions in a are evaluated against the same +/// in one call. Batching heterogeneous questions is a first-class part of the contract; how a provider executes them +/// is not prescribed. +/// +/// +/// This abstraction is a reference implementation of the shape proposed for Microsoft.Extensions.AI in +/// dotnet/extensions#7764. It is experimental and is +/// expected to be replaced by the Microsoft.Extensions.AI type when one ships; consumers should treat the names +/// here as provisional. A model-reported probability is not an empirically calibrated accuracy unless the specific +/// provider and workload establish that property, and a probability is never an authorization decision. +/// +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public interface IDecisionClient : IDisposable +{ + /// Evaluates every question in against its state. + /// The state to judge and the questions to ask about it. + /// Optional per-call options, such as a model override. + /// The to monitor for cancellation requests. + /// One answer per question, keyed by . + /// The provider refused the request or returned an unusable response. + Task GetResponseAsync(DecisionRequest request, DecisionOptions? options = null, CancellationToken cancellationToken = default); + + /// Asks the client for an object of the specified type. + /// The type of object being requested. + /// An optional key that can be used to help identify the target service. + /// The found object, otherwise . + /// + /// The purpose of this method is to allow for the retrieval of strongly typed services that may be provided by the + /// client, including itself, its , or any services it wraps. + /// + object? GetService(Type serviceType, object? serviceKey = null); +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ScoreDecisionAnswer.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ScoreDecisionAnswer.cs new file mode 100644 index 00000000000..ed6bc4ee197 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ScoreDecisionAnswer.cs @@ -0,0 +1,66 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// The answer to a : a continuous score on the ordered scale and the probability +/// distribution over every level. +/// +/// +/// For N levels at positions 0 ... N - 1, contains an entry for every level, +/// each between 0 and 1, summing to approximately 1, and the portable meaning of is +/// Σ(levelIndex × P(levelIndex)), so 0 <= Score <= N - 1. A provider adapter may compute +/// from the distribution when the provider does not return it. Two answers with the same score can +/// carry very different distributions, which is why the distribution is the portable value. The distribution supplied +/// at construction is validated (values within 0 to 1); the collection itself stays mutable, matching the proposed +/// Microsoft.Extensions.AI shape, so producers that mutate it afterwards own its invariants. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class ScoreDecisionAnswer : DecisionAnswer +{ + private double _score; + private double? _confidence; + + /// Initializes a new instance of the class. + public ScoreDecisionAnswer() + { + this.Probabilities = new Dictionary(); + } + + /// Initializes a new instance of the class. + /// The probability-weighted position on the scale. + /// The probability of each level, keyed by level index. + /// is . + /// is not finite. + public ScoreDecisionAnswer(double score, IDictionary probabilities) + { + this.Score = score; + this.Probabilities = EnsureDistribution(probabilities, nameof(probabilities)); + } + + /// Gets or sets the probability-weighted position on the scale. + /// The value is not finite. + public double Score + { + get => this._score; + set => this._score = double.IsNaN(value) || double.IsInfinity(value) + ? throw new ArgumentOutOfRangeException(nameof(value), value, "The score must be a finite number.") + : value; + } + + /// Gets the probability of each level, keyed by the level's index in . + public IDictionary Probabilities { get; } + + /// Gets or sets an optional provider-derived confidence in the score, from 0 to 1. + /// The value is not finite or is outside 0 to 1. + public double? Confidence + { + get => this._confidence; + set => this._confidence = EnsureOptionalProbability(value, nameof(value)); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ScoreDecisionQuestion.cs b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ScoreDecisionQuestion.cs new file mode 100644 index 00000000000..4333819c02e --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/Decisions/ScoreDecisionQuestion.cs @@ -0,0 +1,53 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// A question that places the state on a caller-defined ordered scale, reported as a +/// carrying a continuous score and a probability distribution over every level. +/// +/// +/// The list order of defines the ordinal position of each level: Levels[0] is position 0, +/// Levels[1] is position 1, and so on. The portable meaning of is the +/// probability-weighted position, Σ(levelIndex × P(levelIndex)). A scale needs at least two levels. Providers may +/// impose an upper limit; that limit is not part of this contract. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class ScoreDecisionQuestion : DecisionQuestion +{ + /// + /// Initializes a new instance of the class. + /// + /// The identifier of the question, unique within a request. + /// The scoring question to evaluate against the state. + /// The levels of the scale, lowest first. + /// is or contains a element. + /// has fewer than two members. + public ScoreDecisionQuestion(string id, string instructions, IEnumerable levels) + : base(id, instructions) + { + _ = Throw.IfNull(levels); + + var list = new List(); + foreach (DecisionScoreLevel level in levels) + { + list.Add(Throw.IfNull(level, nameof(levels))); + } + + if (list.Count < 2) + { + throw new ArgumentException("A score question needs at least two levels.", nameof(levels)); + } + + this.Levels = list; + } + + /// Gets the levels of the scale, lowest first. The index of a level is its ordinal position. + public IList Levels { get; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net10.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net10.0/PublicAPI.Unshipped.txt index 4f67fa96c03..d7d45053ca3 100644 --- a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net10.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net10.0/PublicAPI.Unshipped.txt @@ -14,3 +14,116 @@ [MAAI001]Microsoft.Agents.AI.AgentSessionStoreKey.WithPartition(string! name, string! value) -> Microsoft.Agents.AI.AgentSessionStoreKey! [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.Equals(object? obj) -> bool [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.GetHashCode() -> int +[MAAI001]Microsoft.Agents.AI.IDecisionClient +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetResponseAsync(Microsoft.Agents.AI.DecisionRequest! request, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetService(System.Type! serviceType, object? serviceKey = null) -> object? +[MAAI001]Microsoft.Agents.AI.DecisionRequest +[MAAI001]Microsoft.Agents.AI.DecisionRequest.DecisionRequest(System.Text.Json.JsonElement state, System.Collections.Generic.IEnumerable! questions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionRequest.Questions.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.get -> System.Text.Json.JsonElement +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions +[MAAI001]Microsoft.Agents.AI.DecisionOptions.DecisionOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.DecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Id.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.BinaryDecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.get -> Microsoft.Agents.AI.BinaryDecisionCriteria? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.BinaryDecisionCriteria() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.ChoiceDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! choices) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.Choices.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionChoice +[MAAI001]Microsoft.Agents.AI.DecisionChoice.DecisionChoice(string! name, string? description = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Name.get -> string! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.ScoreDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! levels) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.Levels.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.DecisionScoreLevel(string! description) -> void +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse() -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse(System.Collections.Generic.IDictionary! answers) -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Answers.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.get -> Microsoft.Extensions.AI.UsageDetails? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.DecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer(double trueProbability) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.get -> double +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.ChoiceDecisionAnswer(string! selectedChoice, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.get -> string! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer(double score, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.get -> double +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DecisionClientMetadata(string? providerName = null, System.Uri? providerUri = null, string? defaultModelId = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DefaultModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderName.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderUri.get -> System.Uri? +[MAAI001]Microsoft.Agents.AI.DecisionClientException +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException() -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message, System.Exception? innerException) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(Microsoft.Agents.AI.DecisionFailureKind kind, string! message, int? statusCode = null, System.Exception? innerException = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.IsTransient.get -> bool +[MAAI001]Microsoft.Agents.AI.DecisionClientException.Kind.get -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientException.StatusCode.get -> int? +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Authentication = 1 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidRequest = 2 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidResponse = 6 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Overloaded = 4 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.ProviderUnavailable = 5 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.RateLimited = 3 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Unknown = 0 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientExtensions +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetResponseAsync(this Microsoft.Agents.AI.IDecisionClient! client, TState state, System.Text.Json.Serialization.Metadata.JsonTypeInfo! stateTypeInfo, System.Collections.Generic.IEnumerable! questions, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetService(this Microsoft.Agents.AI.IDecisionClient! client, object? serviceKey = null) -> TService? +[MAAI001]virtual Microsoft.Agents.AI.DecisionOptions.Clone() -> Microsoft.Agents.AI.DecisionOptions! diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net472/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net472/PublicAPI.Unshipped.txt index 4f67fa96c03..d7d45053ca3 100644 --- a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net472/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net472/PublicAPI.Unshipped.txt @@ -14,3 +14,116 @@ [MAAI001]Microsoft.Agents.AI.AgentSessionStoreKey.WithPartition(string! name, string! value) -> Microsoft.Agents.AI.AgentSessionStoreKey! [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.Equals(object? obj) -> bool [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.GetHashCode() -> int +[MAAI001]Microsoft.Agents.AI.IDecisionClient +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetResponseAsync(Microsoft.Agents.AI.DecisionRequest! request, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetService(System.Type! serviceType, object? serviceKey = null) -> object? +[MAAI001]Microsoft.Agents.AI.DecisionRequest +[MAAI001]Microsoft.Agents.AI.DecisionRequest.DecisionRequest(System.Text.Json.JsonElement state, System.Collections.Generic.IEnumerable! questions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionRequest.Questions.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.get -> System.Text.Json.JsonElement +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions +[MAAI001]Microsoft.Agents.AI.DecisionOptions.DecisionOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.DecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Id.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.BinaryDecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.get -> Microsoft.Agents.AI.BinaryDecisionCriteria? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.BinaryDecisionCriteria() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.ChoiceDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! choices) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.Choices.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionChoice +[MAAI001]Microsoft.Agents.AI.DecisionChoice.DecisionChoice(string! name, string? description = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Name.get -> string! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.ScoreDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! levels) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.Levels.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.DecisionScoreLevel(string! description) -> void +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse() -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse(System.Collections.Generic.IDictionary! answers) -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Answers.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.get -> Microsoft.Extensions.AI.UsageDetails? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.DecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer(double trueProbability) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.get -> double +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.ChoiceDecisionAnswer(string! selectedChoice, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.get -> string! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer(double score, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.get -> double +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DecisionClientMetadata(string? providerName = null, System.Uri? providerUri = null, string? defaultModelId = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DefaultModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderName.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderUri.get -> System.Uri? +[MAAI001]Microsoft.Agents.AI.DecisionClientException +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException() -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message, System.Exception? innerException) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(Microsoft.Agents.AI.DecisionFailureKind kind, string! message, int? statusCode = null, System.Exception? innerException = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.IsTransient.get -> bool +[MAAI001]Microsoft.Agents.AI.DecisionClientException.Kind.get -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientException.StatusCode.get -> int? +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Authentication = 1 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidRequest = 2 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidResponse = 6 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Overloaded = 4 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.ProviderUnavailable = 5 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.RateLimited = 3 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Unknown = 0 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientExtensions +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetResponseAsync(this Microsoft.Agents.AI.IDecisionClient! client, TState state, System.Text.Json.Serialization.Metadata.JsonTypeInfo! stateTypeInfo, System.Collections.Generic.IEnumerable! questions, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetService(this Microsoft.Agents.AI.IDecisionClient! client, object? serviceKey = null) -> TService? +[MAAI001]virtual Microsoft.Agents.AI.DecisionOptions.Clone() -> Microsoft.Agents.AI.DecisionOptions! diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net8.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net8.0/PublicAPI.Unshipped.txt index 4f67fa96c03..d7d45053ca3 100644 --- a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net8.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net8.0/PublicAPI.Unshipped.txt @@ -14,3 +14,116 @@ [MAAI001]Microsoft.Agents.AI.AgentSessionStoreKey.WithPartition(string! name, string! value) -> Microsoft.Agents.AI.AgentSessionStoreKey! [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.Equals(object? obj) -> bool [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.GetHashCode() -> int +[MAAI001]Microsoft.Agents.AI.IDecisionClient +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetResponseAsync(Microsoft.Agents.AI.DecisionRequest! request, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetService(System.Type! serviceType, object? serviceKey = null) -> object? +[MAAI001]Microsoft.Agents.AI.DecisionRequest +[MAAI001]Microsoft.Agents.AI.DecisionRequest.DecisionRequest(System.Text.Json.JsonElement state, System.Collections.Generic.IEnumerable! questions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionRequest.Questions.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.get -> System.Text.Json.JsonElement +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions +[MAAI001]Microsoft.Agents.AI.DecisionOptions.DecisionOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.DecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Id.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.BinaryDecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.get -> Microsoft.Agents.AI.BinaryDecisionCriteria? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.BinaryDecisionCriteria() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.ChoiceDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! choices) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.Choices.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionChoice +[MAAI001]Microsoft.Agents.AI.DecisionChoice.DecisionChoice(string! name, string? description = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Name.get -> string! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.ScoreDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! levels) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.Levels.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.DecisionScoreLevel(string! description) -> void +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse() -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse(System.Collections.Generic.IDictionary! answers) -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Answers.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.get -> Microsoft.Extensions.AI.UsageDetails? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.DecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer(double trueProbability) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.get -> double +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.ChoiceDecisionAnswer(string! selectedChoice, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.get -> string! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer(double score, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.get -> double +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DecisionClientMetadata(string? providerName = null, System.Uri? providerUri = null, string? defaultModelId = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DefaultModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderName.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderUri.get -> System.Uri? +[MAAI001]Microsoft.Agents.AI.DecisionClientException +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException() -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message, System.Exception? innerException) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(Microsoft.Agents.AI.DecisionFailureKind kind, string! message, int? statusCode = null, System.Exception? innerException = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.IsTransient.get -> bool +[MAAI001]Microsoft.Agents.AI.DecisionClientException.Kind.get -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientException.StatusCode.get -> int? +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Authentication = 1 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidRequest = 2 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidResponse = 6 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Overloaded = 4 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.ProviderUnavailable = 5 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.RateLimited = 3 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Unknown = 0 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientExtensions +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetResponseAsync(this Microsoft.Agents.AI.IDecisionClient! client, TState state, System.Text.Json.Serialization.Metadata.JsonTypeInfo! stateTypeInfo, System.Collections.Generic.IEnumerable! questions, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetService(this Microsoft.Agents.AI.IDecisionClient! client, object? serviceKey = null) -> TService? +[MAAI001]virtual Microsoft.Agents.AI.DecisionOptions.Clone() -> Microsoft.Agents.AI.DecisionOptions! diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net9.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net9.0/PublicAPI.Unshipped.txt index 4f67fa96c03..d7d45053ca3 100644 --- a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net9.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/net9.0/PublicAPI.Unshipped.txt @@ -14,3 +14,116 @@ [MAAI001]Microsoft.Agents.AI.AgentSessionStoreKey.WithPartition(string! name, string! value) -> Microsoft.Agents.AI.AgentSessionStoreKey! [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.Equals(object? obj) -> bool [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.GetHashCode() -> int +[MAAI001]Microsoft.Agents.AI.IDecisionClient +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetResponseAsync(Microsoft.Agents.AI.DecisionRequest! request, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetService(System.Type! serviceType, object? serviceKey = null) -> object? +[MAAI001]Microsoft.Agents.AI.DecisionRequest +[MAAI001]Microsoft.Agents.AI.DecisionRequest.DecisionRequest(System.Text.Json.JsonElement state, System.Collections.Generic.IEnumerable! questions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionRequest.Questions.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.get -> System.Text.Json.JsonElement +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions +[MAAI001]Microsoft.Agents.AI.DecisionOptions.DecisionOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.DecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Id.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.BinaryDecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.get -> Microsoft.Agents.AI.BinaryDecisionCriteria? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.BinaryDecisionCriteria() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.ChoiceDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! choices) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.Choices.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionChoice +[MAAI001]Microsoft.Agents.AI.DecisionChoice.DecisionChoice(string! name, string? description = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Name.get -> string! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.ScoreDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! levels) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.Levels.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.DecisionScoreLevel(string! description) -> void +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse() -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse(System.Collections.Generic.IDictionary! answers) -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Answers.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.get -> Microsoft.Extensions.AI.UsageDetails? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.DecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer(double trueProbability) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.get -> double +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.ChoiceDecisionAnswer(string! selectedChoice, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.get -> string! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer(double score, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.get -> double +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DecisionClientMetadata(string? providerName = null, System.Uri? providerUri = null, string? defaultModelId = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DefaultModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderName.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderUri.get -> System.Uri? +[MAAI001]Microsoft.Agents.AI.DecisionClientException +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException() -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message, System.Exception? innerException) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(Microsoft.Agents.AI.DecisionFailureKind kind, string! message, int? statusCode = null, System.Exception? innerException = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.IsTransient.get -> bool +[MAAI001]Microsoft.Agents.AI.DecisionClientException.Kind.get -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientException.StatusCode.get -> int? +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Authentication = 1 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidRequest = 2 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidResponse = 6 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Overloaded = 4 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.ProviderUnavailable = 5 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.RateLimited = 3 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Unknown = 0 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientExtensions +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetResponseAsync(this Microsoft.Agents.AI.IDecisionClient! client, TState state, System.Text.Json.Serialization.Metadata.JsonTypeInfo! stateTypeInfo, System.Collections.Generic.IEnumerable! questions, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetService(this Microsoft.Agents.AI.IDecisionClient! client, object? serviceKey = null) -> TService? +[MAAI001]virtual Microsoft.Agents.AI.DecisionOptions.Clone() -> Microsoft.Agents.AI.DecisionOptions! diff --git a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt index 4f67fa96c03..d7d45053ca3 100644 --- a/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI.Abstractions/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt @@ -14,3 +14,116 @@ [MAAI001]Microsoft.Agents.AI.AgentSessionStoreKey.WithPartition(string! name, string! value) -> Microsoft.Agents.AI.AgentSessionStoreKey! [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.Equals(object? obj) -> bool [MAAI001]override Microsoft.Agents.AI.AgentSessionStoreKey.GetHashCode() -> int +[MAAI001]Microsoft.Agents.AI.IDecisionClient +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetResponseAsync(Microsoft.Agents.AI.DecisionRequest! request, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]Microsoft.Agents.AI.IDecisionClient.GetService(System.Type! serviceType, object? serviceKey = null) -> object? +[MAAI001]Microsoft.Agents.AI.DecisionRequest +[MAAI001]Microsoft.Agents.AI.DecisionRequest.DecisionRequest(System.Text.Json.JsonElement state, System.Collections.Generic.IEnumerable! questions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionRequest.Questions.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.get -> System.Text.Json.JsonElement +[MAAI001]Microsoft.Agents.AI.DecisionRequest.State.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions +[MAAI001]Microsoft.Agents.AI.DecisionOptions.DecisionOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionOptions.RawRepresentationFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.DecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Id.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionQuestion.Instructions.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.BinaryDecisionQuestion(string! id, string! instructions) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.get -> Microsoft.Agents.AI.BinaryDecisionCriteria? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionQuestion.Criteria.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.BinaryDecisionCriteria() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.FalseDescription.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.BinaryDecisionCriteria.TrueDescription.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.ChoiceDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! choices) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionQuestion.Choices.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionChoice +[MAAI001]Microsoft.Agents.AI.DecisionChoice.DecisionChoice(string! name, string? description = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionChoice.Name.get -> string! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.ScoreDecisionQuestion(string! id, string! instructions, System.Collections.Generic.IEnumerable! levels) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionQuestion.Levels.get -> System.Collections.Generic.IList! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.DecisionScoreLevel(string! description) -> void +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.get -> string! +[MAAI001]Microsoft.Agents.AI.DecisionScoreLevel.Description.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse() -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.DecisionResponse(System.Collections.Generic.IDictionary! answers) -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Answers.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ModelId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.ResponseId.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.get -> Microsoft.Extensions.AI.UsageDetails? +[MAAI001]Microsoft.Agents.AI.DecisionResponse.Usage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.DecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.get -> Microsoft.Extensions.AI.AdditionalPropertiesDictionary? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.AdditionalProperties.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.get -> object? +[MAAI001]Microsoft.Agents.AI.DecisionAnswer.RawRepresentation.set -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.BinaryDecisionAnswer(double trueProbability) -> void +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.get -> double +[MAAI001]Microsoft.Agents.AI.BinaryDecisionAnswer.TrueProbability.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.ChoiceDecisionAnswer(string! selectedChoice, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.get -> string! +[MAAI001]Microsoft.Agents.AI.ChoiceDecisionAnswer.SelectedChoice.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer() -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.ScoreDecisionAnswer(double score, System.Collections.Generic.IDictionary! probabilities) -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.get -> double? +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Confidence.set -> void +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Probabilities.get -> System.Collections.Generic.IDictionary! +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.get -> double +[MAAI001]Microsoft.Agents.AI.ScoreDecisionAnswer.Score.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DecisionClientMetadata(string? providerName = null, System.Uri? providerUri = null, string? defaultModelId = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.DefaultModelId.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderName.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionClientMetadata.ProviderUri.get -> System.Uri? +[MAAI001]Microsoft.Agents.AI.DecisionClientException +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException() -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(string! message, System.Exception? innerException) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.DecisionClientException(Microsoft.Agents.AI.DecisionFailureKind kind, string! message, int? statusCode = null, System.Exception? innerException = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionClientException.IsTransient.get -> bool +[MAAI001]Microsoft.Agents.AI.DecisionClientException.Kind.get -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientException.StatusCode.get -> int? +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Authentication = 1 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidRequest = 2 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.InvalidResponse = 6 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Overloaded = 4 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.ProviderUnavailable = 5 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.RateLimited = 3 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionFailureKind.Unknown = 0 -> Microsoft.Agents.AI.DecisionFailureKind +[MAAI001]Microsoft.Agents.AI.DecisionClientExtensions +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetResponseAsync(this Microsoft.Agents.AI.IDecisionClient! client, TState state, System.Text.Json.Serialization.Metadata.JsonTypeInfo! stateTypeInfo, System.Collections.Generic.IEnumerable! questions, Microsoft.Agents.AI.DecisionOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task! +[MAAI001]static Microsoft.Agents.AI.DecisionClientExtensions.GetService(this Microsoft.Agents.AI.IDecisionClient! client, object? serviceKey = null) -> TService? +[MAAI001]virtual Microsoft.Agents.AI.DecisionOptions.Clone() -> Microsoft.Agents.AI.DecisionOptions! diff --git a/dotnet/src/Microsoft.Agents.AI.TypeSafe/FeatureIndex.cs b/dotnet/src/Microsoft.Agents.AI.TypeSafe/FeatureIndex.cs new file mode 100644 index 00000000000..03bfdf147a8 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.TypeSafe/FeatureIndex.cs @@ -0,0 +1,18 @@ +// Copyright (c) Microsoft. All rights reserved. + +namespace Microsoft.Agents.AI.TypeSafe; + +internal enum FeatureIndex +{ + TypeSafe = 75, +} + +internal static class FeatureUsageMarker +{ + public static void MarkUsed() + { +#pragma warning disable MAAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed. + FeatureUsage.MarkUsed((int)FeatureIndex.TypeSafe); +#pragma warning restore MAAI001 + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.TypeSafe/Microsoft.Agents.AI.TypeSafe.csproj b/dotnet/src/Microsoft.Agents.AI.TypeSafe/Microsoft.Agents.AI.TypeSafe.csproj new file mode 100644 index 00000000000..0104c79d169 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.TypeSafe/Microsoft.Agents.AI.TypeSafe.csproj @@ -0,0 +1,30 @@ + + + + $(TargetFrameworksCore) + preview + $(NoWarn);MEAI001;MAAI001 + + + + + + true + true + + + + + Microsoft Agent Framework TypeSafe + Provides an IDecisionClient implementation for TypeSafe AI's System One decision models (Jev) for the Microsoft Agent Framework. + + + + + + + + + + + diff --git a/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeDecisionClient.cs b/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeDecisionClient.cs new file mode 100644 index 00000000000..5cfd5b780e4 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeDecisionClient.cs @@ -0,0 +1,182 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using System.Net.Http; +using System.Net.Http.Headers; +using System.Text; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI.TypeSafe; + +/// +/// An for TypeSafe AI's System One decision models (Jev), speaking the System One HTTP +/// protocol: one POST carrying { model, state, questions }, one reply carrying { model, answers, usage }. +/// +/// +/// +/// The provider-neutral question kinds map onto the protocol's primitives: to +/// noul, to choice, and to +/// score. All questions in a request are sent in one call. The protocol accepts text-only state (a string, a +/// JSON object, or an array of text) of up to roughly 32k tokens, at most 255 choices per choice question, and 2 to 10 +/// levels per score question; the client rejects requests that exceed those provider limits before sending them. +/// +/// +/// Strict parsing. Every question must come back with an answer of the matching kind, every probability +/// must be finite and within 0 to 1, and a choice answer must select one of the requested choices; anything else is a +/// with , never a fabricated value. +/// A binary answer carries no confidence because the protocol does not return one. +/// +/// +/// No hidden retries. Rate limiting (429) and overload (529) are thrown with +/// set so the caller can back off deliberately and account for it. +/// The API key is never included in an exception message; provider bodies quoted in messages are redacted and bounded. +/// +/// +/// Security considerations: the state of every request is sent to an external inference service. Send +/// only what the questions need, and only configure a provider you trust with that data. The returned probabilities are +/// model-reported values, not calibrated guarantees, and must not be used as an authorization decision. +/// +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class TypeSafeDecisionClient : IDecisionClient +{ + private readonly HttpClient _httpClient; + private readonly bool _ownsHttpClient; + private readonly string _apiKey; + private readonly Uri _endpoint; + private readonly string _modelId; + private readonly DecisionClientMetadata _metadata; + + /// + /// Initializes a new instance of the class. + /// + /// The TypeSafe API key (or the relay's key when targets a relay). + /// Optional configuration. When , defaults are used. + /// + /// An optional caller-owned , for example one with a resilience pipeline or a test handler. + /// When , the client creates and owns one with . + /// + /// is . + /// is empty or whitespace, or the configured endpoint or model identifier is invalid. + public TypeSafeDecisionClient(string apiKey, TypeSafeDecisionClientOptions? options = null, HttpClient? httpClient = null) + { + this._apiKey = Throw.IfNullOrWhitespace(apiKey); + + Uri endpoint = options?.Endpoint ?? new Uri(TypeSafeDecisionClientOptions.DefaultEndpoint); + if (!endpoint.IsAbsoluteUri) + { + throw new ArgumentException("The endpoint must be an absolute URL.", nameof(options)); + } + + if (endpoint.Scheme != Uri.UriSchemeHttps && !endpoint.IsLoopback) + { + throw new ArgumentException("The endpoint must use https; a loopback http address is allowed for local testing.", nameof(options)); + } + + string modelId = options?.ModelId ?? TypeSafeDecisionClientOptions.DefaultModelId; + if (string.IsNullOrWhiteSpace(modelId)) + { + throw new ArgumentException("The model identifier must not be empty.", nameof(options)); + } + + this._endpoint = endpoint; + this._modelId = modelId; + this._ownsHttpClient = httpClient is null; + this._httpClient = httpClient ?? new HttpClient { Timeout = options?.Timeout ?? TimeSpan.FromSeconds(60) }; + this._metadata = new DecisionClientMetadata("typesafe", endpoint, modelId); + + FeatureUsageMarker.MarkUsed(); + } + + /// Gets the request URL this client posts to. + public Uri Endpoint => this._endpoint; + + /// Gets the model identifier sent when a request does not override it. + public string ModelId => this._modelId; + + /// + public async Task GetResponseAsync(DecisionRequest request, DecisionOptions? options = null, CancellationToken cancellationToken = default) + { + _ = Throw.IfNull(request); + + string modelId = options?.ModelId ?? this._modelId; + if (string.IsNullOrWhiteSpace(modelId)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, "The model identifier override must not be empty."); + } + + string json = TypeSafeProtocol.SerializeRequest(request, modelId); + + using var message = new HttpRequestMessage(HttpMethod.Post, this._endpoint) + { + Content = new StringContent(json, Encoding.UTF8, "application/json"), + }; + message.Headers.Authorization = new AuthenticationHeaderValue("Bearer", this._apiKey); + message.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")); + + string host = this._endpoint.Host; + HttpResponseMessage response; + try + { + response = await this._httpClient.SendAsync(message, HttpCompletionOption.ResponseContentRead, cancellationToken).ConfigureAwait(false); + } + catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) + { + throw; + } + catch (OperationCanceledException ex) + { + // HttpClient reports its own timeout as a cancellation the caller did not request. + throw new DecisionClientException( + DecisionFailureKind.ProviderUnavailable, + $"The decision call to {host} timed out after {this._httpClient.Timeout.TotalSeconds:F0}s without a response.", + innerException: ex); + } + catch (HttpRequestException ex) + { + throw new DecisionClientException(DecisionFailureKind.Unknown, $"The decision call to {host} failed before a response: {ex.Message}", innerException: ex); + } + + using (response) + { +#if NET + string body = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); +#else + string body = await response.Content.ReadAsStringAsync().ConfigureAwait(false); +#endif + + if (!response.IsSuccessStatusCode) + { + // A provider that echoes the request in an error body would otherwise put the bearer token into our own + // exception message. Only error bodies are excerpted, so only they are redacted; a success body is + // parsed verbatim and never quoted. + throw TypeSafeProtocol.ClassifyFailure((int)response.StatusCode, body.Replace(this._apiKey, "[redacted]"), host); + } + + return TypeSafeProtocol.ParseResponse(body, request, host); + } + } + + /// + public object? GetService(Type serviceType, object? serviceKey = null) + { + _ = Throw.IfNull(serviceType); + + return serviceKey is null && serviceType.IsInstanceOfType(this) ? this + : serviceKey is null && serviceType.IsInstanceOfType(this._metadata) ? this._metadata + : null; + } + + /// + public void Dispose() + { + if (this._ownsHttpClient) + { + this._httpClient.Dispose(); + } + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeDecisionClientOptions.cs b/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeDecisionClientOptions.cs new file mode 100644 index 00000000000..f6492ed87dc --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeDecisionClientOptions.cs @@ -0,0 +1,42 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI.TypeSafe; + +/// +/// Provides configuration options for . +/// +/// +/// This is a class rather than a record on purpose: a record's generated ToString() would print any secret that +/// lives on it. The API key is passed to the client constructor and is never stored on the options. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class TypeSafeDecisionClientOptions +{ + /// TypeSafe's System One endpoint (POST), per the TypeSafe API reference. + public const string DefaultEndpoint = "https://api.typesafe.ai/v1/systemone"; + + /// + /// OpenRouter's relay of the same protocol. When targeting it, use OpenRouter's model identifiers (for example + /// typesafe/jev-1.13) and an OpenRouter API key. + /// + public const string OpenRouterEndpoint = "https://openrouter.ai/api/v1/systemone"; + + /// + /// TypeSafe's flagship alias. It moves between releases; the a response carries + /// is the resolved build. Pin a versioned identifier such as jev-1.13.0 for reproducible runs. + /// + public const string DefaultModelId = "jev-latest"; + + /// Gets or sets the full request URL. Defaults to . + public Uri Endpoint { get; set; } = new(DefaultEndpoint); + + /// Gets or sets the model identifier sent when a request does not override it. Defaults to . + public string ModelId { get; set; } = DefaultModelId; + + /// Gets or sets the request timeout applied when the client creates its own . Defaults to 60 seconds. + public TimeSpan Timeout { get; set; } = TimeSpan.FromSeconds(60); +} diff --git a/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeProtocol.cs b/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeProtocol.cs new file mode 100644 index 00000000000..aaf34e73b2f --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.TypeSafe/TypeSafeProtocol.cs @@ -0,0 +1,497 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Globalization; +using System.IO; +using System.Text; +using System.Text.Json; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.TypeSafe; + +/// +/// The System One wire shape in one place: request serialization, provider-limit validation, response parsing, and +/// failure classification. Internal so the tests can pin the protocol without a network. +/// +internal static class TypeSafeProtocol +{ + /// The most choices a System One choice question accepts. A provider limit, not a contract one. + internal const int MaxChoices = 255; + + /// The most levels a System One score question accepts. A provider limit, not a contract one. + internal const int MaxScoreLevels = 10; + + private const int BodyExcerptLength = 500; + + internal static string SerializeRequest(DecisionRequest request, string modelId) + { + ValidateQuestions(request); + + using var stream = new MemoryStream(); + using (var writer = new Utf8JsonWriter(stream)) + { + writer.WriteStartObject(); + writer.WriteString("model", modelId); + + writer.WritePropertyName("state"); + if (request.State.ValueKind == JsonValueKind.Undefined) + { + writer.WriteNullValue(); + } + else + { + request.State.WriteTo(writer); + } + + writer.WritePropertyName("questions"); + writer.WriteStartObject(); + foreach (DecisionQuestion question in request.Questions) + { + writer.WritePropertyName(question.Id); + WriteQuestion(writer, question); + } + + writer.WriteEndObject(); + writer.WriteEndObject(); + } + + return Encoding.UTF8.GetString(stream.ToArray()); + } + + private static void ValidateQuestions(DecisionRequest request) + { + if (request.Questions.Count == 0) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, "A decision request needs at least one question."); + } + + var ids = new HashSet(StringComparer.Ordinal); + foreach (DecisionQuestion question in request.Questions) + { + if (question is null) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, "A decision request contains a null question."); + } + + if (string.IsNullOrWhiteSpace(question.Id)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, "Question identifiers must not be empty."); + } + + if (!ids.Add(question.Id)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Question identifiers must be unique within a request; '{question.Id}' appears more than once."); + } + + if (string.IsNullOrWhiteSpace(question.Instructions)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Question '{question.Id}' has no instructions."); + } + } + } + + private static void WriteQuestion(Utf8JsonWriter writer, DecisionQuestion question) + { + writer.WriteStartObject(); + switch (question) + { + case BinaryDecisionQuestion binary: + writer.WriteString("type", "noul"); + writer.WriteString("instructions", binary.Instructions); + if (binary.Criteria is { } criteria && (criteria.TrueDescription is not null || criteria.FalseDescription is not null)) + { + writer.WritePropertyName("criteria"); + writer.WriteStartObject(); + if (criteria.TrueDescription is not null) + { + writer.WriteString("true", criteria.TrueDescription); + } + + if (criteria.FalseDescription is not null) + { + writer.WriteString("false", criteria.FalseDescription); + } + + writer.WriteEndObject(); + } + + break; + + case ChoiceDecisionQuestion choice: + ValidateChoices(choice); + if (choice.Choices.Count > MaxChoices) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"A System One choice question accepts at most {MaxChoices} choices; question '{choice.Id}' has {choice.Choices.Count}."); + } + + writer.WriteString("type", "choice"); + writer.WriteString("instructions", choice.Instructions); + writer.WritePropertyName("criteria"); + writer.WriteStartObject(); + foreach (DecisionChoice option in choice.Choices) + { + writer.WriteString(option.Name, option.Description ?? option.Name); + } + + writer.WriteEndObject(); + break; + + case ScoreDecisionQuestion score: + ValidateLevels(score); + if (score.Levels.Count > MaxScoreLevels) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"A System One score question accepts at most {MaxScoreLevels} levels; question '{score.Id}' has {score.Levels.Count}."); + } + + writer.WriteString("type", "score"); + writer.WriteString("instructions", score.Instructions); + writer.WritePropertyName("criteria"); + writer.WriteStartArray(); + foreach (DecisionScoreLevel level in score.Levels) + { + writer.WriteStringValue(level.Description); + } + + writer.WriteEndArray(); + break; + + default: + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Question '{question.Id}' is of an unsupported kind ({question.GetType().Name})."); + } + + writer.WriteEndObject(); + } + + /// Re-checks a choice question's invariants at call time, since its collection is mutable after construction. + private static void ValidateChoices(ChoiceDecisionQuestion question) + { + if (question.Choices.Count < 2) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Choice question '{question.Id}' needs at least two choices."); + } + + var names = new HashSet(StringComparer.Ordinal); + foreach (DecisionChoice option in question.Choices) + { + if (option is null || string.IsNullOrWhiteSpace(option.Name)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Choice question '{question.Id}' contains a null or unnamed choice."); + } + + if (!names.Add(option.Name)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Choice question '{question.Id}' lists choice '{option.Name}' more than once."); + } + } + } + + /// Re-checks a score question's invariants at call time, since its collection is mutable after construction. + private static void ValidateLevels(ScoreDecisionQuestion question) + { + if (question.Levels.Count < 2) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Score question '{question.Id}' needs at least two levels."); + } + + foreach (DecisionScoreLevel level in question.Levels) + { + if (level is null || string.IsNullOrWhiteSpace(level.Description)) + { + throw new DecisionClientException(DecisionFailureKind.InvalidRequest, $"Score question '{question.Id}' contains a null or undescribed level."); + } + } + } + + internal static DecisionClientException ClassifyFailure(int statusCode, string body, string host) + { + DecisionFailureKind kind = statusCode switch + { + 401 or 403 => DecisionFailureKind.Authentication, + 400 or 422 => DecisionFailureKind.InvalidRequest, + 429 => DecisionFailureKind.RateLimited, + 529 => DecisionFailureKind.Overloaded, + >= 500 and <= 599 => DecisionFailureKind.ProviderUnavailable, + _ => DecisionFailureKind.Unknown, + }; + + return new DecisionClientException(kind, $"{host} answered HTTP {statusCode} ({kind}): {Excerpt(body)}", statusCode); + } + + internal static DecisionResponse ParseResponse(string body, DecisionRequest request, string host) + { + JsonDocument document; + try + { + document = JsonDocument.Parse(body); + } + catch (JsonException ex) + { + throw Invalid($"{host} answered with a body that is not JSON: {Excerpt(body)}", ex); + } + + using (document) + { + JsonElement root = document.RootElement; + if (root.ValueKind != JsonValueKind.Object) + { + throw Invalid($"{host} answered with a JSON {root.ValueKind}, not an object."); + } + + if (!root.TryGetProperty("model", out JsonElement modelElement) || modelElement.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(modelElement.GetString())) + { + throw Invalid($"{host} answered without a 'model' field naming what answered."); + } + + if (!root.TryGetProperty("answers", out JsonElement answersElement) || answersElement.ValueKind != JsonValueKind.Object) + { + throw Invalid($"{host} answered without an 'answers' object."); + } + + var answers = new Dictionary(StringComparer.Ordinal); + foreach (DecisionQuestion question in request.Questions) + { + if (!answersElement.TryGetProperty(question.Id, out JsonElement answerElement) || answerElement.ValueKind != JsonValueKind.Object) + { + throw Invalid($"{host} returned no answer for question '{question.Id}'."); + } + + answers[question.Id] = ParseAnswer(question, answerElement, host); + } + + var response = new DecisionResponse(answers) + { + ModelId = modelElement.GetString(), + RawRepresentation = root.Clone(), + }; + + if (root.TryGetProperty("id", out JsonElement idElement) && idElement.ValueKind == JsonValueKind.String) + { + response.ResponseId = idElement.GetString(); + } + + if (root.TryGetProperty("usage", out JsonElement usageElement) && usageElement.ValueKind == JsonValueKind.Object) + { + long input = ReadLong(usageElement, "input_tokens", host); + long output = ReadLong(usageElement, "output_tokens", host); + response.Usage = new UsageDetails + { + InputTokenCount = input, + OutputTokenCount = output, + TotalTokenCount = input + output, + }; + + // OpenRouter's relay reports the billed cost; TypeSafe does not. Surface it without pretending it is a token count. + if (usageElement.TryGetProperty("cost", out JsonElement costElement) && costElement.ValueKind == JsonValueKind.Number) + { + response.AdditionalProperties ??= []; + response.AdditionalProperties["cost"] = costElement.GetDouble(); + } + } + + return response; + } + } + + private static DecisionAnswer ParseAnswer(DecisionQuestion question, JsonElement element, string host) + { + string expected = question switch + { + BinaryDecisionQuestion => "noul", + ChoiceDecisionQuestion => "choice", + ScoreDecisionQuestion => "score", + _ => throw Invalid($"Question '{question.Id}' is of an unsupported kind ({question.GetType().Name})."), + }; + + if (!element.TryGetProperty("type", out JsonElement typeElement) || typeElement.ValueKind != JsonValueKind.String) + { + throw Invalid($"{host}: answer '{question.Id}' has no 'type'."); + } + + string? type = typeElement.GetString(); + if (!string.Equals(type, expected, StringComparison.Ordinal)) + { + throw Invalid($"{host}: answer '{question.Id}' is a '{type}' answer to a '{expected}' question."); + } + + try + { + switch (question) + { + case BinaryDecisionQuestion: + return new BinaryDecisionAnswer(ReadDouble(element, "noul", question.Id, host)) { RawRepresentation = element.Clone() }; + + case ChoiceDecisionQuestion choiceQuestion: + { + string selected = ReadString(element, "choice", question.Id, host); + Dictionary probabilities = ReadDistribution(element, question.Id, host); + + foreach (DecisionChoice option in choiceQuestion.Choices) + { + if (!probabilities.ContainsKey(option.Name)) + { + throw Invalid($"{host}: answer '{question.Id}' has no probability for choice '{option.Name}'."); + } + } + + bool selectedIsRequested = false; + foreach (DecisionChoice option in choiceQuestion.Choices) + { + if (string.Equals(option.Name, selected, StringComparison.Ordinal)) + { + selectedIsRequested = true; + break; + } + } + + if (!selectedIsRequested) + { + throw Invalid($"{host}: answer '{question.Id}' selected '{selected}', which is not one of the requested choices."); + } + + return new ChoiceDecisionAnswer(selected, probabilities) + { + Confidence = ReadOptionalDouble(element, "confidence", question.Id, host), + RawRepresentation = element.Clone(), + }; + } + + case ScoreDecisionQuestion scoreQuestion: + { + var probabilities = new Dictionary(); + foreach (KeyValuePair pair in ReadDistribution(element, question.Id, host)) + { + // Levels are 0-indexed on the wire: "0" is the first level supplied. + if (!int.TryParse(pair.Key, NumberStyles.Integer, CultureInfo.InvariantCulture, out int index) || index < 0 || index >= scoreQuestion.Levels.Count) + { + throw Invalid($"{host}: answer '{question.Id}' has a probability for unknown level '{pair.Key}'."); + } + + probabilities[index] = pair.Value; + } + + for (int i = 0; i < scoreQuestion.Levels.Count; i++) + { + if (!probabilities.ContainsKey(i)) + { + throw Invalid($"{host}: answer '{question.Id}' has no probability for level {i}."); + } + } + + double score = ReadDouble(element, "score", question.Id, host); + if (double.IsNaN(score) || score < 0 || score > scoreQuestion.Levels.Count - 1) + { + throw Invalid($"{host}: answer '{question.Id}' has score {score}, outside the requested scale 0..{scoreQuestion.Levels.Count - 1}."); + } + + var answer = new ScoreDecisionAnswer(score, probabilities) + { + Confidence = ReadOptionalDouble(element, "confidence", question.Id, host), + RawRepresentation = element.Clone(), + }; + + if (element.TryGetProperty("legend", out JsonElement legend) && legend.ValueKind == JsonValueKind.Object) + { + answer.AdditionalProperties = new AdditionalPropertiesDictionary { ["legend"] = legend.Clone() }; + } + + return answer; + } + + default: + throw Invalid($"Question '{question.Id}' is of an unsupported kind ({question.GetType().Name})."); + } + } + catch (ArgumentException ex) + { + // The answer types validate their own ranges (probability outside 0..1, blank choice). + throw Invalid($"{host}: answer '{question.Id}' is out of range: {ex.Message}", ex); + } + } + + private static double ReadDouble(JsonElement element, string name, string id, string host) + { + if (!element.TryGetProperty(name, out JsonElement value) || value.ValueKind != JsonValueKind.Number) + { + throw Invalid($"{host}: answer '{id}' has no numeric '{name}'."); + } + + return value.GetDouble(); + } + + private static double? ReadOptionalDouble(JsonElement element, string name, string id, string host) + { + if (!element.TryGetProperty(name, out JsonElement value) || value.ValueKind == JsonValueKind.Null) + { + return null; + } + + if (value.ValueKind != JsonValueKind.Number) + { + throw Invalid($"{host}: answer '{id}' has a non-numeric '{name}'."); + } + + return value.GetDouble(); + } + + private static long ReadLong(JsonElement element, string name, string host) + { + if (!element.TryGetProperty(name, out JsonElement value) || value.ValueKind != JsonValueKind.Number) + { + throw Invalid($"{host}: usage has no numeric '{name}'."); + } + + return value.TryGetInt64(out long result) ? result : (long)value.GetDouble(); + } + + private static string ReadString(JsonElement element, string name, string id, string host) + { + if (!element.TryGetProperty(name, out JsonElement value) || value.ValueKind != JsonValueKind.String) + { + throw Invalid($"{host}: answer '{id}' has no string '{name}'."); + } + + return value.GetString()!; + } + + private static Dictionary ReadDistribution(JsonElement element, string id, string host) + { + if (!element.TryGetProperty("probabilities", out JsonElement value) || value.ValueKind != JsonValueKind.Object) + { + throw Invalid($"{host}: answer '{id}' has no 'probabilities' object."); + } + + var map = new Dictionary(StringComparer.Ordinal); + foreach (JsonProperty property in value.EnumerateObject()) + { + if (property.Value.ValueKind != JsonValueKind.Number) + { + throw Invalid($"{host}: answer '{id}' has a non-numeric probability for '{property.Name}'."); + } + + double probability = property.Value.GetDouble(); + if (double.IsNaN(probability) || probability < 0 || probability > 1) + { + throw Invalid($"{host}: answer '{id}' has a probability outside 0..1 for '{property.Name}'."); + } + + map[property.Name] = probability; + } + + return map; + } + + private static DecisionClientException Invalid(string message, Exception? inner = null) => + new(DecisionFailureKind.InvalidResponse, message, statusCode: null, innerException: inner); + + private static string Excerpt(string body) + { + if (string.IsNullOrWhiteSpace(body)) + { + return ""; + } + + string flat = body.Replace('\r', ' ').Replace('\n', ' ').Trim(); + return flat.Length <= BodyExcerptLength ? flat : string.Concat(flat.AsSpan(0, BodyExcerptLength), "…"); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluator.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluator.cs new file mode 100644 index 00000000000..25aa361696b --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluator.cs @@ -0,0 +1,267 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.Text.Json; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using Microsoft.Shared.DiagnosticIds; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI; + +/// +/// A that asks an for the model-reported probability that the +/// user's original request has been fully addressed, and stops requesting iterations once that probability reaches a +/// configured completion threshold. +/// +/// +/// +/// This evaluator targets decision-oriented inference: models that evaluate application state against a bounded yes/no +/// question and return a probability directly, rather than generating text. It complements +/// , which uses a generative judge and can therefore also +/// produce a free-form gap analysis. A decision model reports only a probability, so when this evaluator continues the +/// loop it carries the deterministic as feedback +/// instead of model-generated text. +/// +/// +/// After each iteration the evaluator sends the client one (identifier +/// , text ) against the +/// projected loop state and reads the . The evaluator, not the client, +/// applies : a probability at or above the threshold +/// yields , anything below yields . +/// +/// +/// Because evaluates its evaluators in order and stops only when all of them return +/// , placing this evaluator before an yields a +/// cheap-then-strong cascade: when the decision model judges the work incomplete the expensive judge is skipped and the +/// agent runs again, and when it judges the work complete the strong judge verifies before the loop ends. +/// +/// +/// By default the state sent to the client is a minimal text projection: the role and text of each original request +/// message, the latest response text, and the iteration number. Like , it judges the +/// latest response; agents that build an answer incrementally across turns need a +/// that accumulates. Because the projection cannot faithfully +/// represent non-text content, the evaluator throws rather than silently judging an incomplete representation when the +/// original request contains content other than ; supply a state factory to judge such requests. +/// +/// +/// Operational failures of the client (a or any other exception, or a missing or +/// mismatched answer) are not treated as "incomplete". They are handled according to +/// , which defaults to propagating the exception; a failure the +/// client reports as transient () follows +/// when one is configured. Cancellation requested +/// through the supplied token always propagates. +/// +/// +/// When a logger factory is supplied, the evaluator logs one line per evaluation (iteration, model-reported probability, +/// threshold, and outcome) at debug level, and a warning when a failure policy other than throwing is applied. The +/// warning carries only the exception type, failure kind, status code, and transience, never the exception message or +/// the projected state. +/// +/// +/// Security considerations: Using this evaluator is an explicit opt-in. The decision client is an +/// additional external inference boundary: on every iteration it receives the projected original request and the +/// agent's latest response, both of which may contain sensitive or untrusted content. Only configure a client you trust +/// as much as the primary model, and keep the state projection minimal. The probability is a model-reported value, not +/// a calibrated guarantee; it must not be used as an authorization or approval signal. +/// +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionLoopEvaluator : LoopEvaluator +{ + /// The default value of . + public const double DefaultCompletionThreshold = 0.90; + + /// The default value of . + public const string DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?"; + + /// The default value of . + public const string DefaultCompletedDescription = "The original request has been fully addressed and no material requested work remains."; + + /// The default value of . + public const string DefaultIncompleteDescription = "Material requested work remains incomplete or missing."; + + /// The default value of . + public const string DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it."; + + /// The of the completion question sent to the client. + public const string CompletionQuestionId = "task_completed"; + + private readonly IDecisionClient _decisionClient; + private readonly double _completionThreshold; + private readonly string _completionQuestion; + private readonly BinaryDecisionCriteria? _criteria; + private readonly string? _continueFeedbackMessage; + private readonly Func? _stateFactory; + private readonly DecisionLoopFailureBehavior _failureBehavior; + private readonly DecisionLoopFailureBehavior? _transientFailureBehavior; + private readonly ILogger _logger; + + /// + /// Initializes a new instance of the class. + /// + /// + /// The decision client that answers the completion question. Security: the client is sent the + /// projected original request and the agent's latest response on every iteration, so only configure a client that + /// points at a service you trust as much as the primary model — see the type-level security considerations. + /// + /// Optional configuration. When , defaults are used. + /// Optional factory used to create the evaluator's logger. + /// is . + /// is not between 0 and 1 inclusive. + /// is empty or whitespace. + public DecisionLoopEvaluator(IDecisionClient decisionClient, DecisionLoopEvaluatorOptions? options = null, ILoggerFactory? loggerFactory = null) + { + this._decisionClient = Throw.IfNull(decisionClient); + + double threshold = options?.CompletionThreshold ?? DefaultCompletionThreshold; + if (double.IsNaN(threshold) || threshold < 0 || threshold > 1) + { + throw new ArgumentOutOfRangeException(nameof(options), threshold, $"{nameof(DecisionLoopEvaluatorOptions.CompletionThreshold)} must be between 0 and 1 inclusive."); + } + + string question = options?.CompletionQuestion ?? DefaultCompletionQuestion; + if (string.IsNullOrWhiteSpace(question)) + { + throw new ArgumentException($"{nameof(DecisionLoopEvaluatorOptions.CompletionQuestion)} must not be empty.", nameof(options)); + } + + string? completed = options is null ? DefaultCompletedDescription : options.CompletedDescription; + string? incomplete = options is null ? DefaultIncompleteDescription : options.IncompleteDescription; + + this._completionThreshold = threshold; + this._completionQuestion = question; + this._criteria = completed is null && incomplete is null + ? null + : new BinaryDecisionCriteria { TrueDescription = completed, FalseDescription = incomplete }; + this._continueFeedbackMessage = options is null ? DefaultContinueFeedbackMessage : options.ContinueFeedbackMessage; + this._stateFactory = options?.StateFactory; + this._failureBehavior = options?.FailureBehavior ?? DecisionLoopFailureBehavior.Throw; + this._transientFailureBehavior = options?.TransientFailureBehavior; + this._logger = (loggerFactory ?? NullLoggerFactory.Instance).CreateLogger(); + } + + /// + public override async ValueTask EvaluateAsync(LoopContext context, CancellationToken cancellationToken = default) + { + _ = Throw.IfNull(context); + + // State projection errors (including rejected non-text content) are configuration problems and always propagate; + // the failure policy applies only to the decision client itself. + JsonElement state = this._stateFactory is not null + ? this._stateFactory(context) + : CreateDefaultState(context); + + var question = new BinaryDecisionQuestion(CompletionQuestionId, this._completionQuestion) { Criteria = this._criteria }; + var request = new DecisionRequest(state, [question]); + + double probability; + try + { + DecisionResponse response = await this._decisionClient.GetResponseAsync(request, options: null, cancellationToken).ConfigureAwait(false); + + if (!response.Answers.TryGetValue(CompletionQuestionId, out DecisionAnswer? rawAnswer) || rawAnswer is not BinaryDecisionAnswer answer) + { + throw new InvalidOperationException($"The decision client did not return a binary answer for question '{CompletionQuestionId}'."); + } + + probability = answer.TrueProbability; + } + catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) + { + throw; + } + catch (Exception ex) when (this.ResolveFailureBehavior(ex) == DecisionLoopFailureBehavior.Continue) + { + this.LogFailurePolicyApplied(ex, context.Iteration, DecisionLoopFailureBehavior.Continue); + return LoopEvaluation.Continue(this._continueFeedbackMessage); + } + catch (Exception ex) when (this.ResolveFailureBehavior(ex) == DecisionLoopFailureBehavior.DeferToNextEvaluator) + { + this.LogFailurePolicyApplied(ex, context.Iteration, DecisionLoopFailureBehavior.DeferToNextEvaluator); + return LoopEvaluation.Stop(); + } + + // At or above the threshold this evaluator does not request another iteration. Later evaluators (if any) are + // still consulted by the LoopAgent, which is what enables the cheap-then-strong cascade. + bool complete = probability >= this._completionThreshold; + + if (this._logger.IsEnabled(LogLevel.Debug)) + { + this._logger.LogDebug( + "DecisionLoopEvaluator iteration {Iteration}: model-reported P(complete) = {Probability:F3}, threshold {Threshold:F2}, outcome {Outcome}.", + context.Iteration, + probability, + this._completionThreshold, + complete ? "no-continuation" : "continue"); + } + + return complete + ? LoopEvaluation.Stop() + : LoopEvaluation.Continue(this._continueFeedbackMessage); + } + + /// + /// Selects the failure behavior for : the transient behavior, when one is configured and + /// the client reports the failure as transient; otherwise the general failure behavior. + /// + private DecisionLoopFailureBehavior ResolveFailureBehavior(Exception exception) => + this._transientFailureBehavior is { } transient && exception is DecisionClientException { IsTransient: true } + ? transient + : this._failureBehavior; + + private void LogFailurePolicyApplied(Exception exception, int iteration, DecisionLoopFailureBehavior behavior) + { + if (this._logger.IsEnabled(LogLevel.Warning)) + { + // Only classification fields are logged. The exception object is deliberately not attached: a provider + // error body excerpted into its message could echo request data, and this evaluator never logs the state. + this._logger.LogWarning( + "DecisionLoopEvaluator iteration {Iteration}: the decision client failed ({ExceptionType}, kind {FailureKind}, status {StatusCode}, transient: {IsTransient}); applying {FailureBehavior}.", + iteration, + exception.GetType().Name, + exception is DecisionClientException dce ? dce.Kind.ToString() : "n/a", + exception is DecisionClientException { StatusCode: { } status } ? status.ToString(System.Globalization.CultureInfo.InvariantCulture) : "n/a", + exception is DecisionClientException { IsTransient: true }, + behavior); + } + } + + /// + /// Builds the default, minimal text-only state: the role and text of each original request message, the latest + /// response text, and the iteration number. Throws when the original request carries non-text content, because a + /// text projection would silently misrepresent it. + /// + private static JsonElement CreateDefaultState(LoopContext context) + { + var originalRequest = new List(context.InitialMessages.Count); + foreach (ChatMessage message in context.InitialMessages) + { + foreach (AIContent content in message.Contents) + { + if (content is not TextContent) + { + throw new InvalidOperationException( + $"The original request contains non-text content ({content.GetType().Name}) that the default state projection cannot represent. " + + $"Supply {nameof(DecisionLoopEvaluatorOptions)}.{nameof(DecisionLoopEvaluatorOptions.StateFactory)} to judge this request."); + } + } + + originalRequest.Add(new DecisionLoopMessage { Role = message.Role.Value, Text = message.Text }); + } + + var state = new DecisionLoopState + { + OriginalRequest = originalRequest, + LatestResponse = context.LastResponse.Text, + Iteration = context.Iteration, + }; + + return JsonSerializer.SerializeToElement(state, LoopJsonContext.Default.DecisionLoopState); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluatorOptions.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluatorOptions.cs new file mode 100644 index 00000000000..e5d62f14085 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopEvaluatorOptions.cs @@ -0,0 +1,91 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Diagnostics.CodeAnalysis; +using System.Text.Json; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Provides configuration options for . +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public sealed class DecisionLoopEvaluatorOptions +{ + /// + /// Gets or sets the model-reported probability of completion at or above which the evaluator stops requesting + /// iterations. Defaults to . + /// + /// + /// The threshold is application policy, not a property of the decision source. A false stop returns an incomplete + /// result while a false continue only costs another iteration, so a deliberately high threshold is usually the + /// safer choice. Must be between 0 and 1 inclusive. + /// + public double CompletionThreshold { get; set; } = DecisionLoopEvaluator.DefaultCompletionThreshold; + + /// + /// Gets or sets the yes/no proposition sent to the decision client. Defaults to + /// . + /// + /// + /// Phrase it so that a high probability means the loop should stop. A value falls back to the + /// default; an empty or whitespace value is rejected by the evaluator constructor. + /// + public string? CompletionQuestion { get; set; } = DecisionLoopEvaluator.DefaultCompletionQuestion; + + /// + /// Gets or sets an optional description of what a "yes" (completed) answer means, sent to the decision client as + /// . Defaults to + /// ; set to to omit it. + /// + public string? CompletedDescription { get; set; } = DecisionLoopEvaluator.DefaultCompletedDescription; + + /// + /// Gets or sets an optional description of what a "no" (incomplete) answer means, sent to the decision client as + /// . Defaults to + /// ; set to to omit it. + /// + public string? IncompleteDescription { get; set; } = DecisionLoopEvaluator.DefaultIncompleteDescription; + + /// + /// Gets or sets the feedback carried into the next iteration when the evaluator decides the request is not yet + /// complete. Defaults to . + /// + /// + /// A decision model reports a probability, not an explanation, so this text is deterministic application text rather + /// than model output. Set it to or whitespace to continue without feedback; note that in the + /// default reused-session mode the then re-invokes the agent with no new input messages. + /// + public string? ContinueFeedbackMessage { get; set; } = DecisionLoopEvaluator.DefaultContinueFeedbackMessage; + + /// + /// Gets or sets a factory that builds the serialized state sent to the decision client from the current + /// , replacing the evaluator's default projection. + /// + /// + /// The default projection contains only the text of the original request messages, the latest response text, and + /// the iteration number, and it rejects requests containing non-text content. Supply a factory to judge different or + /// richer state (for example a textual rendering of images, or additional session facts). The application owns + /// whatever additional data the factory chooses to disclose to the decision source. + /// + public Func? StateFactory { get; set; } + + /// + /// Gets or sets how the evaluator reacts when the decision client fails operationally. Defaults to + /// . + /// + public DecisionLoopFailureBehavior FailureBehavior { get; set; } = DecisionLoopFailureBehavior.Throw; + + /// + /// Gets or sets how the evaluator reacts when the decision client reports a failure it classifies as transient + /// (: rate limiting, overload, or a provider-side outage), or + /// to apply to those failures as well. Defaults to . + /// + /// + /// This lets a loop survive a momentary provider hiccup (for example continue, or defer to a later evaluator) while + /// still surfacing permanent problems such as a rejected API key or an invalid request through . + /// Any retry-like effect stays visible: the applied policy is logged at warning level when a logger is configured. + /// + public DecisionLoopFailureBehavior? TransientFailureBehavior { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopFailureBehavior.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopFailureBehavior.cs new file mode 100644 index 00000000000..6f26b036ee4 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopFailureBehavior.cs @@ -0,0 +1,37 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics.CodeAnalysis; +using Microsoft.Shared.DiagnosticIds; + +namespace Microsoft.Agents.AI; + +/// +/// Specifies how a reacts when the decision client fails operationally (for +/// example a provider, network, or protocol error, or an invalid probability) rather than returning a probability. +/// +/// +/// An operational failure is not a semantic answer: it does not mean the task is incomplete. The default, +/// , surfaces the failure to the caller. The other behaviors let an application choose an explicit +/// policy instead. No behavior is ever selected implicitly based on the provider. +/// +[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] +public enum DecisionLoopFailureBehavior +{ + /// Propagate the failure to the caller of the loop. This is the default. + Throw = 0, + + /// + /// Request another iteration, carrying the configured + /// as feedback. The loop remains bounded by + /// . + /// + Continue = 1, + + /// + /// Return so that later evaluators configured on the decide + /// instead. Because only means "this evaluator does not request another iteration", + /// the loop ends when the decision evaluator is the last (or only) evaluator and no other evaluator requests + /// continuation. + /// + DeferToNextEvaluator = 2, +} diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopState.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopState.cs new file mode 100644 index 00000000000..7ec4b8378d5 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/DecisionLoopState.cs @@ -0,0 +1,32 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; + +namespace Microsoft.Agents.AI; + +/// +/// The default, minimal state projection that sends to its decision source. +/// +internal sealed class DecisionLoopState +{ + /// Gets or sets the text projection of the original request messages. + public IList OriginalRequest { get; set; } = []; + + /// Gets or sets the text of the agent's latest response. + public string LatestResponse { get; set; } = string.Empty; + + /// Gets or sets the number of completed iterations. + public int Iteration { get; set; } +} + +/// +/// The text projection of a single original request message in . +/// +internal sealed class DecisionLoopMessage +{ + /// Gets or sets the message role. + public string Role { get; set; } = string.Empty; + + /// Gets or sets the message text. + public string Text { get; set; } = string.Empty; +} diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopAgent.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopAgent.cs index 398dc061565..0abad1152ad 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopAgent.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopAgent.cs @@ -25,7 +25,7 @@ namespace Microsoft.Agents.AI; /// After each run of the wrapped agent, the configured evaluators are asked whether to re-invoke the agent and what /// feedback to carry forward. This enables patterns such as iterative refinement, working through a task list, or /// judging whether the original request was answered. Out-of-the-box evaluators include -/// , , and +/// , , , and /// . /// /// @@ -33,7 +33,9 @@ namespace Microsoft.Agents.AI; /// asks to re-invoke wins: its feedback drives the next iteration and the remaining evaluators are not evaluated. The /// loop stops only when every evaluator asks to stop. Consequently, evaluator order is priority order and /// means "this evaluator does not request continuation" rather than a veto that -/// terminates the loop; place stop-only guards accordingly. +/// terminates the loop; place stop-only guards accordingly. This ordering also enables a cheap-then-strong cascade: a +/// placed before an skips the expensive judge +/// whenever the cheap decision requests continuation, and lets the judge verify whenever it does not. /// /// /// The caller's initial messages are sent to the wrapped agent exactly once. By default (when diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopEvaluator.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopEvaluator.cs index 9e1872ce8f1..41a80ed7841 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopEvaluator.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopEvaluator.cs @@ -19,8 +19,8 @@ namespace Microsoft.Agents.AI; /// that consumes it. /// /// -/// Out-of-the-box implementations include , , -/// , and . Implementations should be stateless and safe to share across +/// Out-of-the-box implementations include , , +/// , , and . Implementations should be stateless and safe to share across /// concurrent loop runs; any per-run state must be stored on the supplied . /// /// diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopJsonContext.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopJsonContext.cs index 8d69383e3f4..97223b4a79b 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopJsonContext.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Loop/LoopJsonContext.cs @@ -8,9 +8,11 @@ namespace Microsoft.Agents.AI; /// /// Source-generated for loop types that require JSON serialization, such as the -/// structured used by . +/// structured used by and the +/// projection used by . /// [JsonSourceGenerationOptions(JsonSerializerDefaults.Web)] [JsonSerializable(typeof(JudgeVerdict))] +[JsonSerializable(typeof(DecisionLoopState))] [ExcludeFromCodeCoverage] internal sealed partial class LoopJsonContext : JsonSerializerContext; diff --git a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net10.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net10.0/PublicAPI.Unshipped.txt index 1ec4e243d09..796dd651ad4 100644 --- a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net10.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net10.0/PublicAPI.Unshipped.txt @@ -22,3 +22,34 @@ Microsoft.Agents.AI.AgentModeProviderOptions.DisableModeSetTool.set -> void [MAAI001]static Microsoft.Agents.AI.OpenTelemetryAgent.DefaultSourceName.get -> string! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.FindMatchingFilesAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.SearchAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator.DecisionLoopEvaluator(Microsoft.Agents.AI.IDecisionClient! decisionClient, Microsoft.Agents.AI.DecisionLoopEvaluatorOptions? options = null, Microsoft.Extensions.Logging.ILoggerFactory? loggerFactory = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.get -> double +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.DecisionLoopEvaluatorOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Continue = 1 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.DeferToNextEvaluator = 2 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Throw = 0 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.CompletionQuestionId = "task_completed" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletedDescription = "The original request has been fully addressed and no material requested work remains." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionThreshold = 0.9 -> double +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultIncompleteDescription = "Material requested work remains incomplete or missing." -> string! +[MAAI001]override Microsoft.Agents.AI.DecisionLoopEvaluator.EvaluateAsync(Microsoft.Agents.AI.LoopContext! context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask diff --git a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net472/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net472/PublicAPI.Unshipped.txt index 1ec4e243d09..796dd651ad4 100644 --- a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net472/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net472/PublicAPI.Unshipped.txt @@ -22,3 +22,34 @@ Microsoft.Agents.AI.AgentModeProviderOptions.DisableModeSetTool.set -> void [MAAI001]static Microsoft.Agents.AI.OpenTelemetryAgent.DefaultSourceName.get -> string! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.FindMatchingFilesAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.SearchAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator.DecisionLoopEvaluator(Microsoft.Agents.AI.IDecisionClient! decisionClient, Microsoft.Agents.AI.DecisionLoopEvaluatorOptions? options = null, Microsoft.Extensions.Logging.ILoggerFactory? loggerFactory = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.get -> double +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.DecisionLoopEvaluatorOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Continue = 1 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.DeferToNextEvaluator = 2 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Throw = 0 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.CompletionQuestionId = "task_completed" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletedDescription = "The original request has been fully addressed and no material requested work remains." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionThreshold = 0.9 -> double +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultIncompleteDescription = "Material requested work remains incomplete or missing." -> string! +[MAAI001]override Microsoft.Agents.AI.DecisionLoopEvaluator.EvaluateAsync(Microsoft.Agents.AI.LoopContext! context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask diff --git a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net8.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net8.0/PublicAPI.Unshipped.txt index 1ec4e243d09..796dd651ad4 100644 --- a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net8.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net8.0/PublicAPI.Unshipped.txt @@ -22,3 +22,34 @@ Microsoft.Agents.AI.AgentModeProviderOptions.DisableModeSetTool.set -> void [MAAI001]static Microsoft.Agents.AI.OpenTelemetryAgent.DefaultSourceName.get -> string! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.FindMatchingFilesAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.SearchAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator.DecisionLoopEvaluator(Microsoft.Agents.AI.IDecisionClient! decisionClient, Microsoft.Agents.AI.DecisionLoopEvaluatorOptions? options = null, Microsoft.Extensions.Logging.ILoggerFactory? loggerFactory = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.get -> double +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.DecisionLoopEvaluatorOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Continue = 1 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.DeferToNextEvaluator = 2 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Throw = 0 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.CompletionQuestionId = "task_completed" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletedDescription = "The original request has been fully addressed and no material requested work remains." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionThreshold = 0.9 -> double +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultIncompleteDescription = "Material requested work remains incomplete or missing." -> string! +[MAAI001]override Microsoft.Agents.AI.DecisionLoopEvaluator.EvaluateAsync(Microsoft.Agents.AI.LoopContext! context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask diff --git a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net9.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net9.0/PublicAPI.Unshipped.txt index 1ec4e243d09..796dd651ad4 100644 --- a/dotnet/src/Microsoft.Agents.AI/PublicAPI/net9.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI/PublicAPI/net9.0/PublicAPI.Unshipped.txt @@ -22,3 +22,34 @@ Microsoft.Agents.AI.AgentModeProviderOptions.DisableModeSetTool.set -> void [MAAI001]static Microsoft.Agents.AI.OpenTelemetryAgent.DefaultSourceName.get -> string! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.FindMatchingFilesAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.SearchAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator.DecisionLoopEvaluator(Microsoft.Agents.AI.IDecisionClient! decisionClient, Microsoft.Agents.AI.DecisionLoopEvaluatorOptions? options = null, Microsoft.Extensions.Logging.ILoggerFactory? loggerFactory = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.get -> double +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.DecisionLoopEvaluatorOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Continue = 1 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.DeferToNextEvaluator = 2 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Throw = 0 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.CompletionQuestionId = "task_completed" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletedDescription = "The original request has been fully addressed and no material requested work remains." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionThreshold = 0.9 -> double +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultIncompleteDescription = "Material requested work remains incomplete or missing." -> string! +[MAAI001]override Microsoft.Agents.AI.DecisionLoopEvaluator.EvaluateAsync(Microsoft.Agents.AI.LoopContext! context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask diff --git a/dotnet/src/Microsoft.Agents.AI/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt b/dotnet/src/Microsoft.Agents.AI/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt index 1ec4e243d09..796dd651ad4 100644 --- a/dotnet/src/Microsoft.Agents.AI/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt +++ b/dotnet/src/Microsoft.Agents.AI/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt @@ -22,3 +22,34 @@ Microsoft.Agents.AI.AgentModeProviderOptions.DisableModeSetTool.set -> void [MAAI001]static Microsoft.Agents.AI.OpenTelemetryAgent.DefaultSourceName.get -> string! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.FindMatchingFilesAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! [MAAI001]virtual Microsoft.Agents.AI.AgentFileStore.SearchAsync(string! directory, string! regexPattern, string? globPattern = null, bool recursive = false, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!>! +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluator.DecisionLoopEvaluator(Microsoft.Agents.AI.IDecisionClient! decisionClient, Microsoft.Agents.AI.DecisionLoopEvaluatorOptions? options = null, Microsoft.Extensions.Logging.ILoggerFactory? loggerFactory = null) -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletedDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionQuestion.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.get -> double +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.CompletionThreshold.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.ContinueFeedbackMessage.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.DecisionLoopEvaluatorOptions() -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.FailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.get -> string? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.IncompleteDescription.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.get -> System.Func? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.StateFactory.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.get -> Microsoft.Agents.AI.DecisionLoopFailureBehavior? +[MAAI001]Microsoft.Agents.AI.DecisionLoopEvaluatorOptions.TransientFailureBehavior.set -> void +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Continue = 1 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.DeferToNextEvaluator = 2 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]Microsoft.Agents.AI.DecisionLoopFailureBehavior.Throw = 0 -> Microsoft.Agents.AI.DecisionLoopFailureBehavior +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.CompletionQuestionId = "task_completed" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletedDescription = "The original request has been fully addressed and no material requested work remains." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionQuestion = "Has the agent fully addressed the user's original request?" -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultCompletionThreshold = 0.9 -> double +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultContinueFeedbackMessage = "The original request is not yet fully addressed. Continue working on it." -> string! +[MAAI001]const Microsoft.Agents.AI.DecisionLoopEvaluator.DefaultIncompleteDescription = "Material requested work remains incomplete or missing." -> string! +[MAAI001]override Microsoft.Agents.AI.DecisionLoopEvaluator.EvaluateAsync(Microsoft.Agents.AI.LoopContext! context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask diff --git a/dotnet/tests/Microsoft.Agents.AI.FeatureRegistry.UnitTests/FeatureRegistryTests.cs b/dotnet/tests/Microsoft.Agents.AI.FeatureRegistry.UnitTests/FeatureRegistryTests.cs index 386b6b833f6..87d79bc48f2 100644 --- a/dotnet/tests/Microsoft.Agents.AI.FeatureRegistry.UnitTests/FeatureRegistryTests.cs +++ b/dotnet/tests/Microsoft.Agents.AI.FeatureRegistry.UnitTests/FeatureRegistryTests.cs @@ -58,6 +58,7 @@ public sealed class FeatureRegistryTests ["foundry.agent"] = "Microsoft.Agents.AI.Foundry", ["foundry.memory"] = "Microsoft.Agents.AI.Foundry", ["foundry.evals"] = "Microsoft.Agents.AI.Foundry", + ["typesafe"] = "Microsoft.Agents.AI.TypeSafe", ["foundry.toolbox"] = "Microsoft.Agents.AI.Foundry", ["foundry_hosting"] = "Microsoft.Agents.AI.Foundry.Hosting", ["openai"] = "Microsoft.Agents.AI.OpenAI", diff --git a/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/Microsoft.Agents.AI.TypeSafe.UnitTests.csproj b/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/Microsoft.Agents.AI.TypeSafe.UnitTests.csproj new file mode 100644 index 00000000000..f5b187ff4d5 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/Microsoft.Agents.AI.TypeSafe.UnitTests.csproj @@ -0,0 +1,18 @@ + + + + $(TargetFrameworksCore) + $(NoWarn);MAAI001 + + + + + + + + + + + + + diff --git a/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/TypeSafeDecisionClientTests.cs b/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/TypeSafeDecisionClientTests.cs new file mode 100644 index 00000000000..20df2f3c6e6 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/TypeSafeDecisionClientTests.cs @@ -0,0 +1,236 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Net; +using System.Net.Http; +using System.Text.Json; +using System.Threading; +using System.Threading.Tasks; + +namespace Microsoft.Agents.AI.TypeSafe.UnitTests; + +/// +/// Unit tests for the class, using a stub handler in place of the network. +/// +public class TypeSafeDecisionClientTests +{ + private const string OkBody = """{ "model": "jev-1.13.0", "answers": { "q": { "type": "noul", "noul": 0.75 } }, "usage": { "input_tokens": 12, "output_tokens": 0 } }"""; + + private static DecisionRequest Request() + { + using JsonDocument state = JsonDocument.Parse("\"some state\""); + return new DecisionRequest(state.RootElement.Clone(), [new BinaryDecisionQuestion("q", "Is it?")]); + } + + #region Constructor + + [Theory] + [InlineData(null)] + [InlineData("")] + [InlineData(" ")] + public void Constructor_BlankApiKey_Throws(string? apiKey) + { + Assert.ThrowsAny(() => new TypeSafeDecisionClient(apiKey!)); + } + + [Fact] + public void Constructor_HttpNonLoopbackEndpoint_Throws() + { + Assert.Throws("options", () => new TypeSafeDecisionClient("k", new() { Endpoint = new Uri("http://example.com/v1/systemone") })); + } + + [Fact] + public void Constructor_Defaults_UseTypeSafeEndpointAndAlias() + { + using var client = new TypeSafeDecisionClient("k"); + + Assert.Equal(new Uri(TypeSafeDecisionClientOptions.DefaultEndpoint), client.Endpoint); + Assert.Equal(TypeSafeDecisionClientOptions.DefaultModelId, client.ModelId); + } + + [Fact] + public void Constructor_MarksFeatureUsage() + { + FeatureUsageAssert.Reset(); + + using var client = new TypeSafeDecisionClient("k"); + + FeatureUsageAssert.Marked((int)FeatureIndex.TypeSafe); + } + + #endregion + + #region GetService + + [Fact] + public void GetService_ReturnsSelfAndMetadata() + { + using var client = new TypeSafeDecisionClient("k", new() { ModelId = "jev-1.13.0" }); + + Assert.Same(client, client.GetService(typeof(IDecisionClient))); + Assert.Same(client, client.GetService(typeof(TypeSafeDecisionClient))); + var metadata = Assert.IsType(client.GetService(typeof(DecisionClientMetadata))); + Assert.Equal("typesafe", metadata.ProviderName); + Assert.Equal("jev-1.13.0", metadata.DefaultModelId); + Assert.Equal(client.Endpoint, metadata.ProviderUri); + Assert.Null(client.GetService(typeof(string))); + Assert.Null(client.GetService(typeof(IDecisionClient), "key")); + } + + #endregion + + #region GetResponseAsync + + [Fact] + public async Task GetResponseAsync_SendsBearerJsonPostToEndpointAsync() + { + HttpRequestMessage? sent = null; + string? sentBody = null; + using var handler = new StubHandler(async (request, ct) => + { + sent = request; + sentBody = await request.Content!.ReadAsStringAsync(ct); + return new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OkBody) }; + }); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("secret-key", new() { ModelId = "jev-1.13.0" }, http); + + DecisionResponse response = await client.GetResponseAsync(Request()); + + Assert.NotNull(sent); + Assert.Equal(HttpMethod.Post, sent!.Method); + Assert.Equal(new Uri(TypeSafeDecisionClientOptions.DefaultEndpoint), sent.RequestUri); + Assert.Equal("Bearer", sent.Headers.Authorization!.Scheme); + Assert.Equal("secret-key", sent.Headers.Authorization.Parameter); + Assert.Equal("application/json", sent.Content!.Headers.ContentType!.MediaType); + using JsonDocument body = JsonDocument.Parse(sentBody!); + Assert.Equal("jev-1.13.0", body.RootElement.GetProperty("model").GetString()); + Assert.Equal(0.75, Assert.IsType(response.Answers["q"]).TrueProbability, precision: 6); + Assert.Equal("jev-1.13.0", response.ModelId); + } + + [Fact] + public async Task GetResponseAsync_OptionsModelId_OverridesDefaultAsync() + { + string? sentBody = null; + using var handler = new StubHandler(async (request, ct) => + { + sentBody = await request.Content!.ReadAsStringAsync(ct); + return new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OkBody) }; + }); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("k", httpClient: http); + + await client.GetResponseAsync(Request(), new DecisionOptions { ModelId = "jev-preview" }); + + using JsonDocument body = JsonDocument.Parse(sentBody!); + Assert.Equal("jev-preview", body.RootElement.GetProperty("model").GetString()); + } + + [Theory] + [InlineData("")] + [InlineData(" ")] + public async Task GetResponseAsync_BlankModelOverride_ThrowsInvalidRequestWithoutSendingAsync(string modelId) + { + bool sent = false; + using var handler = new StubHandler((_, _) => + { + sent = true; + return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OkBody) }); + }); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("k", httpClient: http); + + var ex = await Assert.ThrowsAsync(() => client.GetResponseAsync(Request(), new DecisionOptions { ModelId = modelId })); + + Assert.Equal(DecisionFailureKind.InvalidRequest, ex.Kind); + Assert.False(sent); + } + + [Fact] + public async Task GetResponseAsync_ProviderError_ThrowsClassifiedAndRedactedAsync() + { + using var handler = new StubHandler((_, _) => Task.FromResult(new HttpResponseMessage(HttpStatusCode.Unauthorized) + { + Content = new StringContent("{\"error\":\"bad key secret-key\"}"), + })); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("secret-key", httpClient: http); + + var ex = await Assert.ThrowsAsync(() => client.GetResponseAsync(Request())); + + Assert.Equal(DecisionFailureKind.Authentication, ex.Kind); + Assert.Equal(401, ex.StatusCode); + Assert.DoesNotContain("secret-key", ex.Message); + Assert.Contains("[redacted]", ex.Message); + } + + [Fact] + public async Task GetResponseAsync_RateLimited_IsTransientAsync() + { + using var handler = new StubHandler((_, _) => Task.FromResult(new HttpResponseMessage((HttpStatusCode)429) { Content = new StringContent("slow down") })); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("k", httpClient: http); + + var ex = await Assert.ThrowsAsync(() => client.GetResponseAsync(Request())); + + Assert.True(ex.IsTransient); + Assert.Equal(DecisionFailureKind.RateLimited, ex.Kind); + } + + [Fact] + public async Task GetResponseAsync_NetworkFailure_ThrowsUnknownWithInnerAsync() + { + using var handler = new StubHandler((_, _) => throw new HttpRequestException("connection refused")); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("k", httpClient: http); + + var ex = await Assert.ThrowsAsync(() => client.GetResponseAsync(Request())); + + Assert.Equal(DecisionFailureKind.Unknown, ex.Kind); + Assert.IsType(ex.InnerException); + } + + [Fact] + public async Task GetResponseAsync_CallerCancellation_PropagatesAsync() + { + using var cts = new CancellationTokenSource(); + using var handler = new StubHandler((_, ct) => + { + cts.Cancel(); + ct.ThrowIfCancellationRequested(); + return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)); + }); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("k", httpClient: http); + + await Assert.ThrowsAnyAsync(() => client.GetResponseAsync(Request(), cancellationToken: cts.Token)); + } + + [Fact] + public async Task GetResponseAsync_UnusableSuccessBody_ThrowsInvalidResponseAsync() + { + using var handler = new StubHandler((_, _) => Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent("{\"model\":\"m\",\"answers\":{}}") })); + using var http = new HttpClient(handler); + using var client = new TypeSafeDecisionClient("k", httpClient: http); + + var ex = await Assert.ThrowsAsync(() => client.GetResponseAsync(Request())); + + Assert.Equal(DecisionFailureKind.InvalidResponse, ex.Kind); + } + + [Fact] + public async Task GetResponseAsync_NullRequest_ThrowsAsync() + { + using var client = new TypeSafeDecisionClient("k"); + + await Assert.ThrowsAsync("request", () => client.GetResponseAsync(null!)); + } + + #endregion + + private sealed class StubHandler(Func> send) : HttpMessageHandler + { + protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) => send(request, cancellationToken); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/TypeSafeProtocolTests.cs b/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/TypeSafeProtocolTests.cs new file mode 100644 index 00000000000..cbff272b9be --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.TypeSafe.UnitTests/TypeSafeProtocolTests.cs @@ -0,0 +1,362 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.TypeSafe.UnitTests; + +/// +/// Pins the System One wire shape without a network: what a request serializes to, and what the parser accepts and +/// refuses. The JSON fixtures follow the shapes documented in TypeSafe's API reference. +/// +public class TypeSafeProtocolTests +{ + private static JsonElement State(string text) + { + using JsonDocument document = JsonDocument.Parse(JsonSerializer.Serialize(text)); + return document.RootElement.Clone(); + } + + private static DecisionRequest Request(params DecisionQuestion[] questions) => new(State("I was charged twice."), questions); + + #region Request serialization + + [Fact] + public void SerializeRequest_BinaryWithoutCriteria_WritesNoulAndOmitsCriteria() + { + var request = Request(new BinaryDecisionQuestion("refund", "Is the customer asking for money back?")); + + using JsonDocument doc = JsonDocument.Parse(TypeSafeProtocol.SerializeRequest(request, "jev-1.13.0")); + JsonElement root = doc.RootElement; + + Assert.Equal("jev-1.13.0", root.GetProperty("model").GetString()); + Assert.Equal("I was charged twice.", root.GetProperty("state").GetString()); + JsonElement q = root.GetProperty("questions").GetProperty("refund"); + Assert.Equal("noul", q.GetProperty("type").GetString()); + Assert.Equal("Is the customer asking for money back?", q.GetProperty("instructions").GetString()); + Assert.False(q.TryGetProperty("criteria", out _)); + } + + [Fact] + public void SerializeRequest_BinaryWithCriteria_WritesTrueAndFalse() + { + var request = Request(new BinaryDecisionQuestion("grounded", "Is it grounded?") + { + Criteria = new BinaryDecisionCriteria { TrueDescription = "Every claim is supported.", FalseDescription = "A claim is unsupported." }, + }); + + using JsonDocument doc = JsonDocument.Parse(TypeSafeProtocol.SerializeRequest(request, "m")); + JsonElement criteria = doc.RootElement.GetProperty("questions").GetProperty("grounded").GetProperty("criteria"); + + Assert.Equal("Every claim is supported.", criteria.GetProperty("true").GetString()); + Assert.Equal("A claim is unsupported.", criteria.GetProperty("false").GetString()); + } + + [Fact] + public void SerializeRequest_ChoiceAndScore_WriteMapAndArray() + { + var request = Request( + new ChoiceDecisionQuestion("dept", "Which team?", [new("billing", "Payments"), new("tech", "Bugs"), new("other")]), + new ScoreDecisionQuestion("mood", "How upset?", [new("calm"), new("concerned"), new("angry")])); + + using JsonDocument doc = JsonDocument.Parse(TypeSafeProtocol.SerializeRequest(request, "m")); + JsonElement questions = doc.RootElement.GetProperty("questions"); + + JsonElement dept = questions.GetProperty("dept"); + Assert.Equal("choice", dept.GetProperty("type").GetString()); + Assert.Equal("Payments", dept.GetProperty("criteria").GetProperty("billing").GetString()); + Assert.Equal("other", dept.GetProperty("criteria").GetProperty("other").GetString()); + + JsonElement mood = questions.GetProperty("mood"); + Assert.Equal("score", mood.GetProperty("type").GetString()); + Assert.Equal(["calm", "concerned", "angry"], mood.GetProperty("criteria").EnumerateArray().Select(e => e.GetString()).ToArray()); + } + + [Fact] + public void SerializeRequest_ObjectState_IsWrittenVerbatim() + { + using JsonDocument state = JsonDocument.Parse("{\"message\":\"hi\",\"order\":\"A-1\"}"); + var request = new DecisionRequest(state.RootElement.Clone(), [new BinaryDecisionQuestion("q", "?")]); + + using JsonDocument doc = JsonDocument.Parse(TypeSafeProtocol.SerializeRequest(request, "m")); + + Assert.Equal("A-1", doc.RootElement.GetProperty("state").GetProperty("order").GetString()); + } + + [Fact] + public void SerializeRequest_NoQuestions_ThrowsInvalidRequest() + { + var request = new DecisionRequest(State("x"), []); + + var ex = Assert.Throws(() => TypeSafeProtocol.SerializeRequest(request, "m")); + + Assert.Equal(DecisionFailureKind.InvalidRequest, ex.Kind); + } + + [Fact] + public void SerializeRequest_DuplicateIdsAddedAfterConstruction_ThrowsInvalidRequest() + { + var request = Request(new BinaryDecisionQuestion("a", "?")); + request.Questions.Add(new BinaryDecisionQuestion("a", "again?")); + + var ex = Assert.Throws(() => TypeSafeProtocol.SerializeRequest(request, "m")); + + Assert.Equal(DecisionFailureKind.InvalidRequest, ex.Kind); + Assert.Contains("'a'", ex.Message); + } + + [Fact] + public void SerializeRequest_ChoiceAboveProviderMaximum_ThrowsInvalidRequestBeforeSending() + { + var choices = Enumerable.Range(0, TypeSafeProtocol.MaxChoices + 1).Select(i => new DecisionChoice($"c{i}")); + var request = Request(new ChoiceDecisionQuestion("c", "Which?", choices)); + + var ex = Assert.Throws(() => TypeSafeProtocol.SerializeRequest(request, "m")); + + Assert.Equal(DecisionFailureKind.InvalidRequest, ex.Kind); + Assert.False(ex.IsTransient); + } + + [Fact] + public void SerializeRequest_ScoreAboveProviderMaximum_ThrowsInvalidRequestBeforeSending() + { + var levels = Enumerable.Range(0, TypeSafeProtocol.MaxScoreLevels + 1).Select(i => new DecisionScoreLevel($"level {i}")); + var request = Request(new ScoreDecisionQuestion("s", "How much?", levels)); + + var ex = Assert.Throws(() => TypeSafeProtocol.SerializeRequest(request, "m")); + + Assert.Equal(DecisionFailureKind.InvalidRequest, ex.Kind); + } + + #endregion + + [Fact] + public void SerializeRequest_ChoicesMutatedAfterConstruction_ThrowsInvalidRequest() + { + var question = new ChoiceDecisionQuestion("c", "Which?", [new("a"), new("b")]); + question.Choices.Add(new DecisionChoice("a")); + var duplicate = Request(question); + + var emptied = new ChoiceDecisionQuestion("c", "Which?", [new("a"), new("b")]); + emptied.Choices.Clear(); + var empty = Request(emptied); + + Assert.Equal(DecisionFailureKind.InvalidRequest, Assert.Throws(() => TypeSafeProtocol.SerializeRequest(duplicate, "m")).Kind); + Assert.Equal(DecisionFailureKind.InvalidRequest, Assert.Throws(() => TypeSafeProtocol.SerializeRequest(empty, "m")).Kind); + } + + [Fact] + public void SerializeRequest_LevelsMutatedAfterConstruction_ThrowsInvalidRequest() + { + var question = new ScoreDecisionQuestion("s", "How much?", [new("lo"), new("hi")]); + question.Levels.Clear(); + question.Levels.Add(new DecisionScoreLevel("only")); + + var ex = Assert.Throws(() => TypeSafeProtocol.SerializeRequest(Request(question), "m")); + + Assert.Equal(DecisionFailureKind.InvalidRequest, ex.Kind); + } + + #region Response parsing + + private const string MixedResponse = """ + { + "model": "jev-1.13.0", + "answers": { + "refund": { "type": "noul", "noul": 0.98 }, + "dept": { "type": "choice", "choice": "billing", "probabilities": { "billing": 0.82, "tech": 0.12, "other": 0.06 }, "confidence": 0.7 }, + "mood": { "type": "score", "score": 1.3, "legend": { "0": "calm", "1": "concerned", "2": "angry" }, "probabilities": { "0": 0.0, "1": 0.7, "2": 0.3 }, "confidence": 0.6 } + }, + "usage": { "input_tokens": 120, "output_tokens": 0 } + } + """; + + private static DecisionRequest MixedRequest() => Request( + new BinaryDecisionQuestion("refund", "?"), + new ChoiceDecisionQuestion("dept", "?", [new("billing"), new("tech"), new("other")]), + new ScoreDecisionQuestion("mood", "?", [new("calm"), new("concerned"), new("angry")])); + + [Fact] + public void ParseResponse_MixedBatch_MapsEveryAnswerKind() + { + DecisionResponse response = TypeSafeProtocol.ParseResponse(MixedResponse, MixedRequest(), "h"); + + Assert.Equal("jev-1.13.0", response.ModelId); + Assert.Equal(3, response.Answers.Count); + + var refund = Assert.IsType(response.Answers["refund"]); + Assert.Equal(0.98, refund.TrueProbability, precision: 6); + + var dept = Assert.IsType(response.Answers["dept"]); + Assert.Equal("billing", dept.SelectedChoice); + Assert.Equal(0.12, dept.Probabilities["tech"], precision: 6); + Assert.Equal(0.7, dept.Confidence!.Value, precision: 6); + + var mood = Assert.IsType(response.Answers["mood"]); + Assert.Equal(1.3, mood.Score, precision: 6); + Assert.Equal(0.7, mood.Probabilities[1], precision: 6); + Assert.Equal(0.6, mood.Confidence!.Value, precision: 6); + Assert.NotNull(mood.AdditionalProperties); + Assert.True(mood.AdditionalProperties!.ContainsKey("legend")); + + Assert.NotNull(response.Usage); + Assert.Equal(120, response.Usage!.InputTokenCount); + Assert.Equal(0, response.Usage.OutputTokenCount); + Assert.Equal(120, response.Usage.TotalTokenCount); + Assert.IsType(response.RawRepresentation); + Assert.IsType(refund.RawRepresentation); + } + + [Fact] + public void ParseResponse_OpenRouterExtras_SurfaceIdAndCost() + { + const string body = """ + { "id": "gen-1", "model": "typesafe/jev-1.13", "answers": { "q": { "type": "noul", "noul": 0.5 } }, "usage": { "input_tokens": 10, "output_tokens": 0, "cost": 0.00042 } } + """; + + DecisionResponse response = TypeSafeProtocol.ParseResponse(body, Request(new BinaryDecisionQuestion("q", "?")), "h"); + + Assert.Equal("gen-1", response.ResponseId); + Assert.Equal(0.00042, Assert.IsType(response.AdditionalProperties!["cost"]), precision: 8); + } + + [Fact] + public void ParseResponse_BinaryAnswer_HasNoConfidence() + { + const string body = """{ "model": "m", "answers": { "q": { "type": "noul", "noul": 0.5 } } }"""; + + DecisionResponse response = TypeSafeProtocol.ParseResponse(body, Request(new BinaryDecisionQuestion("q", "?")), "h"); + + var answer = Assert.IsType(response.Answers["q"]); + Assert.Equal(0.5, answer.TrueProbability); + Assert.Null(response.Usage); + } + + [Theory] + [InlineData("""{ "model": "m", "answers": { } }""", "no answer")] + [InlineData("""{ "model": "m", "answers": { "q": { "type": "choice", "choice": "x", "probabilities": { "x": 1 } } } }""", "'choice' answer to a 'noul'")] + [InlineData("""{ "model": "m", "answers": { "q": { "type": "noul", "noul": 1.5 } } }""", "out of range")] + [InlineData("""{ "model": "m", "answers": { "q": { "type": "noul", "noul": "high" } } }""", "no numeric 'noul'")] + [InlineData("""{ "answers": { "q": { "type": "noul", "noul": 0.5 } } }""", "'model'")] + [InlineData("""{ "model": "m" }""", "'answers'")] + [InlineData("""not json""", "not JSON")] + [InlineData("""[1,2]""", "not an object")] + public void ParseResponse_UnusableBody_ThrowsInvalidResponse(string body, string expectedFragment) + { + var ex = Assert.Throws(() => TypeSafeProtocol.ParseResponse(body, Request(new BinaryDecisionQuestion("q", "?")), "h")); + + Assert.Equal(DecisionFailureKind.InvalidResponse, ex.Kind); + Assert.False(ex.IsTransient); + Assert.Contains(expectedFragment, ex.Message); + } + + [Fact] + public void ParseResponse_ChoiceSelectingUnknownChoice_ThrowsInvalidResponse() + { + const string body = """{ "model": "m", "answers": { "c": { "type": "choice", "choice": "nope", "probabilities": { "a": 0.5, "b": 0.5 } } } }"""; + var request = Request(new ChoiceDecisionQuestion("c", "?", [new("a"), new("b")])); + + var ex = Assert.Throws(() => TypeSafeProtocol.ParseResponse(body, request, "h")); + + Assert.Contains("not one of the requested choices", ex.Message); + } + + [Fact] + public void ParseResponse_ChoiceSelectingExtraReturnedOption_ThrowsInvalidResponse() + { + // The provider returns a probability for an option that was never asked and selects it. + const string body = """{ "model": "m", "answers": { "c": { "type": "choice", "choice": "nope", "probabilities": { "a": 0.3, "b": 0.3, "nope": 0.4 } } } }"""; + var request = Request(new ChoiceDecisionQuestion("c", "?", [new("a"), new("b")])); + + var ex = Assert.Throws(() => TypeSafeProtocol.ParseResponse(body, request, "h")); + + Assert.Contains("not one of the requested choices", ex.Message); + } + + [Theory] + [InlineData(-0.1)] + [InlineData(1.5)] + public void ParseResponse_ScoreOutsideRequestedScale_ThrowsInvalidResponse(double score) + { + string body = $$"""{ "model": "m", "answers": { "s": { "type": "score", "score": {{score.ToString(System.Globalization.CultureInfo.InvariantCulture)}}, "probabilities": { "0": 0.5, "1": 0.5 }, "confidence": 0.5 } } }"""; + var request = Request(new ScoreDecisionQuestion("s", "?", [new("lo"), new("hi")])); + + var ex = Assert.Throws(() => TypeSafeProtocol.ParseResponse(body, request, "h")); + + Assert.Contains("outside the requested scale", ex.Message); + } + + [Fact] + public void ParseResponse_ChoiceMissingAProbability_ThrowsInvalidResponse() + { + const string body = """{ "model": "m", "answers": { "c": { "type": "choice", "choice": "a", "probabilities": { "a": 1.0 } } } }"""; + var request = Request(new ChoiceDecisionQuestion("c", "?", [new("a"), new("b")])); + + var ex = Assert.Throws(() => TypeSafeProtocol.ParseResponse(body, request, "h")); + + Assert.Contains("no probability for choice 'b'", ex.Message); + } + + [Fact] + public void ParseResponse_ScoreWithUnknownLevel_ThrowsInvalidResponse() + { + const string body = """{ "model": "m", "answers": { "s": { "type": "score", "score": 0.5, "probabilities": { "0": 0.5, "1": 0.5, "5": 0.0 }, "confidence": 0.5 } } }"""; + var request = Request(new ScoreDecisionQuestion("s", "?", [new("lo"), new("hi")])); + + var ex = Assert.Throws(() => TypeSafeProtocol.ParseResponse(body, request, "h")); + + Assert.Contains("unknown level '5'", ex.Message); + } + + #endregion + + #region Answer construction invariants + + [Fact] + public void ChoiceDecisionAnswer_InvalidDistributionAtConstruction_Throws() + { + Assert.Throws(() => new ChoiceDecisionAnswer("a", new Dictionary { ["a"] = 1.5 })); + Assert.Throws(() => new ChoiceDecisionAnswer("a", new Dictionary { [" "] = 0.5 })); + Assert.Throws(() => new ScoreDecisionAnswer(0.5, new Dictionary { [0] = double.NaN })); + } + + #endregion + + #region Failure classification + + [Theory] + [InlineData(401, DecisionFailureKind.Authentication, false)] + [InlineData(403, DecisionFailureKind.Authentication, false)] + [InlineData(400, DecisionFailureKind.InvalidRequest, false)] + [InlineData(422, DecisionFailureKind.InvalidRequest, false)] + [InlineData(429, DecisionFailureKind.RateLimited, true)] + [InlineData(529, DecisionFailureKind.Overloaded, true)] + [InlineData(500, DecisionFailureKind.ProviderUnavailable, true)] + [InlineData(503, DecisionFailureKind.ProviderUnavailable, true)] + [InlineData(418, DecisionFailureKind.Unknown, false)] + public void ClassifyFailure_MapsStatusToKindAndTransience(int status, DecisionFailureKind kind, bool transient) + { + DecisionClientException ex = TypeSafeProtocol.ClassifyFailure(status, "{\"error\":\"x\"}", "h"); + + Assert.Equal(kind, ex.Kind); + Assert.Equal(transient, ex.IsTransient); + Assert.Equal(status, ex.StatusCode); + Assert.Contains($"HTTP {status}", ex.Message); + } + + [Fact] + public void ClassifyFailure_LongBody_IsExcerptedAndFlattened() + { + string body = "line1\nline2 " + new string('x', 2000); + + DecisionClientException ex = TypeSafeProtocol.ClassifyFailure(500, body, "h"); + + Assert.DoesNotContain("\n", ex.Message); + Assert.True(ex.Message.Length < 700); + } + + #endregion +} diff --git a/dotnet/tests/Microsoft.Agents.AI.UnitTests/Harness/Loop/DecisionLoopEvaluatorTests.cs b/dotnet/tests/Microsoft.Agents.AI.UnitTests/Harness/Loop/DecisionLoopEvaluatorTests.cs new file mode 100644 index 00000000000..ed4f6af7d96 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.UnitTests/Harness/Loop/DecisionLoopEvaluatorTests.cs @@ -0,0 +1,889 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; +using Microsoft.Extensions.Logging; +using Moq; + +using static Microsoft.Agents.AI.UnitTests.LoopTestHelpers; + +namespace Microsoft.Agents.AI.UnitTests; + +/// +/// Unit tests for the class. +/// +public class DecisionLoopEvaluatorTests +{ + #region Constructor + + /// + /// Verify that the constructor throws when the decision client is null. + /// + [Fact] + public void Constructor_NullClient_Throws() + { + // Act & Assert + Assert.Throws("decisionClient", () => new DecisionLoopEvaluator(null!)); + } + + /// + /// Verify that the constructor rejects a completion threshold outside the 0..1 range. + /// + [Theory] + [InlineData(-0.01)] + [InlineData(1.01)] + [InlineData(double.NaN)] + public void Constructor_InvalidThreshold_Throws(double threshold) + { + // Act & Assert + Assert.Throws("options", () => new DecisionLoopEvaluator(Always(0.5), new() { CompletionThreshold = threshold })); + } + + /// + /// Verify that the constructor rejects a blank completion question. + /// + [Theory] + [InlineData("")] + [InlineData(" ")] + public void Constructor_BlankQuestion_Throws(string question) + { + // Act & Assert + Assert.Throws("options", () => new DecisionLoopEvaluator(Always(0.5), new() { CompletionQuestion = question })); + } + + /// + /// Verify that a null completion question falls back to the default question. + /// + [Fact] + public async Task Constructor_NullQuestion_UsesDefaultAsync() + { + // Arrange + var client = new FakeDecisionClient(1.0); + var evaluator = new DecisionLoopEvaluator(client, new() { CompletionQuestion = null }); + + // Act + await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.Equal(DecisionLoopEvaluator.DefaultCompletionQuestion, client.LastQuestion!.Instructions); + } + + /// + /// Verify that the boundary thresholds 0 and 1 are accepted. + /// + [Theory] + [InlineData(0.0)] + [InlineData(1.0)] + public void Constructor_BoundaryThreshold_Accepted(double threshold) + { + // Act + var evaluator = new DecisionLoopEvaluator(Always(0.5), new() { CompletionThreshold = threshold }); + + // Assert + Assert.NotNull(evaluator); + } + + #endregion + + #region Threshold behavior + + /// + /// Verify that a probability at or above the threshold stops (does not request continuation). + /// + [Theory] + [InlineData(0.90, 0.90)] + [InlineData(0.90, 0.95)] + [InlineData(0.90, 1.0)] + [InlineData(0.0, 0.0)] + public async Task EvaluateAsync_AtOrAboveThreshold_StopsAsync(double threshold, double probability) + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Always(probability), new() { CompletionThreshold = threshold }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.False(evaluation.ShouldReinvoke); + } + + /// + /// Verify that a probability below the threshold continues with the default feedback message. + /// + [Theory] + [InlineData(0.90, 0.89)] + [InlineData(0.90, 0.0)] + [InlineData(1.0, 0.999)] + public async Task EvaluateAsync_BelowThreshold_ContinuesWithDefaultFeedbackAsync(double threshold, double probability) + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Always(probability), new() { CompletionThreshold = threshold }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.True(evaluation.ShouldReinvoke); + Assert.Equal(DecisionLoopEvaluator.DefaultContinueFeedbackMessage, evaluation.Feedback); + } + + /// + /// Verify that the default threshold is applied when no options are supplied. + /// + [Fact] + public async Task EvaluateAsync_DefaultOptions_UsesDefaultThresholdAsync() + { + // Arrange + var below = new DecisionLoopEvaluator(Always(DecisionLoopEvaluator.DefaultCompletionThreshold - 0.01)); + var at = new DecisionLoopEvaluator(Always(DecisionLoopEvaluator.DefaultCompletionThreshold)); + + // Act & Assert + Assert.True((await below.EvaluateAsync(CreateContext())).ShouldReinvoke); + Assert.False((await at.EvaluateAsync(CreateContext())).ShouldReinvoke); + } + + #endregion + + #region Feedback + + /// + /// Verify that a custom continuation message is returned as feedback. + /// + [Fact] + public async Task EvaluateAsync_CustomFeedback_IsReturnedAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Always(0.1), new() { ContinueFeedbackMessage = "Keep going." }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.True(evaluation.ShouldReinvoke); + Assert.Equal("Keep going.", evaluation.Feedback); + } + + /// + /// Verify that a null or whitespace continuation message yields a continuation without feedback. + /// + [Theory] + [InlineData(null)] + [InlineData("")] + [InlineData(" ")] + public async Task EvaluateAsync_NullFeedback_ContinuesWithoutFeedbackAsync(string? feedback) + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Always(0.1), new() { ContinueFeedbackMessage = feedback }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.True(evaluation.ShouldReinvoke); + Assert.Null(evaluation.Feedback); + } + + #endregion + + #region Request and state projection + + /// + /// Verify the request sent to the client: one binary question with the default text, criteria, and id, and the + /// default text-only state. + /// + [Fact] + public async Task EvaluateAsync_DefaultProjection_SendsExpectedRequestAsync() + { + // Arrange + var client = new FakeDecisionClient(1.0); + var evaluator = new DecisionLoopEvaluator(client); + LoopContext context = CreateContext( + [new ChatMessage(ChatRole.System, "be brief"), new ChatMessage(ChatRole.User, "original question")], + "partial answer"); + context.Iteration = 3; + + // Act + await evaluator.EvaluateAsync(context); + + // Assert + DecisionRequest request = client.LastRequest!; + BinaryDecisionQuestion question = Assert.IsType(Assert.Single(request.Questions)); + Assert.Equal(DecisionLoopEvaluator.CompletionQuestionId, question.Id); + Assert.Equal(DecisionLoopEvaluator.DefaultCompletionQuestion, question.Instructions); + Assert.NotNull(question.Criteria); + Assert.Equal(DecisionLoopEvaluator.DefaultCompletedDescription, question.Criteria!.TrueDescription); + Assert.Equal(DecisionLoopEvaluator.DefaultIncompleteDescription, question.Criteria.FalseDescription); + Assert.Null(client.LastOptions); + + JsonElement state = request.State; + Assert.Equal(JsonValueKind.Object, state.ValueKind); + Assert.Equal("partial answer", state.GetProperty("latestResponse").GetString()); + Assert.Equal(3, state.GetProperty("iteration").GetInt32()); + + JsonElement messages = state.GetProperty("originalRequest"); + Assert.Equal(2, messages.GetArrayLength()); + Assert.Equal("system", messages[0].GetProperty("role").GetString()); + Assert.Equal("be brief", messages[0].GetProperty("text").GetString()); + Assert.Equal("user", messages[1].GetProperty("role").GetString()); + Assert.Equal("original question", messages[1].GetProperty("text").GetString()); + } + + /// + /// Verify that custom question text and descriptions flow into the request. + /// + [Fact] + public async Task EvaluateAsync_CustomQuestion_FlowsIntoRequestAsync() + { + // Arrange + var client = new FakeDecisionClient(1.0); + var evaluator = new DecisionLoopEvaluator(client, new() + { + CompletionQuestion = "Is the report finished?", + CompletedDescription = null, + IncompleteDescription = "Sections are missing.", + }); + + // Act + await evaluator.EvaluateAsync(CreateContext()); + + // Assert + BinaryDecisionQuestion question = client.LastQuestion!; + Assert.Equal("Is the report finished?", question.Instructions); + Assert.Null(question.Criteria!.TrueDescription); + Assert.Equal("Sections are missing.", question.Criteria.FalseDescription); + } + + /// + /// Verify that when both descriptions are null no criteria object is sent. + /// + [Fact] + public async Task EvaluateAsync_NoDescriptions_OmitsCriteriaAsync() + { + // Arrange + var client = new FakeDecisionClient(1.0); + var evaluator = new DecisionLoopEvaluator(client, new() { CompletedDescription = null, IncompleteDescription = null }); + + // Act + await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.Null(client.LastQuestion!.Criteria); + } + + /// + /// Verify that non-text content in the original request is rejected by the default projection rather than dropped. + /// + [Fact] + public async Task EvaluateAsync_NonTextContent_ThrowsAsync() + { + // Arrange + var client = new FakeDecisionClient(1.0); + var evaluator = new DecisionLoopEvaluator(client); + LoopContext context = CreateContext( + [new ChatMessage(ChatRole.User, [new TextContent("describe this"), new DataContent(new byte[] { 1, 2, 3 }, "image/png")])], + "an answer"); + + // Act & Assert + InvalidOperationException ex = await Assert.ThrowsAsync(async () => await evaluator.EvaluateAsync(context)); + Assert.Contains(nameof(DataContent), ex.Message); + Assert.Contains(nameof(DecisionLoopEvaluatorOptions.StateFactory), ex.Message); + Assert.Null(client.LastRequest); + } + + /// + /// Verify that the non-text rejection is not subject to the failure policy: it is a configuration error. + /// + [Fact] + public async Task EvaluateAsync_NonTextContent_IgnoresFailureBehaviorAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Always(1.0), new() { FailureBehavior = DecisionLoopFailureBehavior.Continue }); + LoopContext context = CreateContext( + [new ChatMessage(ChatRole.User, [new DataContent(new byte[] { 1 }, "image/png")])], + "an answer"); + + // Act & Assert + await Assert.ThrowsAsync(async () => await evaluator.EvaluateAsync(context)); + } + + /// + /// Verify that a custom state factory replaces the default projection and can carry non-text requests. + /// + [Fact] + public async Task EvaluateAsync_StateFactory_ReplacesDefaultProjectionAsync() + { + // Arrange + var client = new FakeDecisionClient(1.0); + LoopContext? factoryContext = null; + var evaluator = new DecisionLoopEvaluator(client, new() + { + StateFactory = context => + { + factoryContext = context; + using JsonDocument document = JsonDocument.Parse("\"custom state\""); + return document.RootElement.Clone(); + }, + }); + LoopContext context = CreateContext( + [new ChatMessage(ChatRole.User, [new DataContent(new byte[] { 1 }, "image/png")])], + "an answer"); + + // Act + await evaluator.EvaluateAsync(context); + + // Assert + Assert.Same(context, factoryContext); + Assert.Equal(JsonValueKind.String, client.LastRequest!.State.ValueKind); + Assert.Equal("custom state", client.LastRequest.State.GetString()); + } + + /// + /// Verify that the evaluator does not mutate the loop context, session, or feedback log. + /// + [Fact] + public async Task EvaluateAsync_DoesNotMutateContextAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Always(0.2)); + LoopContext context = CreateContext(); + AgentSession session = context.Session; + AgentResponse response = context.LastResponse; + + // Act + await evaluator.EvaluateAsync(context); + + // Assert + Assert.Same(session, context.Session); + Assert.Same(response, context.LastResponse); + Assert.Empty(context.Feedback); + Assert.Empty(context.AdditionalProperties); + } + + #endregion + + #region Failure behavior + + /// + /// Verify that a client exception propagates by default. + /// + [Fact] + public async Task EvaluateAsync_ClientThrows_DefaultPropagatesAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator(Throwing(new DecisionClientException(DecisionFailureKind.ProviderUnavailable, "provider down"))); + + // Act & Assert + DecisionClientException ex = await Assert.ThrowsAsync(async () => await evaluator.EvaluateAsync(CreateContext())); + Assert.Equal("provider down", ex.Message); + } + + /// + /// Verify that with the Continue policy a client exception requests another iteration with the configured feedback. + /// + [Fact] + public async Task EvaluateAsync_ClientThrows_ContinuePolicy_ContinuesAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator( + Throwing(new DecisionClientException(DecisionFailureKind.RateLimited, "slow down")), + new() { FailureBehavior = DecisionLoopFailureBehavior.Continue, ContinueFeedbackMessage = "retry" }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.True(evaluation.ShouldReinvoke); + Assert.Equal("retry", evaluation.Feedback); + } + + /// + /// Verify that with the DeferToNextEvaluator policy a client exception yields Stop. + /// + [Fact] + public async Task EvaluateAsync_ClientThrows_DeferPolicy_StopsAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator( + Throwing(new InvalidOperationException("provider down")), + new() { FailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.False(evaluation.ShouldReinvoke); + } + + /// + /// Verify that a response without a binary answer for the completion question is an operational failure under every policy. + /// + [Theory] + [InlineData(DecisionLoopFailureBehavior.Throw)] + [InlineData(DecisionLoopFailureBehavior.Continue)] + [InlineData(DecisionLoopFailureBehavior.DeferToNextEvaluator)] + public async Task EvaluateAsync_MissingOrWrongAnswer_IsOperationalFailureAsync(DecisionLoopFailureBehavior behavior) + { + // Arrange: the client answers a different question id, and with a choice answer. + var client = new FakeDecisionClient(_ => new DecisionResponse(new Dictionary + { + ["other"] = new ChoiceDecisionAnswer("a", new Dictionary { ["a"] = 1.0 }), + })); + var evaluator = new DecisionLoopEvaluator(client, new() { FailureBehavior = behavior }); + + // Act & Assert + switch (behavior) + { + case DecisionLoopFailureBehavior.Throw: + InvalidOperationException ex = await Assert.ThrowsAsync(async () => await evaluator.EvaluateAsync(CreateContext())); + Assert.Contains(DecisionLoopEvaluator.CompletionQuestionId, ex.Message); + break; + case DecisionLoopFailureBehavior.Continue: + Assert.True((await evaluator.EvaluateAsync(CreateContext())).ShouldReinvoke); + break; + default: + Assert.False((await evaluator.EvaluateAsync(CreateContext())).ShouldReinvoke); + break; + } + } + + /// + /// Verify that caller-requested cancellation always propagates, regardless of the failure policy. + /// + [Theory] + [InlineData(DecisionLoopFailureBehavior.Throw)] + [InlineData(DecisionLoopFailureBehavior.Continue)] + [InlineData(DecisionLoopFailureBehavior.DeferToNextEvaluator)] + public async Task EvaluateAsync_Cancelled_PropagatesCancellationAsync(DecisionLoopFailureBehavior behavior) + { + // Arrange + using var cts = new CancellationTokenSource(); + CancellationToken observed = default; + var client = new FakeDecisionClient((_, cancellationToken) => + { + observed = cancellationToken; + cts.Cancel(); + cancellationToken.ThrowIfCancellationRequested(); + return Task.FromResult(Binary(1.0)); + }); + var evaluator = new DecisionLoopEvaluator(client, new() { FailureBehavior = behavior }); + + // Act & Assert + await Assert.ThrowsAnyAsync(async () => await evaluator.EvaluateAsync(CreateContext(), cts.Token)); + Assert.Equal(cts.Token, observed); + } + + /// + /// Verify that an thrown by the client without a caller cancellation request + /// is treated as an ordinary operational failure (for example a provider-side timeout) and follows the policy. + /// + [Fact] + public async Task EvaluateAsync_ProviderTimeout_FollowsFailurePolicyAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator( + Throwing(new OperationCanceledException("provider timeout")), + new() { FailureBehavior = DecisionLoopFailureBehavior.Continue }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.True(evaluation.ShouldReinvoke); + } + + #endregion + + #region Transient failures and logging + + /// + /// Verify that a transient client failure follows the transient policy while a permanent one follows the general policy. + /// + [Fact] + public async Task EvaluateAsync_TransientFailure_FollowsTransientPolicy_PermanentStillThrowsAsync() + { + // Arrange + var options = new DecisionLoopEvaluatorOptions + { + FailureBehavior = DecisionLoopFailureBehavior.Throw, + TransientFailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator, + }; + var transient = new DecisionLoopEvaluator(Throwing(new DecisionClientException(DecisionFailureKind.RateLimited, "slow down")), options); + var permanent = new DecisionLoopEvaluator(Throwing(new DecisionClientException(DecisionFailureKind.Authentication, "bad key")), options); + var unclassified = new DecisionLoopEvaluator(Throwing(new InvalidOperationException("boom")), options); + + // Act & Assert + Assert.False((await transient.EvaluateAsync(CreateContext())).ShouldReinvoke); + await Assert.ThrowsAsync(async () => await permanent.EvaluateAsync(CreateContext())); + await Assert.ThrowsAsync(async () => await unclassified.EvaluateAsync(CreateContext())); + } + + /// + /// Verify that without a transient policy, a transient failure follows the general failure behavior. + /// + [Theory] + [InlineData(DecisionLoopFailureBehavior.Throw)] + [InlineData(DecisionLoopFailureBehavior.Continue)] + [InlineData(DecisionLoopFailureBehavior.DeferToNextEvaluator)] + public async Task EvaluateAsync_TransientFailure_NoTransientPolicy_FollowsFailureBehaviorAsync(DecisionLoopFailureBehavior behavior) + { + // Arrange + var evaluator = new DecisionLoopEvaluator( + Throwing(new DecisionClientException(DecisionFailureKind.Overloaded, "busy")), + new() { FailureBehavior = behavior, TransientFailureBehavior = null }); + + // Act & Assert + switch (behavior) + { + case DecisionLoopFailureBehavior.Throw: + await Assert.ThrowsAsync(async () => await evaluator.EvaluateAsync(CreateContext())); + break; + case DecisionLoopFailureBehavior.Continue: + Assert.True((await evaluator.EvaluateAsync(CreateContext())).ShouldReinvoke); + break; + default: + Assert.False((await evaluator.EvaluateAsync(CreateContext())).ShouldReinvoke); + break; + } + } + + /// + /// Verify that a transient policy of Continue keeps the loop going with feedback while the transient exception is not surfaced. + /// + [Fact] + public async Task EvaluateAsync_TransientFailure_ContinuePolicy_ContinuesWithFeedbackAsync() + { + // Arrange + var evaluator = new DecisionLoopEvaluator( + Throwing(new DecisionClientException(DecisionFailureKind.ProviderUnavailable, "down")), + new() { TransientFailureBehavior = DecisionLoopFailureBehavior.Continue, ContinueFeedbackMessage = "try again" }); + + // Act + LoopEvaluation evaluation = await evaluator.EvaluateAsync(CreateContext()); + + // Assert + Assert.True(evaluation.ShouldReinvoke); + Assert.Equal("try again", evaluation.Feedback); + } + + /// + /// Verify that each evaluation logs the iteration, probability, threshold, and outcome at debug level, without the state. + /// + [Fact] + public async Task EvaluateAsync_LogsDecisionWithoutStateAsync() + { + // Arrange + var logs = new CapturingLoggerFactory(); + var evaluator = new DecisionLoopEvaluator(Always(0.42), new() { CompletionThreshold = 0.9 }, logs); + LoopContext context = CreateContext([new ChatMessage(ChatRole.User, "SECRET-REQUEST")], "SECRET-RESPONSE"); + context.Iteration = 2; + + // Act + await evaluator.EvaluateAsync(context); + + // Assert + (LogLevel level, string message, Exception? exception) entry = Assert.Single(logs.Entries); + Assert.Equal(LogLevel.Debug, entry.level); + Assert.Null(entry.exception); + Assert.Contains("iteration 2", entry.message); + Assert.Contains("0.420", entry.message); + Assert.Contains("0.90", entry.message); + Assert.Contains("continue", entry.message); + Assert.DoesNotContain("SECRET", entry.message); + } + + /// + /// Verify that applying a failure policy logs a warning carrying the exception, its kind, and transience. + /// + [Fact] + public async Task EvaluateAsync_FailurePolicyApplied_LogsWarningAsync() + { + // Arrange + var logs = new CapturingLoggerFactory(); + var exception = new DecisionClientException(DecisionFailureKind.RateLimited, "slow down {\"echo\":\"SECRET-STATE\"}", statusCode: 429); + var evaluator = new DecisionLoopEvaluator(Throwing(exception), new() { TransientFailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator }, logs); + + // Act + await evaluator.EvaluateAsync(CreateContext()); + + // Assert + (LogLevel level, string message, Exception? loggedException) entry = Assert.Single(logs.Entries); + Assert.Equal(LogLevel.Warning, entry.level); + Assert.Null(entry.loggedException); + Assert.Contains(nameof(DecisionClientException), entry.message); + Assert.Contains("RateLimited", entry.message); + Assert.Contains("status 429", entry.message); + Assert.Contains("transient: True", entry.message); + Assert.Contains(nameof(DecisionLoopFailureBehavior.DeferToNextEvaluator), entry.message); + Assert.DoesNotContain("slow down", entry.message); + } + + /// + /// Verify that a propagated failure is not logged as a handled policy (the caller sees the exception instead). + /// + [Fact] + public async Task EvaluateAsync_ThrowPolicy_DoesNotLogWarningAsync() + { + // Arrange + var logs = new CapturingLoggerFactory(); + var evaluator = new DecisionLoopEvaluator(Throwing(new DecisionClientException(DecisionFailureKind.Authentication, "bad key")), loggerFactory: logs); + + // Act & Assert + await Assert.ThrowsAsync(async () => await evaluator.EvaluateAsync(CreateContext())); + Assert.Empty(logs.Entries); + } + + #endregion + + #region Concurrency + + /// + /// Verify that a single evaluator instance can be shared across concurrent evaluations with independent contexts. + /// + [Fact] + public async Task EvaluateAsync_SharedInstance_IsSafeAcrossConcurrentRunsAsync() + { + // Arrange: the probability is derived from the request state so each run has its own expected outcome. + var client = new FakeDecisionClient(async (request, _) => + { + await Task.Yield(); + return Binary(request.State.GetProperty("latestResponse").GetString() == "done" ? 1.0 : 0.0); + }); + var evaluator = new DecisionLoopEvaluator(client); + + // Act + Task[] runs = Enumerable.Range(0, 50) + .Select(i => evaluator.EvaluateAsync(CreateContext(response: i % 2 == 0 ? "done" : "not yet")).AsTask()) + .ToArray(); + LoopEvaluation[] results = await Task.WhenAll(runs); + + // Assert + for (int i = 0; i < results.Length; i++) + { + Assert.Equal(i % 2 != 0, results[i].ShouldReinvoke); + } + } + + #endregion + + #region LoopAgent composition (cheap decision + strong judge cascade) + + /// + /// Verify that when the decision evaluator continues, a later AI judge is not invoked and the agent runs again. + /// + [Fact] + public async Task LoopAgent_DecisionContinues_SkipsLaterJudgeAsync() + { + // Arrange + int judgeCalls = 0; + var judge = new DelegateLoopEvaluator((_, _) => + { + judgeCalls++; + return new ValueTask(LoopEvaluation.Stop()); + }); + var inner = new InnerAgentCapture(call => new AgentResponse([new ChatMessage(ChatRole.Assistant, $"response {call}")])); + var loop = new LoopAgent(inner.Agent, [new DecisionLoopEvaluator(Always(0.2)), judge], new LoopAgentOptions { MaxIterations = 3 }); + + // Act + await loop.RunAsync([new ChatMessage(ChatRole.User, "task")], new ChatClientAgentSession()); + + // Assert + Assert.Equal(3, inner.CallCount); + Assert.Equal(0, judgeCalls); + Assert.Equal(DecisionLoopEvaluator.DefaultContinueFeedbackMessage, inner.MessagesPerCall[1].Single().Text); + } + + /// + /// Verify that when the decision evaluator stops, the later judge runs and can still request continuation. + /// + [Fact] + public async Task LoopAgent_DecisionStops_LaterJudgeDecidesAsync() + { + // Arrange + int judgeCalls = 0; + var judge = new DelegateLoopEvaluator((_, _) => + { + judgeCalls++; + return new ValueTask(judgeCalls == 1 ? LoopEvaluation.Continue("gap: missing summary") : LoopEvaluation.Stop()); + }); + var inner = new InnerAgentCapture(call => new AgentResponse([new ChatMessage(ChatRole.Assistant, $"response {call}")])); + var loop = new LoopAgent(inner.Agent, [new DecisionLoopEvaluator(Always(0.99)), judge], new LoopAgentOptions { MaxIterations = 5 }); + + // Act + await loop.RunAsync([new ChatMessage(ChatRole.User, "task")], new ChatClientAgentSession()); + + // Assert: iteration 1 -> decision stop, judge continue; iteration 2 -> decision stop, judge stop -> loop ends. + Assert.Equal(2, inner.CallCount); + Assert.Equal(2, judgeCalls); + Assert.Equal("gap: missing summary", inner.MessagesPerCall[1].Single().Text); + } + + /// + /// Verify that when both evaluators stop the loop terminates after a single iteration. + /// + [Fact] + public async Task LoopAgent_BothStop_TerminatesAsync() + { + // Arrange + var inner = new InnerAgentCapture(_ => new AgentResponse([new ChatMessage(ChatRole.Assistant, "done")])); + var loop = new LoopAgent(inner.Agent, [new DecisionLoopEvaluator(Always(1.0)), While(static _ => false)], new LoopAgentOptions { MaxIterations = 5 }); + + // Act + await loop.RunAsync([new ChatMessage(ChatRole.User, "task")], new ChatClientAgentSession()); + + // Assert + Assert.Equal(1, inner.CallCount); + } + + /// + /// Verify that with DeferToNextEvaluator a provider failure lets the later evaluator decide. + /// + [Fact] + public async Task LoopAgent_DecisionFailsWithDefer_LaterJudgeRunsAsync() + { + // Arrange + int judgeCalls = 0; + var judge = new DelegateLoopEvaluator((_, _) => + { + judgeCalls++; + return new ValueTask(judgeCalls == 1 ? LoopEvaluation.Continue("judge says more") : LoopEvaluation.Stop()); + }); + var decision = new DecisionLoopEvaluator( + Throwing(new DecisionClientException(DecisionFailureKind.Overloaded, "provider down")), + new() { FailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator }); + var inner = new InnerAgentCapture(call => new AgentResponse([new ChatMessage(ChatRole.Assistant, $"response {call}")])); + var loop = new LoopAgent(inner.Agent, [decision, judge], new LoopAgentOptions { MaxIterations = 5 }); + + // Act + await loop.RunAsync([new ChatMessage(ChatRole.User, "task")], new ChatClientAgentSession()); + + // Assert + Assert.Equal(2, inner.CallCount); + Assert.Equal(2, judgeCalls); + } + + /// + /// Verify that with DeferToNextEvaluator and no later evaluator, a provider failure ends the loop. + /// + [Fact] + public async Task LoopAgent_DecisionFailsWithDefer_OnlyEvaluator_StopsLoopAsync() + { + // Arrange + var decision = new DecisionLoopEvaluator( + Throwing(new DecisionClientException(DecisionFailureKind.Overloaded, "provider down")), + new() { FailureBehavior = DecisionLoopFailureBehavior.DeferToNextEvaluator }); + var inner = new InnerAgentCapture(_ => new AgentResponse([new ChatMessage(ChatRole.Assistant, "response")])); + var loop = new LoopAgent(inner.Agent, decision, new LoopAgentOptions { MaxIterations = 5 }); + + // Act + await loop.RunAsync([new ChatMessage(ChatRole.User, "task")], new ChatClientAgentSession()); + + // Assert + Assert.Equal(1, inner.CallCount); + } + + /// + /// Verify that with the default Throw policy a provider failure surfaces from the loop run. + /// + [Fact] + public async Task LoopAgent_DecisionFailsWithThrow_PropagatesAsync() + { + // Arrange + var decision = new DecisionLoopEvaluator(Throwing(new DecisionClientException(DecisionFailureKind.Authentication, "bad key"))); + var inner = new InnerAgentCapture(_ => new AgentResponse([new ChatMessage(ChatRole.Assistant, "response")])); + var loop = new LoopAgent(inner.Agent, decision, new LoopAgentOptions { MaxIterations = 5 }); + + // Act & Assert + await Assert.ThrowsAsync(() => loop.RunAsync([new ChatMessage(ChatRole.User, "task")], new ChatClientAgentSession())); + } + + #endregion + + #region Helpers + + private static DecisionResponse Binary(double probability) => + new(new Dictionary { [DecisionLoopEvaluator.CompletionQuestionId] = new BinaryDecisionAnswer(probability) }); + + private static FakeDecisionClient Always(double probability) => new(probability); + + private static FakeDecisionClient Throwing(Exception exception) => new((_, _) => throw exception); + + private static LoopContext CreateContext(IReadOnlyList? initialMessages = null, string response = "partial answer") => new( + new Mock().Object, + new ChatClientAgentSession(), + initialMessages ?? [new ChatMessage(ChatRole.User, "original question")], + new AgentResponse([new ChatMessage(ChatRole.Assistant, response)])); + + /// A minimal that answers from a callback and records the last request. + private sealed class FakeDecisionClient : IDecisionClient + { + private readonly Func> _respond; + + public FakeDecisionClient(double probability) + : this((_, _) => Task.FromResult(Binary(probability))) + { + } + + public FakeDecisionClient(Func respond) + : this((request, _) => Task.FromResult(respond(request))) + { + } + + public FakeDecisionClient(Func> respond) + { + this._respond = respond; + } + + public DecisionRequest? LastRequest { get; private set; } + + public DecisionOptions? LastOptions { get; private set; } + + public BinaryDecisionQuestion? LastQuestion => this.LastRequest?.Questions.OfType().FirstOrDefault(); + + public Task GetResponseAsync(DecisionRequest request, DecisionOptions? options = null, CancellationToken cancellationToken = default) + { + this.LastRequest = request; + this.LastOptions = options; + return this._respond(request, cancellationToken); + } + + public object? GetService(Type serviceType, object? serviceKey = null) => null; + + public void Dispose() + { + } + } + + /// A logger factory that records every log entry for assertions. + private sealed class CapturingLoggerFactory : ILoggerFactory + { + public List<(LogLevel Level, string Message, Exception? Exception)> Entries { get; } = []; + + public ILogger CreateLogger(string categoryName) => new CapturingLogger(this); + + public void AddProvider(ILoggerProvider provider) + { + } + + public void Dispose() + { + } + + private sealed class CapturingLogger(CapturingLoggerFactory owner) : ILogger + { + public IDisposable? BeginScope(TState state) + where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) => + owner.Entries.Add((logLevel, formatter(state, exception), exception)); + } + } + + #endregion +}