Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions docs/specs/004-python-function-calling-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,9 +191,12 @@ Before acting on a model function-call batch, the loop classifies every actionab
call aborts the complete batch before approval state changes or execution. Otherwise approval-required and Host-owned
calls are returned together in model order, while session-backed executable siblings remain deferred. An incomplete
session-backed mixed approval/Host response remains pending without executing a deferred call; a stateless incomplete
response is rejected. Correlation is scoped to the active mixed batch so completed or abandoned historical Host calls
remain unchanged. `ToolApprovalMiddleware` may resolve approval requests through standing or automatic policies, but
it preserves non-approval user-input requests and does not split manual approvals away from their Host-owned siblings.
response is rejected, including when no response is supplied, message roles vary within the model output, or a newer
standalone request follows the incomplete batch. Stateless discovery selects the latest unresolved mixed batch, so a
completed batch remains inert on later turns. Correlation is scoped to the active mixed batch so completed or abandoned
historical Host calls remain unchanged.
`ToolApprovalMiddleware` may resolve approval requests through standing or automatic policies, but it preserves
non-approval user-input requests and does not split or reorder manual approvals relative to their Host-owned siblings.

### Reasoning-bound function-call groups

Expand Down Expand Up @@ -612,15 +615,15 @@ that manually replay messages own the equivalent rule: do not resend an approval
| Scenario | Required invariant | Primary regression test |
|---|---|---|
| Fatal call mixed with pauses | Complete-batch classification raises before approval or execution, independent of call order. | `packages/core/tests/core/test_function_invocation_logic.py::test_mixed_batch_fatal_unknown_precedes_every_pause` |
| Approval and Host-owned calls | Both pause types are returned in model order; a session-backed partial response remains pending across serialization; historical Host calls do not participate; a complete response executes the exact approved arguments once. | `test_mixed_batch_returns_approval_and_host_pause_in_model_order`, `test_mixed_batch_requires_complete_responses_before_execution`, `test_active_mixed_pause_ignores_historical_host_requests` |
| Approval and Host-owned calls | Both pause types are returned in model order; a session-backed partial response remains pending across serialization; a stateless zero-response or partial response fails closed across message roles and cannot be hidden by a newer standalone request; completed mixed batches remain inert; historical Host calls do not participate; a complete response executes the exact approved arguments once. | `test_mixed_batch_returns_approval_and_host_pause_in_model_order`, `test_mixed_batch_requires_complete_responses_before_execution`, `test_stateless_split_mixed_batch_rejects_incomplete_replay_before_execution`, `test_stateless_mixed_batch_across_message_roles_requires_complete_responses`, `test_later_standalone_request_does_not_hide_incomplete_stateless_mixed_batch`, `test_completed_split_stateless_mixed_batch_is_inert_on_later_turn`, `test_active_mixed_pause_ignores_historical_host_requests` |
| Safe and approval-required calls in one batch | Hidden safe calls replay only with the matching visible approval. | `packages/core/tests/core/test_harness_tool_approval.py::test_mixed_batch_hides_already_approved_request_until_approval_replay` |
| Restored approval state | Serialized `ToolApprovalState` restores mixed-batch behavior. | `test_mixed_batch_accepts_restored_tool_approval_state` |
| Unrelated turn before approval | Hidden calls do not execute on an unrelated turn. | `test_hidden_mixed_batch_requests_do_not_replay_on_unrelated_turn` |
| Multiple abandoned batches | Hidden calls replay only for the matching batch. | `test_hidden_mixed_batch_requests_replay_only_for_matching_visible_approval` |
| Queued approvals | One unresolved approval is surfaced per run without premature execution. | `test_tool_approval_middleware_queues_multiple_approval_requests`, `test_tool_approval_middleware_queues_streamed_approval_requests` |
| Middleware state plus hidden core state | State saves do not discard hidden mixed-batch calls. | `test_tool_approval_middleware_preserves_hidden_mixed_batch_requests` |
| Auto-approval callback | Callback receives the original function call and executes the approved set once. | `test_tool_approval_middleware_auto_approval_rule_receives_function_call` |
| Approval policy with Host-owned sibling | Reordering the same calls has the same outcome; auto-approved and safe calls remain deferred until Host input arrives; manual approvals stay in the mixed batch; an approved Host tool returns to Host-owned handling before local execution. | `test_tool_approval_middleware_mixed_batch_is_order_independent`, `test_tool_approval_middleware_auto_approves_with_host_pause_and_cached_safe_call`, `test_tool_approval_middleware_keeps_manual_approvals_together_with_host_pause`, `test_tool_approval_middleware_policy_approval_reclassifies_host_tool` |
| Approval policy with Host-owned sibling | Reordering the same calls has the same outcome; auto-approved and safe calls remain deferred until Host input arrives; separately streamed Host input stays immediately visible while later manual approvals remain in model order and in the same batch; an approved Host tool returns to Host-owned handling before local execution. | `test_tool_approval_middleware_mixed_batch_is_order_independent`, `test_tool_approval_middleware_auto_approves_with_host_pause_and_cached_safe_call`, `test_tool_approval_middleware_keeps_manual_approvals_together_with_host_pause`, `test_tool_approval_middleware_preserves_split_streamed_mixed_batch_order`, `test_tool_approval_middleware_streams_standalone_user_input_immediately`, `test_tool_approval_middleware_policy_approval_reclassifies_host_tool` |
| Shared call budget | Auto-approved re-entry does not reset `max_function_calls`, and every executed approval group counts even when it pauses for input. | `test_tool_approval_middleware_auto_approved_loops_share_function_call_budget`, `test_approval_resume_user_input_counts_toward_function_call_budget` |
| Standing tool rule | Tool-level approval applies only to later matching tools. | `test_tool_approval_middleware_always_approve_tool_rule` |
| Forged standing rule | An unbound or substituted hosted response cannot create a standing middleware approval rule for caller-selected metadata. | `test_tool_approval_middleware_drops_forged_standing_approval`, `test_tool_approval_middleware_rebinds_hosted_standing_approval` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,10 @@ public string? Compatibility
/// <summary>
/// Gets or sets the arbitrary key-value metadata for this skill.
/// </summary>
/// <remarks>
/// Keys are compared case-insensitively. When parsed from a SKILL.md file, duplicate
/// entries retain the first value and key spelling and produce warnings without rejecting the skill.
/// </remarks>
public AdditionalPropertiesDictionary? Metadata { get; set; }

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ namespace Microsoft.Agents.AI;
/// <remarks>
/// Searches directories recursively (up to 2 levels deep) for SKILL.md files.
/// Symbolic links and reparse points below configured roots are not followed during skill discovery.
/// Each file is validated for YAML frontmatter. Resource and script files are discovered by scanning the skill
/// Recognized top-level frontmatter fields must use lowercase names and must not be repeated.
/// Within the optional metadata mapping, keys are compared case-insensitively. The first
/// value is retained for duplicate keys, and subsequent entries produce warnings without rejecting the skill.
/// Resource and script files are discovered by scanning the skill
/// directory for files with matching extensions. Invalid resources are skipped with logged warnings.
/// Resource and script paths are checked against path traversal and symlink escape attacks.
/// Discovered files are revalidated against their trusted skill directory immediately before use.
Expand All @@ -36,19 +39,31 @@ public sealed partial class AgentFileSkillsSource : AgentSkillsSource
private static readonly string[] s_defaultScriptExtensions = [".py", ".js", ".sh", ".ps1", ".cs", ".csx"];
private static readonly string[] s_defaultResourceExtensions = [".md", ".json", ".yaml", ".yml", ".csv", ".xml", ".txt"];

// Case-insensitive lookup identifies known fields; values preserve the required spelling for validation.
private static readonly Dictionary<string, string> s_frontmatterFieldNames = new(StringComparer.OrdinalIgnoreCase)
{
["name"] = "name",
["description"] = "description",
["license"] = "license",
["compatibility"] = "compatibility",
["metadata"] = "metadata",
["allowed-tools"] = "allowed-tools",
};

// Matches YAML frontmatter delimited by "---" lines. Group 1 = content between delimiters.
// Multiline makes ^/$ match line boundaries; Singleline makes . match newlines across the block.
// The \uFEFF? prefix allows an optional UTF-8 BOM that some editors prepend.
private static readonly Regex s_frontmatterRegex = new(@"\A\uFEFF?^---\s*$(.+?)^---\s*$", RegexOptions.Multiline | RegexOptions.Singleline | RegexOptions.Compiled, TimeSpan.FromSeconds(5));

// Matches top-level YAML "key: value" lines. Group 1 = key (supports hyphens for keys like allowed-tools),
// Group 2 = quoted value, Group 3 = unquoted value.
// Accepts single or double quotes; the lazy quantifier trims trailing whitespace on unquoted values.
private static readonly Regex s_yamlKeyValueRegex = new(@"^([\w-]+)\s*:\s*(?:[""'](.+?)[""']|(.+?))\s*$", RegexOptions.Multiline | RegexOptions.Compiled, TimeSpan.FromSeconds(5));
// Matches top-level YAML "key: value" lines. Group 1 = key (including quotes),
// Group 2 = quoted value, Group 3 = unquoted value (possibly empty for keys such as "metadata:").
// A value may start on a later indented line, but cannot consume another top-level field.
// Retain trailing whitespace matching so block-scalar parsing handles leading blank lines as before.
private static readonly Regex s_yamlKeyValueRegex = new(@"^([\w-]+|""[\w-]+""|'[\w-]+')[ \t]*:[ \t]*(?:\r?\n(?:[ \t]*\r?\n)*[ \t]+)?(?:[""']([^\r\n]+?)[""']|([^\r\n]*?))\s*$", RegexOptions.Multiline | RegexOptions.Compiled, TimeSpan.FromSeconds(5));

// Matches a "metadata:" line followed by indented sub-key/value pairs.
// Matches a metadata block, allowing quotes around the root key.
// Group 1 captures the entire indented block beneath the metadata key.
private static readonly Regex s_yamlMetadataBlockRegex = new(@"^metadata\s*:\s*$\n((?:[ \t]+\S.*\n?)+)", RegexOptions.Multiline | RegexOptions.Compiled, TimeSpan.FromSeconds(5));
private static readonly Regex s_yamlMetadataBlockRegex = new(@"^(?:metadata|""metadata""|'metadata')\s*:\s*$\n((?:[ \t]+\S.*\n?)+)", RegexOptions.Multiline | RegexOptions.Compiled, TimeSpan.FromSeconds(5));

// Matches indented YAML "key: value" lines within a metadata block.
// Group 1 = key (supports hyphens), Group 2 = quoted value, Group 3 = unquoted value.
Expand Down Expand Up @@ -235,30 +250,59 @@ private bool TryParseFrontmatter(string content, string skillFilePath, [NotNullW
string? compatibility = null;
string? allowedTools = null;

// Recognized fields use exact lowercase names, with optional quotes. Reject casing variants and
// duplicates rather than silently changing which value the skill exposes.
var seenFields = new HashSet<string>(StringComparer.Ordinal);
foreach (Match kvMatch in s_yamlKeyValueRegex.Matches(yamlContent))
{
string key = kvMatch.Groups[1].Value;
key = key[0] is '"' or '\'' ? key.Substring(1, key.Length - 2) : key;

// Unknown fields are intentionally excluded from this validation for forward compatibility.
if (!s_frontmatterFieldNames.TryGetValue(key, out string? canonicalKey))
{
continue;
}

if (!string.Equals(key, canonicalKey, StringComparison.Ordinal))
{
LogIncorrectlyCasedFrontmatterField(this._logger, skillFilePath, key, canonicalKey);
return false;
}

if (!seenFields.Add(key))
{
LogDuplicateFrontmatterField(this._logger, skillFilePath, key);
return false;
}

// Empty declarations participate in key validation, but leave optional scalar fields unset.
if (!kvMatch.Groups[2].Success && kvMatch.Groups[3].Length == 0)
{
continue;
}

string value = kvMatch.Groups[2].Success
? kvMatch.Groups[2].Value
: ParseYamlScalarValue(yamlContent, kvMatch);

if (string.Equals(key, "name", StringComparison.OrdinalIgnoreCase))
if (string.Equals(key, "name", StringComparison.Ordinal))
{
name = value;
}
else if (string.Equals(key, "description", StringComparison.OrdinalIgnoreCase))
else if (string.Equals(key, "description", StringComparison.Ordinal))
{
description = value;
}
else if (string.Equals(key, "license", StringComparison.OrdinalIgnoreCase))
else if (string.Equals(key, "license", StringComparison.Ordinal))
{
license = value;
}
else if (string.Equals(key, "compatibility", StringComparison.OrdinalIgnoreCase))
else if (string.Equals(key, "compatibility", StringComparison.Ordinal))
{
compatibility = value;
}
else if (string.Equals(key, "allowed-tools", StringComparison.OrdinalIgnoreCase))
else if (string.Equals(key, "allowed-tools", StringComparison.Ordinal))
{
allowedTools = value;
}
Expand All @@ -272,7 +316,14 @@ private bool TryParseFrontmatter(string content, string skillFilePath, [NotNullW
metadata = [];
foreach (Match kvMatch in s_yamlIndentedKeyValueRegex.Matches(metadataMatch.Groups[1].Value))
{
metadata[kvMatch.Groups[1].Value] = kvMatch.Groups[2].Success ? kvMatch.Groups[2].Value : kvMatch.Groups[3].Value;
string key = kvMatch.Groups[1].Value;
string value = kvMatch.Groups[2].Success ? kvMatch.Groups[2].Value : kvMatch.Groups[3].Value;

// Keep the first value and key spelling using the dictionary's case-insensitive comparison.
if (!metadata.TryAdd(key, value))
{
LogDuplicateMetadataKey(this._logger, skillFilePath, key);
}
}
}

Expand Down Expand Up @@ -722,6 +773,15 @@ private static void ValidateExtensions(IEnumerable<string>? extensions)
[LoggerMessage(LogLevel.Error, "SKILL.md at '{SkillFilePath}' has an invalid '{FieldName}' value: {Reason}")]
private static partial void LogInvalidFieldValue(ILogger logger, string skillFilePath, string fieldName, string reason);

[LoggerMessage(LogLevel.Error, "SKILL.md at '{SkillFilePath}' contains duplicate frontmatter field '{FieldName}'")]
private static partial void LogDuplicateFrontmatterField(ILogger logger, string skillFilePath, string fieldName);

[LoggerMessage(LogLevel.Warning, "SKILL.md at '{SkillFilePath}' contains duplicate metadata key '{Key}'; keeping the first value")]
private static partial void LogDuplicateMetadataKey(ILogger logger, string skillFilePath, string key);

[LoggerMessage(LogLevel.Error, "SKILL.md at '{SkillFilePath}' uses incorrectly cased frontmatter field '{FieldName}'; expected '{ExpectedFieldName}'")]
private static partial void LogIncorrectlyCasedFrontmatterField(ILogger logger, string skillFilePath, string fieldName, string expectedFieldName);

[LoggerMessage(LogLevel.Error, "SKILL.md at '{SkillFilePath}': skill name '{SkillName}' does not match parent directory name '{DirectoryName}'")]
private static partial void LogNameDirectoryMismatch(ILogger logger, string skillFilePath, string skillName, string directoryName);

Expand Down
Loading
Loading