Skip to content

Commit cd210c3

Browse files
Document agents.contextFilters and the coding agent step check (#2088)
Closes the documentation half of [CPL-1149](https://linear.app/sourcegraph/issue/CPL-1149). `experimentalFeatures.agents.contextFilters` shipped with Agentic Batch Changes, and this PR updates the according docs. --------- Co-authored-by: Amp <amp@ampcode.com>
1 parent 24f42d9 commit cd210c3

2 files changed

Lines changed: 58 additions & 13 deletions

File tree

‎docs/agentic-batch-changes/configuration.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,10 @@ A step references a secret by name. Sourcegraph injects the value when the step
9898

9999
All code-modifying work runs in isolated containers on [Sourcegraph executors](/admin/executors), one per repository. A container sees the repository and the secrets its steps reference. It has no direct access to code host credentials, other users' secrets, or other repository checkouts. [Coding agent steps](#coding-agent-steps) can still use Sourcegraph MCP to access code with your permissions. Containers can't publish. Publication happens inside Sourcegraph.
100100

101+
### Context filters
102+
103+
Administrators can exclude repositories and files from the agent's Sourcegraph tools with [context filters](/context-filters), configured under `experimentalFeatures.agents.contextFilters`. Excluded content is invisible to the agent's research and to the Sourcegraph MCP tools in its coding agent steps, and a repository excluded as a whole is dropped from the agent's workspaces. Because a coding agent step works on a full checkout, it [does not run at all](/context-filters#coding-agent-steps) in a repository that a rule excludes or that contains an excluded file. Filters do not remove files from the checkout that `run` steps execute on.
104+
101105
### Audit log
102106

103107
Agent activity is recorded in the Sourcegraph audit log under the `batchChangeAgents` entity, with identifiers and no URLs, links, or tokens. On self-hosted instances, route it as you route the rest of the audit log.

‎docs/context-filters.mdx‎

Lines changed: 54 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,35 @@
1-
# Deep Search Context Filters
1+
# Agent Context Filters
22

33
<TierCallout>
44
Supported on [Enterprise](/pricing/plans/enterprise) plans. Configuring
55
context filters requires administrator access.
66
</TierCallout>
77

8-
Context filters are an admin-defined exclusion list for [Deep Search](/deep-search). Anything matched by a filter is invisible to the agent: it never appears in an answer, and it cannot be read, searched, diffed, or resolved by any Deep Search tool.
8+
Context filters are an admin-defined exclusion list for the agents that run inside [Deep Search](/deep-search) and [Agentic Batch Changes](/agentic-batch-changes). Anything matched by a filter is invisible to the agent.
9+
10+
Two site configuration keys hold the rules, and both have the same shape:
11+
12+
- `experimentalFeatures.agents.contextFilters` applies to both agents. Put new rules here.
13+
- `experimentalFeatures.deepSearch.contextFilters` is the older key that predates Agentic Batch Changes. It stays effective, and its rules also apply to both agents, so you do not need to move existing rules. See [Which key to use](#which-key-to-use).
914

1015
## Why use context filters
1116

12-
- **Keep secrets out of the agent's context.** Excluded content never leaves your instance, so it cannot be quoted, summarized, or forwarded to a third-party model provider.
13-
- **Reduce noise.** Vendored dependencies, generated code, and archived repositories can be hidden so Deep Search spends its research on code that matters.
17+
- **Keep secrets out of the agent's context.** Excluded content is never returned by the agent's tools, so the agent cannot quote it, summarize it, or forward it to a third-party model provider.
18+
- **Reduce noise.** Vendored dependencies, generated code, and archived repositories can be hidden so the agent spends its research on code that matters.
1419

1520
## How context filters work
1621

17-
Filtering happens before any content reaches the LLM and does not rely on prompting, so the model cannot reason it's way around the filters.
22+
Filtering happens in the tools the agent uses to read repositories, before any content reaches the LLM, and does not rely on prompting, so the model cannot reason its way around the filters.
1823

19-
To Deep Search, excluded content is indistinguishable from content that does not exist.
24+
To the agent, excluded content is indistinguishable from content that does not exist. Agentic Batch Changes adds one check on top of this: a [coding agent step](#coding-agent-steps) works on a full checkout and therefore cannot run at all in a repository that contains excluded content. Other batch change steps, such as `run` steps, execute on an unfiltered checkout; see [Agentic Batch Changes](#agentic-batch-changes).
2025

2126
## Configuring context filters
2227

2328
An administrator configures filters under **Administration → Configuration → Advanced configuration**, inside the `experimentalFeatures` block of the advanced configuration. Each entry in `exclude` is an independent rule, and content is hidden if it matches any rule.
2429

2530
```jsonc
2631
"experimentalFeatures": {
27-
"deepSearch.contextFilters": {
32+
"agents.contextFilters": {
2833
"exclude": [
2934
// Hide an entire repository.
3035
{ "repoNamePatterns": ["^github\\.com/acme/secrets$"] },
@@ -40,7 +45,13 @@ An administrator configures filters under **Administration → Configuration →
4045
}
4146
```
4247

43-
Config changes apply to the next Deep Search run; no restart is required. A rule must set at least one of the two fields.
48+
Config changes apply to the next agent run; no restart is required. `exclude` must contain at least one rule, a rule must set at least one of the two fields, and each field you set must list at least one pattern. To turn filtering off, remove the key; an empty `exclude` list is rejected. Saving an invalid pattern through the site admin UI is rejected. If an invalid pattern reaches the running configuration another way, for example through `SITE_CONFIG_FILE`, the agents block every repository until it is fixed rather than running with no filters.
49+
50+
### Which key to use
51+
52+
Rules from `agents.contextFilters` and `deepSearch.contextFilters` are combined into one list, and every agent applies the whole list. A rule in either key hides the same content from Deep Search, from the Agentic Batch Changes agent, and from its coding agent steps. There is no way to exclude content from one agent only.
53+
54+
Add new rules to `agents.contextFilters`. Rules you already have in `deepSearch.contextFilters` keep working and need no migration; if both keys are set, their rules are simply combined.
4455

4556
## Pattern syntax
4657

@@ -49,9 +60,9 @@ Both `repoNamePatterns` and `filePathPatterns` take a list of Go [RE2](https://g
4960
- **Patterns are unanchored.** `internal` matches `github.com/acme/internal-tools`. Use `^...$` when you mean the whole string.
5061
- **`.` matches any character except a newline, including `/`.** Escape it as `github\.com`. The pattern `a.b` also matches the repository `a/b`.
5162
- **Backslashes must be doubled in JSON.** The regex `\.env$` is written `"\\.env$"`.
52-
- **Repository names and file paths match case-insensitively.** `README` also excludes `readme`.
53-
- **RE2 does not support backreferences or lookaround assertions.** Patterns like `(a)\1` is rejected.
54-
- **Patterns in a list are combined with OR** `["^a$", "^b$"]` matches either.
63+
- **Repository names match case-insensitively; file paths match case-sensitively.** `acme/Secrets` excludes `github.com/acme/secrets`, but `README` does not exclude `readme.md`. Prefix a file pattern with `(?i)` to make it case-insensitive: `"(?i)readme"`.
64+
- **RE2 does not support backreferences or lookaround assertions.** Patterns like `(a)\1` are rejected.
65+
- **Patterns in a list are combined with OR.** `["^a$", "^b$"]` matches either.
5566

5667
### What the patterns are matched against
5768

@@ -75,7 +86,7 @@ Within a single rule the two fields are combined with AND, and an omitted field
7586
{"repoNamePatterns": ["^github\\.com/acme/secrets$"]}
7687
```
7788

78-
`github.com/acme/secrets` disappears completely: it is dropped from repository lists and search results, its refs and revisions cannot be resolved, and every tool reports it as nonexistent.
89+
`github.com/acme/secrets` disappears completely: it is dropped from repository lists and search results, its refs and revisions cannot be resolved, and every repository tool reports it as nonexistent. A [coding agent step](#coding-agent-steps) targeting it fails with an explicit message instead.
7990

8091
Anchoring matters here. `^github\.com/acme/secrets$` leaves `github.com/acme/secrets-docs` visible, while the unanchored `acme/secrets` hides both.
8192

@@ -85,7 +96,7 @@ Anchoring matters here. `^github\.com/acme/secrets$` leaves `github.com/acme/sec
8596
{"filePathPatterns": ["\\.env$", "^secrets/"]}
8697
```
8798

88-
Every `.env` file, and everything under a top-level `secrets/` directory, is hidden in all repositories. The repositories themselves stay searchable, only the matching files go missing. A files-only rule never hides a repository, so `github.com/acme/backend` still appears in repository lists even if all of its files are filtered.
99+
Every `.env` file, and everything under a top-level `secrets/` directory, is hidden in all repositories. The repositories themselves stay searchable, only the matching files go missing. A files-only rule never hides a repository, so `github.com/acme/backend` still appears in repository lists even if all of its files are filtered. The exception is [coding agent steps](#coding-agent-steps), which cannot run in a repository that contains even one matching file.
89100

90101
### Files within a repository
91102

@@ -98,6 +109,36 @@ Every `.env` file, and everything under a top-level `secrets/` directory, is hid
98109

99110
`deploy/` and `config/prod/` are hidden in `github.com/acme/backend` only. The same paths in a different repo (e.g., `github.com/acme/frontend`) stay visible.
100111

112+
## Agentic Batch Changes
113+
114+
The [Agentic Batch Changes](/agentic-batch-changes) agent applies the combined rules everywhere it reads code through Sourcegraph: when it researches with Deep Search, when it reads files, searches, and resolves repositories directly, and in the [Sourcegraph MCP](/api/mcp) tools available inside its coding agent steps. When the agent's batch spec is resolved into workspaces, repositories matched by a rule that has only `repoNamePatterns` are dropped, so the agent cannot change them even if its spec names them. A rule with `filePathPatterns` does not drop a workspace, even when its `repoNamePatterns` match; it hides the matching files from the agent's tools and, for coding agent steps, blocks the repository as described below.
115+
116+
Workspace filtering applies only to batch specs the agent manages. A batch spec you write and run yourself keeps every workspace, but its coding agent steps are still subject to the check below.
117+
118+
Steps run on a full checkout of the repository in an executor container, and context filters do not remove files from that checkout. A `run` step's script can read any file in the repository, including excluded ones. Coding agent steps are the exception: because a coding agent reads the checkout freely, Sourcegraph refuses to start it rather than let it see excluded content.
119+
120+
### Coding agent steps
121+
122+
A [coding agent step](/agentic-batch-changes/configuration#coding-agent-steps) runs a coding agent on a full checkout of the repository, where individual files cannot be hidden. Instead of hiding files, Sourcegraph refuses to run coding agent steps in a repository when any of the following is true:
123+
124+
- The repository matches a rule that has only `repoNamePatterns`.
125+
- The repository contains at least one file matching a `filePathPatterns` rule that applies to it, either a files-only rule or a rule whose `repoNamePatterns` match the repository. Sourcegraph checks this with a path search at the commit the workspace uses, so one `.env` file anywhere in the repository is enough.
126+
- The repository has a `.sourcegraph/ignore` file, and at least one `filePathPatterns` rule applies to the repository. The ignore file hides paths from search, so Sourcegraph cannot verify that no excluded file is present and fails closed.
127+
128+
The check runs before the executor job for the repository starts, and only when a coding agent step would actually run: an execution whose coding agent steps are all skipped by an `if:` condition or served from the execution cache does not trigger it. When the check blocks an execution, the executor job never starts, so the `run` steps in that job do not run either. Coding agent steps in [changeset hooks](/agentic-batch-changes/configuration#changeset-hooks) are checked the same way.
129+
130+
A blocked repository fails its execution with one of these messages, which the agent sees as well:
131+
132+
- "Coding agent steps cannot run in this repository because it is excluded by the agent context filters in the site configuration (experimentalFeatures.agents.contextFilters)."
133+
- "Coding agent steps cannot run in this repository because it contains files excluded by the agent context filters in the site configuration (experimentalFeatures.agents.contextFilters)."
134+
- "Coding agent steps cannot run in this repository because it has a .sourcegraph/ignore file, which hides files from the search that enforces the agent context filters in the site configuration (experimentalFeatures.agents.contextFilters)."
135+
136+
The message names neither the rule nor the matching file. If the path search itself fails or returns incomplete results, the execution fails with "could not verify the repository against the agent context filters; please retry or contact your administrator" and can be retried.
137+
138+
A blocked coding agent step does not consume a unit of the user's coding agent step [entitlement](/agentic-batch-changes/configuration#usage-limits).
139+
101140
## Verifying a filter
102141

103142
Ask Deep Search a question that would require the excluded content, and @-mention the repository or file directly. A working filter produces an answer that does not reference the content, and the [list of sources](/deep-search) contains none of it. If you @-mention filtered content, Deep Search reports it as not found rather than acknowledging that it was excluded.
143+
144+
To verify the coding agent step check, add a files-only rule that matches a file in a test repository, for example `{"filePathPatterns": ["^context-filter-canary\\.txt$"]}`, and run an agentic batch change with a coding agent step against that repository. The execution for that repository fails with the "contains files excluded" message above instead of running the step. A repository-only rule cannot be verified this way, because the agent drops that repository's workspace before anything executes; to see the "is excluded" message, run a batch spec you wrote yourself with a `codingAgent` step against the excluded repository.

0 commit comments

Comments
 (0)