fix(security): the audit and pentest read each listener's protocol, so Kafka's default GSSAPI and named listeners no longer pass

[bmscomp]

Why

The security audit and the pentest decided SASL and encryption from broker-wide text, not from the listeners. Kafka's defaults and Strimzi's named listeners both broke that. Checked in the code:

• SASL Authentication (check 1) passed when sasl.enabled.mechanisms was non-empty. describeConfigs returns Kafka's default for that key, GSSAPI, on a broker that never sets it. So a broker with only PLAINTEXT://:9092 read "SASL mechanisms: GSSAPI" and passed. SASL Mechanism Strength (check 17) passed the same way, and the pentest's metadata-leak probe reported PROTECTED. SecurityApiIT's own container (KafkaTestResource.java:37, all PLAINTEXT listeners) is such a broker. The IT asserted only the report's shape, so nothing caught it.
• The kafka-cluster chart's Strimzi cluster went the other way. Its plain listener (charts/kafka-cluster/values.yaml:94-99) is SCRAM-SHA-512 with tls: false. Strimzi names it PLAIN-9092 and maps it to SASL_PLAINTEXT, and it sets the mechanisms per listener, so the broker-wide key is empty:
  • check 1 failed, although every client on 9092 authenticates;
  • No Plaintext Listeners (check 2) and the pentest's unencrypted probe passed, because they looked for the text PLAINTEXT://, and PLAIN-9092 never holds it. Check 2 also read advertised.listeners for the same text.
• Listener Protocol Map (check 41) tested contains("PLAINTEXT") && !contains("SASL_PLAINTEXT") on the raw map. Kafka's default map lists all four protocols, so the check passed on every broker that leaves the map unset, and on any map that holds SASL_PLAINTEXT, even beside a PLAINTEXT listener.
• The caveat. security-broker-wide-keys described the PLAINTEXT:// text match and the Strimzi case. No caveat covered the GSSAPI-default false PASS. None of the security refs in kates mcp was anchored, so a ref that moved still passed TestMCPCaveatRefsExist.

Changes

SecurityKafkaHelpers

• listenerProtocols(config): listener name to protocol, in listeners order. The name is the text before ://, upper-cased with Locale.ROOT as Kafka does. The protocol is that name's entry in listener.security.protocol.map, or the name itself when the map has none. A missing or blank listeners counts as Kafka's default PLAINTEXT://:9092.
• sasl(protocol) (SASL_), unencrypted(protocol) (PLAINTEXT or SASL_PLAINTEXT), and saslRequired(protocols): some listener is SASL_ and none is PLAINTEXT.
• listeners(protocols, predicate) formats the matching listeners as PLAIN-9092 (SASL_PLAINTEXT), ....
• saslMechanisms(config, protocols): the union, over the SASL listeners, of listener.name.<listener>.sasl.enabled.mechanisms, each falling back to the broker-wide key.
• All are package-private statics, so both services call them and a mocked or subclassed helpers bean cannot stub them.

SecurityService

• Check 1 passes on saslRequired. Its detail names the SASL listeners. A failure says "No SASL listener", or names the PLAINTEXT listeners when they sit beside a SASL one.
• Check 2 warns when any listener in use is unencrypted, and names them ("traffic sent unencrypted"). Its PASS text is now "All listeners use TLS". advertised.listeners is no longer read: its names are a subset of the listener names.
• Check 17 says "No SASL listener" when there is none. Otherwise it judges the mechanisms from saslMechanisms.
• Check 41 judges the listeners in use, the same rule as check 2. Its detail is Listeners: NAME (PROTOCOL), ... for every listener.

SecurityPentestService

• metadata-leak is VULNERABLE when check 1's rule fails. Its text now covers a PLAINTEXT listener beside a SASL one.
• unencrypted is VULNERABLE when a listener is PLAINTEXT or SASL_PLAINTEXT, and names it.

kates mcp

• security-broker-wide-keys is rewritten under the same id. It says what the four checks and two probes read now, what unencrypted and SASL mean, and that an SSL listener counts as neither. It says the per-listener mechanisms read is not checked on a live cluster yet. It also says the keystore, truststore, mTLS, TLS and certificate checks still read only broker-wide ssl.*, so on the chart's cluster they report no keystore and no client authentication while the tls listener has both.
• Every ref into SecurityService.java, SecurityPentestService.java and SecurityKafkaHelpers.java that this edit moved is re-pointed and wrapped in mcpAnchoredRef:
  • mcp_tools_security.go: security-config-read-silent, security-tls-config-only, security-broker-wide-keys, security-drift-new-checks and security-compliance-by-category;
  • mcp_caveats.go: pentest-config-only, cve-fixed-list, and the two moved refs of security-trend-in-memory. Its 42,58-69 ref does not move and is left as it is.
• The comment refs at mcp_tools_security.go:71 and :713 are re-pointed too.

Other

• skills/kates-cli/SKILL.md: the plaintext bullet matches the new caveat, including the unverified per-listener mechanisms read.
• eval/mcp/oracle.py: the comment ref to SecurityService.java:42,1653-1693 is re-pointed. eval/mcp/tasks.json is not touched.

Tests

• New SecurityListenerProtocolsTest. It uses SecurityServiceTrendTest's harness, with a real SecurityKafkaHelpers whose fetchBrokerConfig is overridden. Each case asserts the report has no error key, because runAudit grades an exception F.
  • strimziChartListeners, run with a broker-wide sasl.enabled.mechanisms of "" and of GSSAPI. It uses the chart's listeners (REPLICATION-9091, PLAIN-9092, TLS-9093) and its map, which also names CONTROLPLANE-9090, plus listener.name.plain-9092.sasl.enabled.mechanisms=SCRAM-SHA-512. Results: SASL Authentication PASS, No Plaintext Listeners WARN naming PLAIN-9092, Listener Protocol Map WARN, SASL Mechanism Strength PASS, metadata-leak PROTECTED, unencrypted VULNERABLE.
  • vanillaPlaintextDefaultGssapi: PLAINTEXT://:9092, the default map, GSSAPI. Results: auth FAIL, mechanism strength WARN, plaintext WARN, protocol map WARN, both probes VULNERABLE.
  • saslSslOnly: SASL_SSL://:9094, SCRAM-SHA-512. Auth, plaintext, protocol map and mechanism strength all PASS, and both probes are PROTECTED.
  • plaintextBesideSasl: a PLAINTEXT listener beside a SASL one fails check 1 and leaks metadata (D1).
• SecurityApiIT now asserts that the PLAINTEXT container fails SASL Authentication and warns on No Plaintext Listeners with BROKER (PLAINTEXT) in the detail. Only a config read from the real broker names that listener; an unread config would show the default PLAINTEXT. It also asserts the container is VULNERABLE to metadata-leak and unencrypted.

Run locally, all passing:

• git diff --name-only origin/main...HEAD: 9 paths under kates/, cli/cmd/, eval/mcp/ and skills/. So the Java and CLI checks apply. Nothing under docs/, charts/, .github/, scripts/, Makefile or *.y(a)ml changed, so the book, chart, actionlint, shellcheck, yamllint, make check-help and docs gen-*.sh --check checks were skipped. eval/ and skills/ trigger no CI job.
• cd kates && ./mvnw -o test -Dquarkus.grpc.codegen.skip=true -Dquarkus.http.test-port=0: 1,471 tests, 0 failures, 0 errors, 0 skipped. SecurityListenerProtocolsTest does not stub describeAcls, so fetchAcls logs a WARN with an NPE stack trace on each audit or pentest call. That is log noise, not a failure.
• spotless:check on JDK 25: exit 0.
• cd cli && go test ./... and go test -p 1 -race -count=1 ./...: all packages ok. The caveat-ref anchor tests pass against the moved lines.
• go vet ./..., golangci-lint run --new-from-rev=origin/main ./... (0 issues), govulncheck ./... (no vulnerabilities).
• scripts/check-cli-style.sh, scripts/check-cli-compat.sh.
• python3 -m unittest discover eval/mcp against the built CLI: 240 tests OK. That is not a CI job, but oracle.py changed. The new SecurityService.java:1653-1693 ref was checked by hand: 1653 is scoreHistory's javadoc, 1693 closes the method.

CI only:

• SecurityApiIT and the other *IT tests (Build & Test runs ./mvnw verify, which needs Docker).
• Dependency CVE scan (Trivy on the Maven dependencies).
• CI · Native image build and smoke test.
• CI · Integration on Kind.
• CI · Docker image build validation.
• CodeQL, Go and Java.
• github-advanced-security, red on every PR from a GitHub-side quota error.

Decisions

• D1. SASL Authentication and metadata-leak pass when some listener is SASL_* and none is PLAINTEXT. A SASL-only rule would let an open PLAINTEXT listener beside a SASL one read "Unauthenticated Metadata: PROTECTED".
• D2. SASL Mechanism Strength reads listener.name.<listener>.sasl.enabled.mechanisms for each SASL listener and falls back to the broker-wide key. Without this, the chart's cluster shows SASL Authentication PASS beside "No SASL mechanisms configured". The caveat says the per-listener read is unverified on a live cluster.
• D3. Check 41 judges the listeners in use. It now gives check 2's verdict under its own CIS id, and both count toward the grade.
• D4. A missing listeners means PLAINTEXT://:9092. An unread config then grades as an unsecured broker, not as "All listeners use TLS or SASL", which fits security-config-read-silent.
• D6. Every security ref this PR re-points is anchored, as in fix(cli): the other caveat refs cite the code they describe, and fail when it moves #295.
• eval/mcp/tasks.json is left alone: it is frozen (eval/mcp/README.md:81-83).

Before merge: the lab DescribeConfigs check. Run kafka-configs.sh --describe --all --entity-type brokers --entity-name <id> against krafter. It uses the same DescribeConfigs call as the audit, and shows two things: whether listener.name.plain-9092.sasl.enabled.mechanisms is returned, and what the broker-wide sasl.enabled.mechanisms is. If the per-listener key is not returned, SASL Mechanism Strength on the chart's cluster reads "No SASL mechanisms configured", as the caveat says.

Deliberately not done

• Grades change with no config change. Say so in the release notes, because a kates security gate --min-grade in CI or a drift against a saved baseline will show it.
  • On the chart's cluster: check 1 FAIL to PASS, checks 2 and 41 PASS to WARN, unencrypted PROTECTED to VULNERABLE.
  • On a vanilla broker with the GSSAPI default: check 1 PASS to FAIL, check 17 PASS to WARN.
• Not changed:
  • eval/mcp/tasks.json (frozen).
  • The hand-made fixtures that quote old detail strings, used only for decoding: cli/client/mcp_security_test.go:16,65 and eval/mcp/testdata/api/security_audit.json:16.
  • The historical refs in plans/ and docs/backend-enhancement-plan.md.
• The verdicts still come from the first broker listed (caveat security-config-read-silent).
• Follow-up: check 11 reads security.inter.broker.protocol. Kafka ignores that key when inter.broker.listener.name is set, and Strimzi sets it to REPLICATION-9091. So check 11 gives a false WARN on Strimzi, from the same root cause.

Landing

Lands after #376 (plan PR 7), #377 (plan PR 8) and #378 (plan PR 9). Each edits cli/cmd/mcp_caveats.go, the one file this PR shares with them:

• fix(engine): each INTEGRITY run reads in a group of its own, and a consumer with no committed offset starts at its task's first record, not the topic's oldest #376 turns :105-106 into four lines and edits :117-118.
• fix(engine): a status poll writes only over the status it read, so a cancelled run stays cancelled #377 edits :70, :92, :115-116, :159 and :171.
• fix(engine): run retention keeps the run a baseline names, so regression checks keep working #378 edits :171.

This PR edits :129 (into two lines), :137-138 and :150-151. Those are disjoint from the lines above, and git merge-tree merges this branch with each of the three cleanly. None of them touches SecurityService.java, SecurityPentestService.java or SecurityKafkaHelpers.java, so no ref this PR points at moves.

Recipe once those have merged:

1. Rebase onto main.
2. If a ref into the security files did move, re-point it from a diff line map. The refs are now anchored, so TestMCPCaveatRefAnchors fails on a ref that was missed.
3. Rerun cd cli && go test ./cmd -run TestMCPCaveat -count=1.