Repository navigation
Conversation
### What problem does this PR solve?
Problem Summary: At verbose log levels, FragmentMgr formats complete pipeline parameters and external query plans. File scan properties, table descriptors and transaction parameters can contain storage credentials, passwords or tokens. Replace these dumps with a shared summary of query and fragment identifiers and instance counts, and keep only identifiers in the external open log. Execution parameters remain unchanged.
### Release note
Verbose fragment execution logs no longer include full execution parameters, query options or external query plans that may contain credentials.
### Check List (For Author)
- Test: clang-format 16 and whitespace checks passed; focused BE unit tests and ASAN build are in progress
- Unit Test: add synthetic credential coverage and capture both pipeline and external request logs
- Behavior changed: Yes; verbose diagnostics contain identifiers and counts instead of full parameter dumps
- Does this need documentation: No
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
No actionable new findings in the five changed files at head 01ffe8c. No existing inline findings require carry-forward. No additional user focus was supplied.
Critical checkpoint conclusions:
- Goal and evidence: all three FragmentMgr full request/options/external-plan logging sites now emit only numeric identifiers and instance counts. The formatter does not read credential-bearing strings or mutate execution parameters. Five added tests cover summaries, unknown credential properties, defaults, immutability and actual internal/external logging sites; inspected statically only.
- Scope: the shared const formatter and logging replacements are small and focused.
- Concurrency: request-local reads add no shared state or locks; the test sink protects its buffer with a mutex. Existing dispatch/map locking remains intact.
- Lifecycle/static initialization: no production ownership or static initialization changes. Tests stop/join the manager before removing the sink and restore logging/debug-point state; native harness initialization and ExecEnv teardown were traced.
- Configuration: no production configuration added or changed.
- Compatibility: no wire/storage format or existing interface changes; helper declaration, includes and source collection are consistent with this branch.
- Thrift/protobuf: no definitions modified; existing field types and consumers were checked.
- Parallel paths: normal/prepare RPC, callback, transaction stream-load and external-open paths reach the sanitized common overload; the external pre-construction log is also replaced.
- Conditions/error handling: no production branches or Status propagation changed; default/absent fields do not require indexing or credential-specific filtering.
- Test coverage: both files are included by the native test glob. Site-level tests assert the intended early debug-point failure and log contents; they do not exercise later preparation/execution.
- Expected results: no golden files changed; exact ID/count assertions agree with the formatter.
- Observability: query/fragment correlation and instance count remain available; external instance correlation is retained. Removing full diagnostic payloads is intentional; no new metrics are needed.
- Transactions/persistence: no commit, EditLog, version, delete-bitmap or failover logic changes.
- Writes/atomicity/crashes: no execution-data mutation or write-order change; committed-data visibility and memory ownership invariants are unaffected.
- FE/BE variables: no transmitted variable or sending path changed.
- Performance/memory: bounded fixed-field formatting replaces recursive full-object serialization; disabled VLOG evaluation remains lazy, with no significant retained allocation.
- Other risks: upstream/downstream dumps, shared parent parameters, external descriptor creation and fixture global-state interactions were checked and dispositioned; no unresolved in-scope candidate remains.
Coverage limits: this was a static review; no build or test was run, and generated-header compilation was not verified. A separate pre-existing HTTP stream-load parameter dump remains outside these edited FragmentMgr logging sites, so this review does not establish a repository-wide absence of sensitive logs. All changed-file coverage and the two identified risk questions are complete.
Review converged after 2 round(s).
What problem does this PR solve?
Related PR: #69540
Problem Summary: FragmentMgr prints complete pipeline execution parameters at VLOG(10), including file-scan properties, descriptor passwords and transaction tokens. The external open path also prints a complete query plan at VLOG(1). Replace these dumps with query/fragment identifiers and instance counts without modifying execution parameters.
Release note
Verbose fragment execution logs no longer include full execution parameters, query options or external query plans that may contain credentials.
Evidence
FragmentParamsLogTest.*:FragmentMgrLogTest.*: 5 executed, 5 passed, 0 failed, 0 skipped (Clang21/ASAN). Fresh GTest XML and unchanged source inputs were checked.Check List (For Author)
Merge Danger
Door: two-way
Blast Radius: diagnostics
Execution and protocol parameters are unchanged; only verbose log contents change.
Check List (For Reviewer who merge this PR)