Repository navigation
[fix](lakesoul) Avoid logging credentials on branch-4.1 - #69473
Conversation
Replace the complete scanner parameter dump with query ID and batch size, and remove the FE catalog-property dump. Preserve authentication options. Add focused log-capture regressions and an opt-in test profile for the retained LakeSoul scanner without enabling it in default builds.
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
Reviewed all seven changed files and the related FE -> Thrift options -> BE/JNI -> native-reader flow at b75a4de. No actionable new finding was identified. The scanner startup log now selects query ID and batch size, the FE catalog-property dump is removed, and authentication option forwarding is preserved.
The review converged after one round: both complete-scope reviewers and the separate test/runtime risk reviewer returned NO_NEW_VALUABLE_FINDINGS on the unchanged empty finding set. All candidate and risk questions were dismissed with evidence; no unresolved candidate or uncovered changed path remains. There were no existing inline findings to carry forward and no additional user-provided focus.
Critical-checkpoint conclusions:
- Goal and tests: the two credential-bearing map dumps are addressed. The three new scanner cases and one FE case check log suppression and preserved options at source level.
- Scope: the two production logging changes and their test/profile support are focused.
- Concurrency and locking: no new production thread, shared mutation, lock or lock-order dependency. Tests restore logger state in finally.
- Lifecycle and initialization: production native/Arrow ownership and cleanup are unchanged. Mock/appender cleanup and the mocked native-construction boundary were checked statically, including exact-version dependency source.
- Configuration: no runtime setting is added. The lakesoul-scanner-tests Maven profile is explicitly opt-in and preserves the normal module exclusion.
- Compatibility: no production signature, storage format or options serialization change requires rolling-upgrade handling.
- Shared definitions: no Thrift/protobuf definition changes; field-ID comparison is not applicable.
- Parallel paths: both FE and scanner logging sites and related range/predicate/native transport paths were inspected; no additional issue introduced by this patch was found.
- Conditions: production option/read-mode/null gates are unchanged. The scanner test includes missing query_id.
- Coverage: tests inspect INFO/DEBUG logging, unknown future parameters, signed-path tokens in scanner params, credential propagation and unchanged input maps. Native IO is mocked; no live SQL/integration coverage is claimed.
- Expected results: no regression output file changes. The new assertions match the inspected logging and option-mapping behavior; author-reported test passes were not rerun by this review.
- Observability: query ID and batch size retain scanner correlation context. This focused logging fix needs no new metric.
- Transactions and persistence: no commit/visibility, EditLog, replay or failover behavior changes.
- Writes and crash behavior: no data mutation, atomicity, resource-ownership or crash-recovery behavior changes.
- FE/BE variables: wire values and credential propagation are unchanged; query_id was already supplied by the BE runtime.
- Performance: removal of whole-map formatting reduces logging work; no new production loop, allocation pattern or heavy locked operation was introduced.
- Other concerns: logger identity/capture, native mocking, fresh-range population order, test dependency/discovery and default-versus-opt-in packaging were checked without finding a substantiated regression.
Validation limit: this was a static-only review. No builds, tests, native execution or end-to-end SQL scans were run, and repository source files were not modified. Review convergence does not assert runtime validation.
Review converged after 1 round(s).
FE Regression Coverage ReportIncrement line coverage |
### What problem does this PR solve? Issue Number: N/A (no public GitHub issue) Related PR: #69473 Problem Summary: Port the LakeSoul diagnostic logging fix to `branch-4.2`. The retained scanner logs complete parameters at INFO, including object storage credentials carried in `options`. The FE scan node also logs complete catalog properties at DEBUG. Log only the query ID and batch size when opening the scanner, and remove the catalog-property dump. Authentication options continue to reach the reader unchanged. Add the same focused unit regressions and an explicit `lakesoul-scanner-tests` Maven profile. The deprecated scanner remains excluded from default builds, and normal LakeSoul catalog creation remains disabled. ### Release note Prevent sensitive LakeSoul scanner parameters and catalog properties from being written to logs. ### Check List (For Author) - Test - [ ] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: Validation on branch-4.2 with JDK 17 and a matching Thrift 0.24.0 generator: - `LakeSoulJniScannerTest`: 3 tests passed, covering INFO, DEBUG, and a missing query ID; credentials still reach the native reader. - `LakeSoulScanNodeTest`: 1 test passed, verifying credentials stay in scan options and out of DEBUG logs. - All 4 tests fail on credential-leak assertions with the original 4.2 production logging, and pass with this fix. Fresh Surefire XML reports show no skipped tests or errors. - Targeted Checkstyle: 0 violations. `git diff --check` passed. Native I/O is isolated with Mockito. No live LakeSoul SQL scan was run; the branch rejects LakeSoul catalog creation. - Behavior changed: - [ ] No. - [x] Yes. Scanner startup logs contain the query ID and batch size instead of the full parameters; FE no longer logs complete catalog properties. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label
What problem does this PR solve?
Issue Number: N/A (no public GitHub issue)
Problem Summary:
The retained LakeSoul scanner logs complete scanner parameters at INFO, including object storage credentials carried in
options. The FE scan node also logs complete catalog properties at DEBUG.Log only the query ID and batch size when opening the scanner, and remove the catalog-property dump. Authentication options continue to reach the reader unchanged.
Add focused regression tests and an explicit
lakesoul-scanner-testsMaven profile so the deprecated scanner can be tested while remaining excluded from default builds.Release note
Prevent sensitive LakeSoul scanner parameters and catalog properties from being written to logs.
Check List (For Author)
Test
Validation on JDK 17:
LakeSoulJniScannerTest: 3 tests passed, covering INFO, DEBUG, and a missing query ID; credentials still reach the native reader.LakeSoulScanNodeTest: 1 test passed, verifying credentials stay in scan options and out of DEBUG logs.git diff --checkpassed.Native I/O is isolated with Mockito. No live LakeSoul SQL scan was run.
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)