Repository navigation
Conversation
Port the diagnostic logging fix from apache#69473 to branch-4.2. Replace the complete scanner parameter dump with query ID and batch size, and remove the FE catalog-property dump. Preserve authentication options. Include log-capture regressions and an opt-in test profile for the retained scanner without enabling LakeSoul in default builds.
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
No new actionable findings were verified across all seven changed files at head 541eb9f. The logging changes remove the complete scanner-parameter and catalog-property dumps while preserving the existing authentication path. There were no existing inline findings to revalidate and no additional user-provided focus points.
Critical checkpoint conclusions:
- Goal and regression intent: startup logs retain query ID/batch size; FE drops the catalog dump. The three scanner cases and one FE case assert credential exclusion and unchanged option forwarding.
- Scope: the two production logging edits, test dependency, opt-in profile, and focused test/config files match this backport's purpose.
- Concurrency and locking: no production thread, shared state, lock, critical section, or lock-order change; test mocking scopes close and logger level/additivity are restored in finally.
- Lifecycle and initialization: reader/Arrow ownership is unchanged. Constructor mocking intercepts native setup; matching LakeSoul 2.6.2 sources place native allocation in the base constructor, and initializeReader declares IOException. No new static initialization dependency.
- Configuration: no runtime configuration added. lakesoul-scanner-tests has no automatic activation; the scanner remains excluded from default modules.
- Compatibility: no function, reader, storage-format, or rolling-upgrade protocol change.
- Thrift/protobuf: no definitions or field IDs changed; cross-release ID checks are inapplicable.
- Parallel paths: both identified FE/scanner logging sites are handled; normal LakeSoul catalog creation remains disabled.
- Conditions and callers: no production branch added. Missing query_id logs null safely. The remaining FE range log precedes LakeSoul option attachment on a freshly allocated range, and BE supplies query_id from runtime query state.
- Test coverage: INFO, DEBUG, missing query ID, fake credentials, a signed file URL, a future auth parameter, input immutability, and credential forwarding are checked. Predicates and native I/O are deliberately mocked; no live SQL/native scan is covered. Existing Vintage support runs the branch's JUnit 4 style.
- Expected results: no golden output files changed; assertion values match the existing mappings. No executed test result is claimed.
- Observability: query/batch diagnostics remain, with credential-bearing containers removed; this log-only fix needs no new metrics.
- Persistence and transactions: no journal, transaction, visibility-version, or delete-bitmap changes.
- Data writes and crash behavior: no data-write or atomicity change; production cleanup and error propagation are untouched.
- FE-BE propagation: no new variable; catalog options still serialize into the scan descriptor, pass through JNI parameters, and reach the supported native object-store setters unchanged.
- Performance and memory: removes container formatting; the replacement uses a constant-time lookup and scalar diagnostics, with no new significant allocation or repeated scan.
- Other concerns: FE/datasource guide checks, managed Mockito/test logging dependencies, and module selection revealed no additional issue. XML parsing and git diff --check passed.
Static review only: no builds, tests, submodule initialization, or live LakeSoul scan were performed. Runtime behavior and the author's reported test results were not independently reproduced. No unresolved code candidates or uncovered changed paths remain.
The native-mocking assumption was checked against matching upstream NativeIOReader and NativeIOBase sources. No native-loading static initializer was found.
Review converged after 2 round(s).
FE UT Coverage ReportIncrement line coverage `` 🎉 |
FE Regression Coverage ReportIncrement line coverage |
What problem does this PR solve?
Issue Number: N/A (no public GitHub issue)
Related PR: #69473
Problem Summary:
Port the LakeSoul diagnostic logging fix to
branch-4.2. The retained scanner logs complete parameters at INFO, including object storage credentials carried inoptions. The FE scan node also logs complete catalog properties at DEBUG.Log only the query ID and batch size when opening the scanner, and remove the catalog-property dump. Authentication options continue to reach the reader unchanged.
Add the same focused unit regressions and an explicit
lakesoul-scanner-testsMaven profile. The deprecated scanner remains excluded from default builds, and normal LakeSoul catalog creation remains disabled.Release note
Prevent sensitive LakeSoul scanner parameters and catalog properties from being written to logs.
Check List (For Author)
Test
Validation on branch-4.2 with JDK 17 and a matching Thrift 0.24.0 generator:
LakeSoulJniScannerTest: 3 tests passed, covering INFO, DEBUG, and a missing query ID; credentials still reach the native reader.LakeSoulScanNodeTest: 1 test passed, verifying credentials stay in scan options and out of DEBUG logs.git diff --checkpassed.Native I/O is isolated with Mockito. No live LakeSoul SQL scan was run; the branch rejects LakeSoul catalog creation.
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)