Skip to content

[fix](lakesoul) Avoid logging credentials on branch-4.2 - #69535

Open
OIiveirra wants to merge 1 commit into
apache:branch-4.2from
OIiveirra:fix/jira-sdbsec-273-4.2
Open

OIiveirra wants to merge 1 commit into
apache:branch-4.2from
OIiveirra:fix/jira-sdbsec-273-4.2

Conversation

@OIiveirra

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: N/A (no public GitHub issue)

Related PR: #69473

Problem Summary:

Port the LakeSoul diagnostic logging fix to branch-4.2. The retained scanner logs complete parameters at INFO, including object storage credentials carried in options. The FE scan node also logs complete catalog properties at DEBUG.

Log only the query ID and batch size when opening the scanner, and remove the catalog-property dump. Authentication options continue to reach the reader unchanged.

Add the same focused unit regressions and an explicit lakesoul-scanner-tests Maven profile. The deprecated scanner remains excluded from default builds, and normal LakeSoul catalog creation remains disabled.

Release note

Prevent sensitive LakeSoul scanner parameters and catalog properties from being written to logs.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:

    Validation on branch-4.2 with JDK 17 and a matching Thrift 0.24.0 generator:

    • LakeSoulJniScannerTest: 3 tests passed, covering INFO, DEBUG, and a missing query ID; credentials still reach the native reader.
    • LakeSoulScanNodeTest: 1 test passed, verifying credentials stay in scan options and out of DEBUG logs.
    • All 4 tests fail on credential-leak assertions with the original 4.2 production logging, and pass with this fix. Fresh Surefire XML reports show no skipped tests or errors.
    • Targeted Checkstyle: 0 violations. git diff --check passed.

    Native I/O is isolated with Mockito. No live LakeSoul SQL scan was run; the branch rejects LakeSoul catalog creation.

  • Behavior changed:

    • No.
    • Yes. Scanner startup logs contain the query ID and batch size instead of the full parameters; FE no longer logs complete catalog properties.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Port the diagnostic logging fix from apache#69473 to branch-4.2. Replace the
complete scanner parameter dump with query ID and batch size, and remove
the FE catalog-property dump. Preserve authentication options.

Include log-capture regressions and an opt-in test profile for the retained
scanner without enabling LakeSoul in default builds.
@OIiveirra
OIiveirra requested a review from yiguolei as a code owner October 23, 2026 10:55
Copilot AI balanced review requested due to automatic review settings October 23, 2026 10:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@OIiveirra

Copy link
Copy Markdown
Collaborator Author

run buildall

@OIiveirra

Copy link
Copy Markdown
Collaborator Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new actionable findings were verified across all seven changed files at head 541eb9f. The logging changes remove the complete scanner-parameter and catalog-property dumps while preserving the existing authentication path. There were no existing inline findings to revalidate and no additional user-provided focus points.

Critical checkpoint conclusions:

  • Goal and regression intent: startup logs retain query ID/batch size; FE drops the catalog dump. The three scanner cases and one FE case assert credential exclusion and unchanged option forwarding.
  • Scope: the two production logging edits, test dependency, opt-in profile, and focused test/config files match this backport's purpose.
  • Concurrency and locking: no production thread, shared state, lock, critical section, or lock-order change; test mocking scopes close and logger level/additivity are restored in finally.
  • Lifecycle and initialization: reader/Arrow ownership is unchanged. Constructor mocking intercepts native setup; matching LakeSoul 2.6.2 sources place native allocation in the base constructor, and initializeReader declares IOException. No new static initialization dependency.
  • Configuration: no runtime configuration added. lakesoul-scanner-tests has no automatic activation; the scanner remains excluded from default modules.
  • Compatibility: no function, reader, storage-format, or rolling-upgrade protocol change.
  • Thrift/protobuf: no definitions or field IDs changed; cross-release ID checks are inapplicable.
  • Parallel paths: both identified FE/scanner logging sites are handled; normal LakeSoul catalog creation remains disabled.
  • Conditions and callers: no production branch added. Missing query_id logs null safely. The remaining FE range log precedes LakeSoul option attachment on a freshly allocated range, and BE supplies query_id from runtime query state.
  • Test coverage: INFO, DEBUG, missing query ID, fake credentials, a signed file URL, a future auth parameter, input immutability, and credential forwarding are checked. Predicates and native I/O are deliberately mocked; no live SQL/native scan is covered. Existing Vintage support runs the branch's JUnit 4 style.
  • Expected results: no golden output files changed; assertion values match the existing mappings. No executed test result is claimed.
  • Observability: query/batch diagnostics remain, with credential-bearing containers removed; this log-only fix needs no new metrics.
  • Persistence and transactions: no journal, transaction, visibility-version, or delete-bitmap changes.
  • Data writes and crash behavior: no data-write or atomicity change; production cleanup and error propagation are untouched.
  • FE-BE propagation: no new variable; catalog options still serialize into the scan descriptor, pass through JNI parameters, and reach the supported native object-store setters unchanged.
  • Performance and memory: removes container formatting; the replacement uses a constant-time lookup and scalar diagnostics, with no new significant allocation or repeated scan.
  • Other concerns: FE/datasource guide checks, managed Mockito/test logging dependencies, and module selection revealed no additional issue. XML parsing and git diff --check passed.

Static review only: no builds, tests, submodule initialization, or live LakeSoul scan were performed. Runtime behavior and the author's reported test results were not independently reproduced. No unresolved code candidates or uncovered changed paths remain.

The native-mocking assumption was checked against matching upstream NativeIOReader and NativeIOBase sources. No native-loading static initializer was found.

Review converged after 2 round(s).

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100% (0/0) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants