Skip to content

[ISSUE #7597] Externalize the etcd sync and registry integration - #7626

Open
yangjj-iso wants to merge 14 commits into
apache:masterfrom
yangjj-iso:feat/7597-externalize-etcd
Open

yangjj-iso wants to merge 14 commits into
apache:masterfrom
yangjj-iso:feat/7597-externalize-etcd

Conversation

@yangjj-iso

@yangjj-iso yangjj-iso commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Refs #7597. Receiving stack: apache/shenyu-plugin-store#27.

Transfer the complete etcd stack: shared etcd infrastructure, Admin listener, gateway sync, registry and starter. Remove core's implementation/default dependencies and retired jetcd licensing, preserve type names/keyspaces/shared compatibility APIs, and document external dependency selection and rollback. The receiving repository preserves the implementation, tests and legacy deployment examples.

Synchronized with master 92d16c2, including explicit ClassLoader use in external-SPI tests and current license/notice content. Default Admin/Bootstrap runtime inventories contain no io.etcd client after cutover.

Fresh Java 17 validation: the affected reactor packages with main/test compilation and Checkstyle; 14 registry/SPI/classpath tests and 7 dependency-boundary script tests pass. Verify, root RAT and whitespace checks pass. The receiving stack passed 33 tests against the checksum-verified official etcd 3.5.21 native server, including watch recovery and gateway/discovery behavior.

Ready for formal code review. Merge still requires community agreement, the released deprecation cycle, passing full receiving/deployment CI, compatible publication with a single owner for each coordinate and an announced rollback window. Local snapshots/native-server tests do not replace artifact publication or full Docker deployment validation. #7597 remains open.

@yangjj-iso

Copy link
Copy Markdown
Contributor Author

Hi maintainers, I have prepared the paired draft PRs for #7597:

Could you please confirm whether community consensus for externalizing the complete etcd stack has been reached, or point me to the discussion/decision needed before the removal can proceed? It would also help to agree on the compatible plugin-store release, publication-before-cutover sequence, and rollback window so existing etcd deployments have a supported upgrade path without duplicate Maven coordinates.

Local validation passed the 161-module affected build, 29 targeted regressions, and 33 receiving etcd tests against a real etcd 3.5.21 server. Full Testcontainers/Admin-to-gateway deployment execution is still pending because local Docker Desktop could not start. I am keeping both PRs draft until those validation and release gates are satisfied.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: COMMENT (draft — not approving)

Complete in-tree externalization of etcd, consistent with #7597 and the master tracking issue #7306.

What looks correct:

  • All five etcd modules are deleted: shenyu-registry-etcd, shenyu-sync-data-etcd, shenyu-admin-listener-etcd, shenyu-infra-etcd, and shenyu-spring-boot-starter-sync-data-etcd — with parent <module> entries, imports, and application.yml cleaned up.
  • LICENSE/NOTICE (admin + bootstrap dist) drop the jetcd entries; docs/plugin-store-etcd.md documents the external artifact, SPI key, and rollback; the registry README and example/e2e/k8s manifests are updated.
  • check-registry-dependencies.py removes etcd from PROVIDERS (intentionally keeps apollo since apollo is still in-tree — that will be resolved by #7628).
  • CI is green for build and all integrated-test jobs; the e2e failure is the usual compose-startup infra flake, not a code regression.

Blocking-for-merge concern (not for the draft itself):

  • Same sequencing rule as Apollo: this is a deprecation/externalization and must not be merged in the deprecation release. It requires dev-list consensus, a released deprecation cycle, external artifact publication, and an announced rollback window per docs/plugin-store-etcd.md and #7306. Keep the draft gate until the deprecation release has shipped.

Larger blast radius than Apollo — worth a second look:

  • This PR removes etcd from the examples, e2e compose scripts, and k8s manifests, so in-tree CI no longer exercises etcd sync. That is expected for externalization, but please confirm docs/plugin-store-etcd.md fully covers re-adding the external artifact and that no core code unconditionally references the etcd SPI. CI confirms the build is intact, so this is a documentation/operator-readiness check rather than a build risk.

Leaving as a comment because it is a draft; the mechanics are sound.

@yangjj-iso
yangjj-iso marked this pull request as ready for review October 15, 2026 10:42
@yangjj-iso
yangjj-iso marked this pull request as draft October 16, 2026 05:48
@yangjj-iso

Copy link
Copy Markdown
Contributor Author

CI update for head de1f0bc7d: Maven Central download failures recurred after CI was re-triggered:

These failures occur while resolving build-plugin dependencies, before the corresponding integration tests run.

Could a maintainer with workflow rerun permissions please rerun the failed jobs once dependency downloads are healthy? GitHub rejects reruns from my contributor account with Must have admin rights to Repository. Thank you.

# Conflicts:
#	shenyu-admin-listener/shenyu-admin-listener-etcd/pom.xml
#	shenyu-dist/shenyu-admin-dist/src/main/release-docs/LICENSE
#	shenyu-dist/shenyu-admin-dist/src/main/release-docs/NOTICE
#	shenyu-dist/shenyu-bootstrap-dist/src/main/release-docs/LICENSE
#	shenyu-dist/shenyu-bootstrap-dist/src/main/release-docs/NOTICE
#	shenyu-e2e/shenyu-e2e-case/compose/sync/shenyu-sync-etcd-eureka.yml
#	shenyu-e2e/shenyu-e2e-case/compose/sync/shenyu-sync-etcd.yml
#	shenyu-infra/shenyu-infra-etcd/pom.xml
#	shenyu-registry/shenyu-registry-core/src/test/java/org/apache/shenyu/registry/core/ShenyuInstanceRegisterRepositoryFactoryTest.java
#	shenyu-registry/shenyu-registry-etcd/pom.xml
#	shenyu-spring-boot-starter/shenyu-spring-boot-starter-sync-data-center/shenyu-spring-boot-starter-sync-data-etcd/pom.xml
#	shenyu-sync-data-center/shenyu-sync-data-etcd/pom.xml
@yangjj-iso
yangjj-iso marked this pull request as ready for review November 1, 2026 23:22
@codecov-commenter

codecov-commenter commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.59%. Comparing base (1748a0e) to head (825c4df).
⚠️ Report is 1354 commits behind head on master.

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #7626       +/-   ##
=============================================
+ Coverage     61.49%   71.59%   +10.10%     
- Complexity     8818    17987     +9169     
=============================================
  Files          1261     1757      +496     
  Lines         39273    67992    +28719     
  Branches       3725     7964     +4239     
=============================================
+ Hits          24151    48682    +24531     
- Misses        12952    15222     +2270     
- Partials       2170     4088     +1918     
Flag Coverage Δ
admin 74.85% <ø> (?)
client-register-registry 54.06% <ø> (?)
core 65.73% <ø> (?)
dist-extra 61.91% <ø> (?)
gateway-runtime 83.70% <ø> (?)
plugin-ai 76.57% <ø> (?)
plugin-base 79.77% <ø> (?)
plugin-logging 76.65% <ø> (?)
plugin-proxy 77.42% <ø> (?)
starters 74.40% <ø> (?)
starters-agent 93.65% <ø> (?)
sync 64.83% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream Agent analysis

PR #7626 externalizes etcd sync/registry via deletions across modules. FINAL synthesis of 45 partitions and 3 reduction nodes: all PASS, complete_diff true, CI 91 checks passed. No cross-file defects or contract breaks found. Recommend merge readiness.

Evidence

  • CI passed: 91 checks complete and green
  • complete_diff true; 45 partitions all PASS
  • Synthesis reviews reduce-1-0/1-1/1-2 all PASS

Recommended next steps

  • Maintainer final review and merge of PR #7626
  • Confirm external etcd module published separately per issue #7597

Scope: captured diff and GitHub check metadata only; no local tests were executed.

Task: 42df9700-70df-4c80-9a31-76fe1aa1758a · Commit: 825c4df8b270dc84da4738c7f6fcbef7e1412206

Review verdict: PASS

FINAL SYNTHESIS: all_parts_passed true, complete_diff true, CI green (91 checks). Child reviews PASS with no HIGH/CRITICAL findings. Etcd externalization consistent, no cross-file defects.

Publication: APPROVE (EXPLICIT_PASS)

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream Agent analysis

PR #7626 externalizes etcd sync/registry via deletions across modules. FINAL synthesis of 45 partitions and 3 reduction nodes: all PASS, complete_diff true, CI 91 checks passed. No cross-file defects or contract breaks found. Recommend merge readiness.

Evidence

  • CI passed: 91 checks complete and green
  • complete_diff true; 45 partitions all PASS
  • Synthesis reviews reduce-1-0/1-1/1-2 all PASS

Recommended next steps

  • Maintainer final review and merge of PR #7626
  • Confirm external etcd module published separately per issue #7597

Scope: captured diff and GitHub check metadata only; no local tests were executed.

Task: 42df9700-70df-4c80-9a31-76fe1aa1758a · Commit: 825c4df8b270dc84da4738c7f6fcbef7e1412206

Review verdict: PASS

FINAL SYNTHESIS: all_parts_passed true, complete_diff true, CI green (91 checks). Child reviews PASS with no HIGH/CRITICAL findings. Etcd externalization consistent, no cross-file defects.

Publication: COMMENT (MERGEABILITY_NOT_CONFIRMED)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants