Skip to content

chore: sync main with upstream splice main - #78

Open
sadiq1971 wants to merge 222 commits into
mainfrom
chore/sync-main-upstream-0.10.0
Open

sadiq1971 wants to merge 222 commits into
mainfrom
chore/sync-main-upstream-0.10.0

Conversation

@sadiq1971

Copy link
Copy Markdown
Collaborator

Refs ChainSafe/canton-extending-mainnet#114

Summary: Merges the latest upstream canton-network/splice main (cc621212d, 218 commits since our last sync) into main, keeping the dedicated-synchronizer work intact. Bumps the 19 Daml packages whose versions now collide with upstream one patch past upstream, and moves the RegisteredSynchronizer guard and compat test tag with them.

pawelperek-da and others added 30 commits September 15, 2026 12:09
Signed-off-by: Paweł Perek <pawel.perek@digitalasset.com>
Signed-off-by: Paweł Perek <pawel.perek@digitalasset.com>
Signed-off-by: Paweł Perek <pawel.perek@digitalasset.com>
…#7306)

Fixes canton-network#7027

See DACH-NY/canton#35367

[ci]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
Probably needs a bit of tweaking but easiest to just deploy to cilr
and see how it goes.

[static]

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Signed-off-by: Paweł Perek <pawel.perek@digitalasset.com>
… subtrees (canton-network#7297)

* make rsync exclusions actually exclude

* remove unused subtrees

- canton/base/
  - executors
  - grpc-utils
  - ledger-resources
  - logging-entries
  - nameof
  - nonempty
  - nonempty-cats
  - ports
  - resources
  - resources-grpc
  - resources-pekko
  - rs-grpc-bridge
  - rs-grpc-pekko
  - scala-utils
  - scalatest-utils
  - test-evidence
  - testing-utils
  - timer-utils
  - observability/tracing
- canton/community/
  - aws-kms-driver
  - conformance-testing
  - daml-script-tests
  - daml-lf/
    - api-type-signature
    - archive
    - data
    - data-bench
    - data-tests
    - encoder
    - engine
    - ide-ledger
    - interpreter
    - language
    - ledger-api-value
    - ledger-api-value-proto
    - parser
    - repl
    - snapshot
    - snapshot-proto
    - spec
    - stable-packages
    - tests
    - transaction
    - transaction-tests
    - upgrades-matrix
    - upgrades-matrix-integration
    - validation
  - docs
  - kms-driver-testing
  - ledger-test-tool
  - microbench
  - mock-kms-driver
  - performance-driver
  - sequencer-driver-api-conformance-tests
  - upgrading-integration-tests
  - ledger/
    - ledger-api-string-interning-benchmark
    - ledger-api-tools
    - ledger-json-client
  - ledger-api-scala
  - model-based-testing-drivers
  - model-based-testing-generators
  - model-based-testing-integration-tests
  - traffic-enforcement/api
  - transcode
- canton/protobuf-continuity-check
- canton/release-notes

---------

Assisted-by: Copilot:claude-4.8-opus
Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
Not tested but matches the docs at https://istio.io/latest/docs/setup/install/helm/.

[static]

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
[static]

Currently fails with

```
[alerting.alert-rule.conditionNotExist] Condition acs commitment
component failed does not exist, must be one of [acs commitment
component health acs commitment component is not in state started]
```

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
…y time advancing too far (canton-network#7261)

Signed-off-by: JYC11 <jaeyoon.c@gmail.com>
[static]

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
…k#7327)

I am stupid the condition succeeding is when we alert so it was the
wrong way around.

[static]

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
…redAndConnected` (canton-network#7311)

Fixes DACH-NY/cn-test-failures#10088 , which is a rare race condition between validator app init and an actual LSU.

Not super happy with this fix; my best alternative suggestion would be to crash the validator app if that happens (which will make more sense on clusters) and tweak the `LsuIntegrationTest` so this race is more unlikely there.

The race seems too rare to motivate a bigger intervention where we bubble up the error to retry the affected init step with a different `psid`...

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
* whitelist just the cluster hostname and also include the port (canton-network#7325)

* whitelist just the cluster hostname and also include the port

[static]

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* [static]

Signed-off-by: DA Automation <splice-maintainers@digitalasset.com>

---------

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>
Signed-off-by: DA Automation <splice-maintainers@digitalasset.com>
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
Co-authored-by: DA Automation <splice-maintainers@digitalasset.com>
[static]

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>
[static]

Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>
canton-network#7303)

* [ci] SummarizingMiningRoundTrigger: no-op when own Scan totals not yet computed

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Also handling bftReadTotals failure as not yet computed

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

---------

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
* Moderately increase pending time for `High Rate of Discarded Confirmation Requests` alert (canton-network#7331)

Gives the CometBFT watchdog a slightly higher chance of intervening before we get bothered.

[static]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* update expected

Signed-off-by: DA Automation <splice-maintainers@digitalasset.com>

* [static]

Signed-off-by: DA Automation <splice-maintainers@digitalasset.com>

---------

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
Signed-off-by: DA Automation <splice-maintainers@digitalasset.com>
Co-authored-by: Martin Florian <martin.florian@digitalasset.com>
Co-authored-by: DA Automation <splice-maintainers@digitalasset.com>
See DACH-NY/canton-network-internal#7103

This will need careful bumping PRs on the internal side...

[static]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
[ci]

Needed that while debugging some issue.

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
…-network#7321)

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
…e cn-test-failures#9445 (canton-network#7334)

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
…rk#7348)

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
…rk#7352)

[static]

The short version is that YAML is great.

Slightly longer: this is not essential but it certainly makes things nicer.

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
* move custom logback-test.xml out of util-observability, to new fork lib

* don't build test sources in any canton submodules

* move 3 files from canton-community-common test code to fork library

* remove unused observability-metrics-testing

* remove unused test->test deps

* header config for forked module

* fix pekko-stream-testkit dependencies at test runtime

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
…ignAndMint (canton-network#7302)

Signed-off-by: JYC11 <jaeyoon.c@gmail.com>
…nton-network#7360)

[static]

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
OriolMunoz-da and others added 24 commits October 9, 2026 09:55
---------

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
…e update history sanity check (canton-network#7693)

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
Co-authored-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
…ose (canton-network#7699)

Fixes DACH-NY/cn-test-failures#10319
[ci]

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
[ci]
already ignored in LSU tests

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
canton-network#7441)

* [ci] Let ResetDecentralizedNamespace tolerate an owner whose proposal is no longer needed once the other owners have authorized the reset

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] ResetDecentralizedNamespace: skip an owner's proposal only when its errors are the known rejections of an unneeded proposal, otherwise retry the reset

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

---------

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
…#7686)

* [ci] Scan: debug flag to forget the recorded history start on startup, so it is determined again from the current data

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Scan history start store: reset takes parentheses, since it has a side effect

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

---------

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
[static]

Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>
…servers booting (canton-network#7710)

---------

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
…one segment at a time (canton-network#7657)

* [ci] Bulk storage backfilling lists one segment at a time through the two-phase BFT call and copies only from the peers that hold it

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage backfilling logs an error and waits when no peer will ever hold the objects it needs

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage backfilling lists its peers' objects in every encoding, so it copies protobuf_json as well as compact_json

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage listing: no peer will hold the objects only when every encoding says so, any other mix waits

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage listing: each merge answers the peer listing itself, and a snapshot is available when every encoding has the same record time or none

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage listing: a peer listing's available result is its value, since it is not always a list of objects

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage backfilling copies each encoding only from the peers that agreed on that encoding's listing

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Scan connection: name the bulk listing call after what it does, a BFT call among the Scans that committed up to the given time

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage listing: objects and snapshots on peers, named for the peer Scans that agreed on them

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Backfilling test listing: name the page of whole folders that fit the page size

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage backfilling: the Scans a listing comes from are its peers everywhere, and the snapshot check names its record times

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Two-phase BFT call: a scan that cannot be reached counts as not yet, so no scan will have the data only when every configured scan says never

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk listing BFT call keeps the log levels the listing used before: consensus failures and too few scans at DEBUG, disagreements at INFO

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage backfilling: updateObjectsPage and nextSegment names, encodings listed and copied one after the other, a NonEmptyList per encoding, a longer retry when no peer will hold the data, and a warning when the encodings list different snapshots

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

---------

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
…anton-network#7658)

* [ci] New-network integration test: an SV joining inside the founder's first segment copies the segments before its own from the founder

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] New-network integration test: name the objects backfilling copies, update segments ending by the first own segment and snapshots up to it

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] New-network integration test: the founder, starting at genesis, completes its backfill without copying

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] New-network integration test: find an object's segment folder with the ScanStorageConfig helper

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] New-network integration test: compare sv2's staging with sv1's committed objects re-read in the final check, since sv1 commits a segment one object at a time

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

---------

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
… start, copy updates, copy snapshots, finish (canton-network#7709)

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
bump istio to 1.30.5

---------

Signed-off-by: krzysztofczyz-da <krzysztof.czyz@digitalasset.com>
…rk#7711)

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
Before the snapshot gets pruned

[ci]

Signed-off-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
Co-authored-by: moritz.kiefer@digitalasset.com <moritz.kiefer@purelyfunctional.org>
…sedIntegrationTest (canton-network#7722)

Signed-off-by: Paweł Perek <pawel.perek@digitalasset.com>
…pty snapshot (canton-network#7708)

* [ci] Bulk storage backfilling persists the snapshot cursor when it skips an empty snapshot

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Bulk storage backfilling tests: an empty update segment, a snapshot the peers only have older, and a listing the peers disagree on

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Copier test: a peer serving zero bytes fails the checksum and the next peer is used

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

* [ci] Scan connection tests: bulk listing asks the committed progress, lists from the founder alone when the others backfill, and reports never when all do

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

---------

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>
[static]

Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>
[ci]

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
…anton-network#7729)

Add maxItems: 1000 to validator_ids and reject larger requests with 400
in /v0/validators/validator-faucets; deduplicate the IDs before the IN
clause and defensively limit by defaultLimit (1000).

Assisted-by: Claude:claude-5.5-opus
Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
…7730)

Cap the views read for one verdict at 100000 with a HardLimit, failing
the read if it is exceeded rather than returning incomplete views.
10 MiB limits (from some messy byte math) allow at most about that many
views per verdict, so the cap is not expected to be hit.

Assisted-by: Claude:claude-5.5-opus
Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
Merges upstream main at cc62121 (0.10.0 dev, latest release 0.9.1, Canton
3.6.1) into main.

Conflict resolution:
- LSU: keep our LsuTransferTriggerBase split and apply upstream's Canton
  API change there (tryAsPhysical on the successor synchronizer id).
- Sync operator: take upstream's com.digitalasset.nonempty rename, and
  give its mediator the subscription liveness limits upstream added to
  the SV mediator, configurable the same way.
- Validator: keep our synchronizer config import next to upstream's
  party-to-participant store import.
- Tests: keep both new ScanStoreTest groups.
- Renumber our Flyway migrations to V078-V080 behind upstream's V077.
- Generated Daml artifacts and test package versions follow upstream;
  the next commit bumps our packages past the 0.9.1 release.

Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>
Upstream released the versions our Daml changes were carried at, with
different content. Bumps every affected package one patch above upstream
via damlBumpPackageVersions upstream/main, keeps upstream's checked-in
DARs at the released versions, and regenerates dars.lock, DarResources
and the npm lock.

RegisteredSynchronizer now first ships in splice-amulet 0.1.25 and
splice-dso-governance 0.1.31, so the dedicated synchronizer guard and the
SpliceDsoGovernance compatibility test tag move to those versions.

Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>
@isegall-da

Copy link
Copy Markdown

Thanks @sadiq1971 I think you also need to pull in the latest release-line to make CI happy

@sadiq1971 sadiq1971 self-assigned this Oct 14, 2026
Moves the new SyncOperatorBftIntegrationTest to the SpliceDsoGovernance_0_1_31 tag.

Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>

@moritzkiefer-da moritzkiefer-da left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.