Switch to relying on package names instead of module names #1976

OriolMunoz-da · 2025-08-20T13:19:26Z

Fixes #829

Pull Request Checklist

Cluster Testing

If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

Include any change that might be observable by our partners or affect their deployment in the release notes.
Specify fixed issues with Fixes #n, and mention issues worked on using #n
Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da · 2025-08-21T14:46:49Z

/cluster_test

github-actions · 2025-08-21T14:47:15Z

Deploy cluster test triggered for Commit 3b845ce5b206cfe9ff5ed5af7ef95ee9acbc514e in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/28163

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da · 2025-08-25T15:09:37Z

apps/common/src/test/scala/org/lfdecentralizedtrust/splice/store/MultiDomainAcsStoreTest.scala

+      }
+    }
+
+    "prevent against ingestion of same (moduleName, entityName) with different package name - via interface fallback" in {


this test is the best summary for the reason of being of this PR

OriolMunoz-da · 2025-08-25T15:27:54Z

...main/resources/db/migration/canton-network/postgres/stable/V043__package_name_acs_stores.sql

+-- strictly not necessary, but lets us define the constraint as mandatory afterwards,
+-- and also makes index creation faster since there's no data to index
+truncate table acs_store_template cascade;
+truncate table user_wallet_acs_store cascade;
+truncate table external_party_wallet_acs_store cascade;
+truncate table validator_acs_store cascade;
+truncate table sv_acs_store cascade;
+truncate table dso_acs_store cascade;
+truncate table scan_acs_store cascade;
+truncate table splitwell_acs_store cascade;


double-check me that this is not a bad idea

note that all descriptors are bumped

OriolMunoz-da · 2025-08-25T15:28:26Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/MultiDomainAcsStore.scala

-    * another list; that is why the contract ID by itself cannot be used by
-    * itself as the source of the sort key.
-    */
-  def listAssignedContractsNotOnDomainN(


unused except in a test (also deleted)

OriolMunoz-da · 2025-08-25T15:29:13Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/AcsQueries.scala

      storeId: AcsStoreId,
      migrationId: Long,
-      where: SQLActionBuilder,
+      companion: C,


this should've been the case all along, instead of having where ti_q_n = ? on every query

agreed, thanks for pulling through that simplification!

OriolMunoz-da · 2025-08-25T15:29:53Z

apps/common/src/test/scala/org/lfdecentralizedtrust/splice/store/MultiDomainAcsStoreTest.scala

      MultiDomainAcsStoreTest.generatedCoids.value.size should (be >= 900 and be <= 1000)
    }

-    "read assignment-mismatched contracts in a stable order" in {


tested listAssignedContractsNotOnDomainN, deleted

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da · 2025-08-25T15:31:39Z

/cluster_test

github-actions · 2025-08-25T15:32:13Z

Deploy cluster test triggered for Commit 1680a64b4cb0ace131dd19435f78cb0d69064cea in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/28663

moritzkiefer-da

Very nice, thanks a lot!

Please run a performance test before merging so we can avoid surprises for long downtimes due to ACS backfilling.

moritzkiefer-da · 2025-08-26T06:06:38Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/AcsQueries.scala

      storeId: AcsStoreId,
      migrationId: Long,
-      where: SQLActionBuilder,
+      companion: C,


agreed, thanks for pulling through that simplification!

moritzkiefer-da · 2025-08-26T06:13:10Z

...main/resources/db/migration/canton-network/postgres/stable/V043__package_name_acs_stores.sql

@@ -0,0 +1,362 @@
+-- strictly not necessary, but lets us define the constraint as mandatory afterwards,


Do we know what performance we get for a full ACS backfill? I'm a bit nervous about this given that the ACS queries are known to not perform well without pruning and there is no pruning atm on SV participants.

How about on CILR you take one SV (maybe best to use the SV runbook given that it resets daily anyway so if it goes wrong we can just reset again) manually force a reingestion by dropping the data from the table and last ingested offset?

If it does take too long, the easy short term option we have is to merge this into the 3.4 branch only. Afaict, there isn't an immediate need to have this be on 3.3.

How about on CILR you take one SV (maybe best to use the SV runbook given that it resets daily anyway so if it goes wrong we can just reset again) manually force a reingestion by dropping the data from the table and last ingested offset?

What is an "ACS backfill"? After truncating the ACS tables and bumping the store descriptor, the new ACS store will be initialized from the ACS endpoint of the ledger API, is that what you meant? Apart from the participant being slow to deliver the ACS, we are not batching SQL inserts when consuming the ACS

override def ingestAcs() = { ... // TODO (#989): batch inserts }

and we load the whole ACS into one in-memory structure:

private def ingestAcsAndInFlight(offset: Long) = { ... // TODO(#863): stream contracts instead of ingesting them as a single Seq

so I agree that one test on CILR is a good idea.

is that what you meant?

yes

@OriolMunoz-da So given the very rapid increase in ACS size on mainnet I'm rather nervous about merging this atm. Can we hold off on this?

Or merge it into 3.4? There we will ingest the acs anyway so it's not making anything worse

...allet/src/main/scala/org/lfdecentralizedtrust/splice/wallet/store/db/DbUserWalletStore.scala

Signed-off-by: Oriol Muñoz <[email protected]>

...main/resources/db/migration/canton-network/postgres/stable/V047__package_name_acs_stores.sql

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/AcsQueries.scala

OriolMunoz-da · 2025-08-27T13:56:07Z

timings for ACS reingestion:

a bit more than 2m for scan, logs:

and SV app doesn't reingest, see #2041

Signed-off-by: Oriol Muñoz <[email protected]>

[ci]

rautenrieth-da

Thanks!

…lback [ci] Signed-off-by: Oriol Muñoz <[email protected]>

moritzkiefer-da

Nice work thank you!

...main/resources/db/migration/canton-network/postgres/stable/V047__package_name_acs_stores.sql

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/util/QualifiedName.scala

OriolMunoz-da added 9 commits August 20, 2025 12:19

get a failing test

ff086b9

Signed-off-by: Oriol Muñoz <[email protected]>

Break when the results would be bad [ci]

ea2f40a

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] fix things

ee72588

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] more fixes

cae3fad

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] was that a gha/k8s failure or did i cause that? let's find out

4199050

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] refactor so many queries something is bound to break

0c01122

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] remove many instances of getFromResources

096ea57

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] fix splitwell

24b712a

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] fix bad query

3b845ce

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da added 6 commits August 21, 2025 15:40

unused variable

50077da

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] bump descriptors

15d666a

Signed-off-by: Oriol Muñoz <[email protected]>

Merge branch 'main' into oriol/packagename-stores-interface-fallback

a76111d

fix many compile errors

59c5890

Signed-off-by: Oriol Muñoz <[email protected]>

add indexes with todo

3184d5a

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] change definition of all indexes involving tid

5910a9f

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da changed the title ~~Oriol/packagename stores interface fallback~~ Switch to relying on package names instead of module names Aug 25, 2025

OriolMunoz-da commented Aug 25, 2025

View reviewed changes

[ci] also truncate store_last_ingested_offsets

1680a64

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da marked this pull request as ready for review August 25, 2025 15:31

OriolMunoz-da requested review from moritzkiefer-da and rautenrieth-da August 25, 2025 15:31

moritzkiefer-da approved these changes Aug 26, 2025

View reviewed changes

OriolMunoz-da commented Aug 26, 2025

View reviewed changes

...allet/src/main/scala/org/lfdecentralizedtrust/splice/wallet/store/db/DbUserWalletStore.scala Outdated Show resolved Hide resolved

OriolMunoz-da added 2 commits August 26, 2025 15:01

[ci] revert bump of txlog descriptor

5bfdbca

Signed-off-by: Oriol Muñoz <[email protected]>

wip

cb9983f

Signed-off-by: Oriol Muñoz <[email protected]>

rautenrieth-da reviewed Aug 27, 2025

View reviewed changes

...main/resources/db/migration/canton-network/postgres/stable/V047__package_name_acs_stores.sql Show resolved Hide resolved

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/AcsQueries.scala Outdated Show resolved Hide resolved

OriolMunoz-da mentioned this pull request Aug 27, 2025

Handle bumping acs store descriptors to already existing versions #2033

Open

OriolMunoz-da added 3 commits September 5, 2025 11:34

fix comment

16df3a3

Signed-off-by: Oriol Muñoz <[email protected]>

Merge branch 'main' into oriol/packagename-stores-interface-fallback

5ca46b9

[ci] update migration

553b6c9

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da requested a review from pawelperek-da September 5, 2025 14:33

OriolMunoz-da added 7 commits October 1, 2025 12:11

Merge branch 'main' into oriol/packagename-stores-interface-fallback

583b0cb

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] run

df27b53

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] fix migrations

684be68

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] Ignore ACS_COMMITMENT_DEGRADATION

498948c

Signed-off-by: Oriol Muñoz <[email protected]>

Merge branch 'main' into oriol/packagename-stores-interface-fallback

b82028d

[ci] run

d201c0f

Signed-off-by: Oriol Muñoz <[email protected]>

[ci] fix migrations again

6f8f951

Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da requested a review from rautenrieth-da October 1, 2025 13:42

Merge branch 'main' into oriol/packagename-stores-interface-fallback

74fea0e

[ci]

OriolMunoz-da removed the request for review from pawelperek-da October 1, 2025 17:12

rautenrieth-da approved these changes Oct 1, 2025

View reviewed changes

OriolMunoz-da changed the base branch from main to canton-3.4 October 9, 2025 11:29

Merge branch 'canton-3.4' into oriol/packagename-stores-interface-fal…

2a04dac

…lback [ci] Signed-off-by: Oriol Muñoz <[email protected]>

OriolMunoz-da self-assigned this Oct 9, 2025

OriolMunoz-da added the sprint2540 label Oct 9, 2025

moritzkiefer-da approved these changes Oct 10, 2025

View reviewed changes

...main/resources/db/migration/canton-network/postgres/stable/V047__package_name_acs_stores.sql Show resolved Hide resolved

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/util/QualifiedName.scala Show resolved Hide resolved

OriolMunoz-da mentioned this pull request Oct 10, 2025

Limit visibility of QualifiedName #2648

Open

OriolMunoz-da merged commit 6447ce5 into canton-3.4 Oct 10, 2025
56 checks passed

OriolMunoz-da deleted the oriol/packagename-stores-interface-fallback branch October 10, 2025 09:54

OriolMunoz-da mentioned this pull request Nov 12, 2025

Switch to relying on package names instead of module names #829

Closed

		@@ -0,0 +1,362 @@
		-- strictly not necessary, but lets us define the constraint as mandatory afterwards,

Switch to relying on package names instead of module names #1976

Switch to relying on package names instead of module names #1976

Uh oh!

Conversation

OriolMunoz-da commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist

Cluster Testing

PR Guidelines

Merge Guidelines

Uh oh!

OriolMunoz-da commented Aug 21, 2025

Uh oh!

github-actions bot commented Aug 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OriolMunoz-da commented Aug 25, 2025

Uh oh!

github-actions bot commented Aug 25, 2025

Uh oh!

moritzkiefer-da left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

OriolMunoz-da commented Aug 27, 2025

Uh oh!

rautenrieth-da left a comment

Choose a reason for hiding this comment

Uh oh!

moritzkiefer-da left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

OriolMunoz-da commented Aug 20, 2025 •

edited

Loading