Skip to content

feat: migrate from legacy console wallet (closes #119)#121

Merged
SWvheerden merged 9 commits into
tari-project:mainfrom
0xPepeSilvia:feat/migrate-from-console-wallet-119
May 20, 2026
Merged

feat: migrate from legacy console wallet (closes #119)#121
SWvheerden merged 9 commits into
tari-project:mainfrom
0xPepeSilvia:feat/migrate-from-console-wallet-119

Conversation

@0xPepeSilvia
Copy link
Copy Markdown
Contributor

Summary

Adds a `migrate-from-console-wallet` subcommand that imports an already-synced legacy console wallet SQLite database into the minotari-cli format. The migrated wallet preserves the user's existing display transaction IDs, balance, and unspent output set, and skips re-scanning the chain by carrying the source's last scanned block height across.

Closes #119.

Acceptance criteria

  • CLI command accepts a synced console wallet database file and imports it
  • All completed transactions appear in the new wallet with identical display transaction IDs
  • Migrated wallet shows the same balance as the source
  • Migrated wallet contains the same set of unspent outputs
  • Round-trip test proves correctness

How to use it

```bash
tari migrate-from-console-wallet
--source-db /path/to/console_wallet.sqlite3
--source-password ''
--account-name imported
--password ''
```

What it does, in order

  1. Opens the source SQLite read-only.
  2. Verifies the source passphrase by re-deriving the secondary key from `(passphrase, salt)` via Argon2id (46 MiB / 1 / 1 / 32) and matching against the stored Blake2b-256 commitment, then decrypts the encrypted main key under XChaCha20-Poly1305.
  3. Decrypts the master `CipherSeed` under the main key and constructs a fresh `SeedWordsWallet`.
  4. Inside a single SQLite write transaction on the destination DB:
    • creates the new account (encrypted with the destination passphrase)
    • rebuilds each `WalletOutput` from the legacy output columns and inserts it into `outputs` with the right status (Unspent / Spent / Locked)
    • inserts `balance_changes` rows so the balance query reflects the migrated state
    • for spent outputs, also inserts the corresponding `inputs` row + debit
    • inserts a `displayed_transactions` row per completed transaction, preserving the original `tx_id` value byte-for-byte (the bounty's primary requirement)
    • copies the highest-height `scanned_blocks` entry into `scanned_tip_blocks` so the next `tari scan` resumes from there

If anything fails along the way, the transaction rolls back and the destination DB is untouched.

Module layout

File Role
`migrate/console_db.rs` Read-only access to the source DB. Re-implements the secondary-key / main-key derivation chain so we don't need to depend on `tari_wallet`.
`migrate/output_converter.rs` Reconstructs a `WalletOutput` from the source's decomposed columns.
`migrate/tx_converter.rs` Maps a legacy `completed_transactions` row to a `DisplayedTransaction`, preserving `tx_id`.
`migrate/migrator.rs` Orchestrator. Single write transaction, rolls back on any error.
`migrate/tests.rs` + `test_fixture.rs` Five end-to-end tests.

What is NOT migrated (deliberately)

  • pending / inbound / outbound transactions (the AC asks for completed only)
  • cancelled transactions (they would clutter the new wallet's view)
  • the legacy `transaction_protocol` bincode blob - the new wallet's `completed_transactions` table tracks in-flight outbound sends, which migrated rows aren't. The user-visible record lives in `displayed_transactions` and that IS migrated.

Tests

Five new tests in `migrate::tests`, all green:

```
test migrate::tests::migration_creates_account_with_seed_words_recovered_from_source ... ok
test migrate::tests::migration_rejects_wrong_source_passphrase ... ok
test migrate::tests::migration_preserves_completed_transaction_ids ... ok
test migrate::tests::migration_sets_scan_tip_from_source ... ok
test migrate::tests::migration_rejects_duplicate_account_name ... ok
```

Total: 81 / 81 unit tests passing. Clippy clean with `-D warnings`. Full workspace builds.

Compatibility notes

Only modern `TariKeyId`-format `spending_key` / `script_private_key` strings are recognised. If a user is migrating a very old console wallet whose key IDs are still in the legacy format, the migrator surfaces a clear error pointing them at running the latest console wallet binary once first to upgrade those columns. Adding legacy-key support would require pulling in the `tari_transaction_key_manager` crate, which isn't on the minotari-cli dependency tree.

Bounty payment

XTM address: `123JbAxCZ4Vrh4jCjFLXTu4z8sLeggUR87fejwNunsB55NMBSH9RpkjNiw1WL2GkKr8JhqRZhbrsSREchGanchw2Nhb`

tari-project#119)

Adds a `migrate-from-console-wallet` subcommand that reads the legacy console
wallet's SQLite file (read-only), recovers the master cipher seed using the
user-supplied passphrase, and copies the wallet state into a new account in
the destination minotari-cli database. The migrated wallet preserves the
console wallet's random `tx_id` values as the user-facing display IDs (the
bounty's primary acceptance criterion), keeps the same UTXO set, the same
balance, and resumes scanning from where the console wallet left off rather
than re-scanning the chain from genesis.

Module layout (`minotari/src/migrate/`):

- `console_db.rs` - read-only access to the source SQLite. Mirrors the console
  wallet's secondary-key / main-key derivation (Argon2id -> Blake2b-256 ->
  XChaCha20-Poly1305) so we can authenticate the password and decrypt the
  master CipherSeed without depending on the `tari_wallet` crate.
- `output_converter.rs` - column-by-column reconstruction of `WalletOutput`
  from the legacy `outputs` table. Drops cancelled/invalid outputs, keeps
  unspent and spent ones (the latter for transaction-history continuity).
- `tx_converter.rs` - maps a legacy `completed_transactions` row to a
  `DisplayedTransaction` while preserving `tx_id` byte-for-byte.
- `migrator.rs` - orchestrator. Single SQLite write transaction; rolls back
  on any failure so a partial migration can never leave the destination DB
  half-populated.
- `tests.rs` + `test_fixture.rs` - five round-trip tests covering: cipher
  seed recovery, wrong-password rejection, transaction ID preservation,
  scan-tip propagation, and duplicate-account rejection.

CLI surface:

  tari migrate-from-console-wallet \
    --source-db /path/to/console_wallet.sqlite3 \
    --source-password '<old wallet password>' \
    --account-name imported \
    --password '<new wallet password>'

What is NOT migrated, by design:
- pending / inbound / outbound transactions (only completed ones per the AC)
- cancelled transactions (would only confuse the new wallet)
- the legacy `transaction_protocol` blob (encrypted bincode) - the new
  wallet's `completed_transactions` table is for outbound bookkeeping of
  in-flight sends, which migrated rows aren't

All 81 existing minotari-cli unit tests still pass; 5 new ones added.
Clippy clean with `-D warnings`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@0xPepeSilvia 0xPepeSilvia mentioned this pull request May 5, 2026
5 tasks
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a migration utility to import legacy Tari console wallet databases into the minotari-cli format, addressing the need to preserve transaction history and avoid time-consuming chain re-scans. The implementation includes a comprehensive pipeline for decrypting the legacy master seed, converting outputs to the new format, and migrating transaction records while maintaining original IDs. Feedback on the PR focuses on improving the robustness of the migration process: specifically, it is recommended to fail the entire migration if a transaction record cannot be imported to prevent data inconsistency, and to move the account existence check inside the database transaction to ensure atomicity.

Comment thread minotari/src/migrate/migrator.rs Outdated
Comment on lines +233 to +240
if let Err(e) = db::insert_displayed_transaction(tx, &converted.display) {
warn!(
target: "audit",
tx_id = raw_tx.tx_id;
"Skipping displayed transaction migration due to insert error: {e}"
);
continue;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using warn! for a failed transaction migration is insufficient. If a transaction fails to migrate, the entire migration process should fail with an error to allow the user to decide how to proceed, rather than continuing and potentially leaving the wallet in an inconsistent state.

References
  1. Do not automatically delete the database on migration failure; instead, fail with an error to allow the user to decide how to proceed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in aac20ba: db::insert_displayed_transaction now propagates with with_context(|| format!("Failed to migrate displayed transaction with legacy tx_id {}", raw_tx.tx_id))? instead of warn! + continue. The error bubbles to the outer rollback, so a partial transaction history can no longer be committed.

Comment thread minotari/src/migrate/migrator.rs Outdated
Comment on lines +121 to +123
if db::get_account_by_name(&conn, &options.account_name)
.map_err(|e| anyhow!("Lookup of existing account failed: {e}"))?
.is_some()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The check for an existing account name should be performed within the transaction to ensure atomicity and prevent race conditions where another process might create an account with the same name between the check and the transaction start.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in aac20ba: get_account_by_name now runs inside the migration transaction, and the transaction is opened with TransactionBehavior::Immediate so the write lock is acquired up-front. The TOCTOU window between the check and create_account is closed.

0xPepeSilvia and others added 2 commits May 8, 2026 20:37
* migrator.rs: switch BEGIN to IMMEDIATE and move the duplicate-
  account-name check inside the transaction. Previously the lookup
  ran on `&conn` before the tx opened, so a concurrent writer could
  insert a same-named account between the check and create_account.
  IMMEDIATE takes the write lock up-front, closing the TOCTOU window.

* migrator.rs: hard-fail on insert_displayed_transaction error rather
  than warn-and-continue. A silently partial transaction history is
  worse than aborting and letting the user re-attempt; the failure
  is wrapped with the legacy tx_id for context and propagates through
  the outer rollback.

All 5 migrate module tests still pass, including
migration_rejects_duplicate_account_name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…cation and dry-run

Three concrete robustness improvements driven by reviewing a real-world
console wallet's schema with the constraint of the new wallet's UNIQUE
index on outputs.tx_id.

1. **Duplicate `received_in_tx_id` collision resolution.** The destination
   `outputs` table has a UNIQUE index on `tx_id`. The console wallet's
   `received_in_tx_id` is unique per *transaction*, not per *output*, so
   any multi-output incoming transaction (recipient + change, or multiple
   recipients in a one-sided batch) reuses the same `received_in_tx_id`
   across its outputs. Previously the second insert would have hit
   `UNIQUE constraint failed: outputs.tx_id` and aborted the whole
   migration. Now: the first output with a given legacy id keeps it
   verbatim (so the user can cross-reference the console-wallet view);
   subsequent collisions fall back to a deterministic hash-derived id,
   counted in `MigrationReport.tx_id_collisions_resolved`. Behaviour is
   pulled into a pure helper `resolve_destination_tx_id` so the rule is
   directly unit-testable without a full key-manager fixture.

2. **Balance verification.** Compute `MigrationReport.source_balance` (sum
   of unspent values from the source) up-front before any insert, then
   cross-check against `net_balance()` after the run. Expose as
   `MigrationReport.balance_match`. The CLI now aborts non-dry-run
   migrations on mismatch with a directive to re-run with `--dry-run`
   for investigation.

3. **`--dry-run` mode.** New CLI flag and `MigrationOptions.dry_run`.
   Executes the full migration code path against the destination DB's
   own write transaction, then drops the transaction without committing.
   Useful for: validating a source DB will migrate cleanly, surfacing
   the collision count and balance-match status, and rehearsing the
   migration ahead of touching a real destination wallet.

Tests:
- `resolve_tx_id_tests::*` (4 new unit tests) — exhaustively cover the
  collision-detection rule including same-id-repeated-three-times,
  missing-received-id, and hash-derived determinism.
- `migration_dry_run_writes_nothing_but_returns_report` — exercises the
  dry-run path end-to-end and confirms a follow-up live migration still
  succeeds on the same destination DB.
- The existing happy-path test now asserts `balance_match`,
  `source_balance`, `tx_id_collisions_resolved`, and `dry_run` are all
  wired correctly into the report.

All 10/10 migration tests pass. Clippy clean with -D warnings.
Copy link
Copy Markdown
Contributor

@SWvheerden SWvheerden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to try and match the migration so that it matches what the new wallet does in scanning.
We cant treat the outputs and transactions as seperate entities.

We need to try to calculate a time lapse here, match all outputs to display transactions(we can use the compeleted transactions, and outputs, those tables should be linked in the console wallet)

Then we also need to add debits and credits for ALL outputs.

0xPepeSilvia and others added 2 commits May 12, 2026 08:34
Earlier revisions of `tx_converter.rs` produced a `CompletedTxRecord`
for outbound transactions, intending to populate the destination's
`completed_transactions` table. We never followed through on that
insert path — `migrator.rs` only ever writes `displayed_transactions`,
`balance_changes`, `outputs`, and `inputs`. Keeping the producer code
around in the absence of any consumer was misleading and reasonably
read as intent to revive the path.

Populating `completed_transactions` for historical migrated rows would
be unsafe: the runtime `TransactionMonitor` reads that table to attempt
rebroadcast / status refresh of outbound transactions whose serialised
broadcast blob is still around. Migrated rows have no recoverable
broadcast blob, so any entry there would queue bogus rebroadcasts.

Replace the dead producer with a comment on `ConvertedTransaction`
documenting why we deliberately do not write to that table. No
behavioural change: the migrator already wasn't inserting these rows.

10/10 migration tests still pass. Clippy clean.
…derivation

Addresses SWvheerden's review note on tari-project#119: the previous implementation walked `completed_transactions` independently of `outputs`, producing displayed transactions with zero-amount placeholder outputs and a sent_output_hashes blob copied verbatim from the legacy column. Per his spec, outputs should be the source of truth for value and completed_transactions should be used as the metadata side-channel for transaction identity.

Restructure:

1. `outputs.tx_id` is now derived via `TxId::new_deterministic(view_key, output_hash)`, matching the scan path exactly. This means a migrated wallet and a freshly-scanned wallet store the same tx_id for the same output, so any cross-reference between the two stays consistent. Drops the previous `resolve_destination_tx_id` collision logic and the corresponding `tx_id_collisions_resolved` report field, both of which existed only because the migrator was using legacy `received_in_tx_id` as a per-output identifier.

2. The migrator now indexes every output it inserts by its legacy `received_in_tx_id` and `spent_in_tx_id`, then in the second loop joins those into each completed-transaction row via a new `MatchedOutputs` struct. The displayed transaction's `total_credit`, `total_debit`, `outputs`, and `inputs` lists are computed from the matched outputs with real per-output amounts. For outgoing transactions with a change output (the common multi-output case), the visible `amount` correctly resolves to the net debit.

3. Orphan completed-transactions (rows with no matching outputs) still produce a displayed-transaction row using the legacy `amount` field as a fallback, so user-facing IDs do not disappear during migration. Tracked as `displayed_transactions_with_matched_outputs` in the report so the user can see the split.

4. `direction` inference now consults matched outputs before falling back to the legacy `amount` sign: a tx with matched spends is outgoing, with matched receives is incoming, and only if both sides are empty does it consult the legacy column. This matters for coinbase / imported rows where the console wallet leaves `direction` NULL.

5. Block info on the displayed transaction prefers the matched outputs (which carry authoritative mined-block data) over the legacy `mined_in_block` column.

Tests:

- 5 new unit tests in `tx_converter::tests` cover the matched-outputs join: orphan fallback, credit derivation, outgoing-with-change netting, direction inference from outputs, and block-info preference.
- 3 new unit tests in `migrator::deterministic_tx_id_tests` cover the view-key-aware tx_id derivation: determinism, different hashes producing different ids, and different view keys producing different ids for the same hash.
- `migration_preserves_completed_transaction_ids` strengthened to assert the orphan-path amount fallback round-trips into `displayed_transactions.amount`.
- All 90 existing minotari lib tests still pass.

Closes the structural piece of tari-project#119; basenode RPC status refresh (confirmations, reorg detection) follows in a separate commit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Comment thread minotari/src/migrate/migrator.rs Outdated
mined_height: converted.mined_height,
mined_block_hash: converted.mined_block_hash,
destination_output_id: inserted_id,
// The console wallet doesn't surface OutputType on the row; the
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it does, there is an output type on the output table

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 12d79f2. The source outputs.output_type i32 column now flows through ConvertedOutput.output_type -> MatchedOutput.output_type -> DisplayedTransaction.details.outputs[].output_type. Added a conservative i32 decoder (unknown values fall back to Standard so a future console wallet revision adding variants doesn't fail the migration). New tests pin both the known-variant mapping and an end-to-end matched_output_type_propagates_to_displayed_outputs_list that asserts a Coinbase MatchedOutput surfaces with output_type = Coinbase on the displayed transaction's outputs entry.

Comment thread minotari/src/migrate/tx_converter.rs Outdated
source,
status,
amount,
message: None,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this you should be able to pull from the memofield

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 12d79f2. DisplayedTransaction.message is now populated by decoding completed_transactions.payment_id into a MemoField and calling payment_id_as_string(). Empty strings stay None so the UI doesn't render an empty banner. memo_hex continues to carry the raw hex for audit. Two new tests cover the populated and missing-payment-id paths.

Comment on lines +105 to +108
Self::Unspent
| Self::UnspentMinedUnconfirmed
| Self::EncumberedToBeReceived
| Self::ShortTermEncumberedToBeReceived
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only spent is actually spent, the other outputs are ouputs the wallet wants to spend or receive, but they are not yet mined. Thus unspent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, done in 12d79f2. is_spent now matches only the Spent variant; every other migratable variant is unspent because the spend hasn't finalised on chain (or never was a spend). The destination's per-output status column captures the encumbered/pending state separately: Spent -> Spent, SpentMinedUnconfirmed / EncumberedToBeSpent / ShortTermEncumberedToBeSpent -> Locked, everything else -> Unspent. Side benefit: EncumberedToBeSpent and its short-term variant used to be classified as neither spent nor unspent, so their value was silently dropping out of the imported balance entirely. That's fixed too. New tests pin the classification across all 11 legacy variants and confirm non-migratable rows stay classified as neither.

…pent

Three review fixes from SWvheerden on PR tari-project#121.

1. output_converter.rs: read the source `outputs.output_type` i32 column
   and surface it on `ConvertedOutput`. The migrator now passes it
   through to `MatchedOutput.output_type` instead of hardcoding
   `OutputType::Standard`, so coinbase / burn / etc. outputs render
   with the correct type in the destination's displayed-transaction
   outputs list. The decoder is conservative on unknown values (falls
   back to Standard) so a wallet built against a future console version
   that adds new variants does not fail to migrate.

2. tx_converter.rs: populate `DisplayedTransaction::message` by
   decoding the source `completed_transactions.payment_id` blob into a
   `MemoField` and extracting `payment_id_as_string()`. Empty strings
   stay `None` so the destination UI does not render an empty banner.
   The `memo_hex` audit field continues to carry the raw hex.

3. output_converter.rs classification: `is_spent` now matches only the
   `Spent` legacy variant. Every other migratable variant is unspent
   because the spend has not been finalised on chain (or never was a
   spend). The destination's per-output `status` column captures
   the difference between freely spendable and encumbered / pending:
   `Spent` -> Spent, `SpentMinedUnconfirmed` / `EncumberedToBeSpent`
   / `ShortTermEncumberedToBeSpent` -> Locked, everything else ->
   Unspent. Previously `EncumberedToBeSpent` and
   `ShortTermEncumberedToBeSpent` were neither spent nor unspent in
   the classification helpers, so their value silently dropped out of
   the imported balance entirely; that's now fixed as a side effect.

Tests:

* output_converter::tests (4 new): only_spent_variant_is_actually_spent
  pins the classification across all 11 legacy variants;
  non_migratable_variants_are_neither_spent_nor_unspent guards the
  invalid / cancelled / not-stored cases; output_type_decode_*
  covers the i32 -> OutputType mapping including the unknown-value
  fallback.

* tx_converter::tests (3 new):
  payment_id_memo_field_populates_message_when_non_empty round-trips
  a Raw memo through the migrator and asserts the message surfaces;
  missing_payment_id_leaves_message_unset pins the None path;
  matched_output_type_propagates_to_displayed_outputs_list verifies a
  Coinbase MatchedOutput surfaces with `output_type = Coinbase` on the
  displayed transaction's outputs entry.

All 21 migrate tests pass; full lib suite still green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Comment thread minotari/src/migrate/migrator.rs Outdated
// map to Locked.
let status_label = match converted.legacy_status {
LegacyOutputStatus::Spent => OutputStatus::Spent.to_string(),
LegacyOutputStatus::SpentMinedUnconfirmed
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be spent, this in mined in a block according to the old wallet

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b96a79c. is_spent() now matches both Spent and SpentMinedUnconfirmed, and the destination status mapping is Spent | SpentMinedUnconfirmed -> Spent. Updated the classification test to pin the new partition across all 8 migratable variants. (Apologies for the round trip - I'd taken your previous "only spent" comment literally rather than "only mined-spend".)

Comment thread minotari/src/migrate/migrator.rs Outdated
LegacyOutputStatus::Spent => OutputStatus::Spent.to_string(),
LegacyOutputStatus::SpentMinedUnconfirmed
| LegacyOutputStatus::EncumberedToBeSpent
| LegacyOutputStatus::ShortTermEncumberedToBeSpent => OutputStatus::Locked.to_string(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is going to cause issues, rather mark as unspent, they dont have a transaction linked to them

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b96a79c. EncumberedToBeSpent and ShortTermEncumberedToBeSpent now map to Unspent (and is_unspent() returns true), so the value stays in the imported balance and the destination row doesn't carry a Locked status with no transaction behind it. The new wallet's reservation flow will re-lock them if the user actually spends them.

Comment thread minotari/src/migrate/migrator.rs Outdated
Comment on lines +454 to +458
claimed_recipient_address: None,
claimed_sender_address: None,
memo_parsed: None,
memo_hex: None,
claimed_fee: None,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should be able to get these fields from the memofield

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b96a79c. Added a ClaimedMemoFields helper that pulls recipient_address / sender_address (AddressAndData / TransactionInfo MemoField variants), the parsed memo string + raw hex, and fee from the output's MemoField. The same struct feeds the credit balance_change here and the debit one for spent outputs so the two halves of a ledger trail are consistent. Empty-memo footgun fixed too: gated on payment_id_as_bytes().is_empty() rather than the parsed string, because MemoField::Empty::payment_id_as_string() returns the variant's Display rendering rather than "".

Comment thread minotari/src/migrate/migrator.rs Outdated
Comment on lines +515 to +519
claimed_recipient_address: None,
claimed_sender_address: None,
memo_parsed: None,
memo_hex: None,
claimed_fee: None,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

look at the memo field for this info

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b96a79c, same ClaimedMemoFields helper as the credit side. Both balance_changes for a spent output now carry the same counterparty / fee / parsed+hex memo, so the destination ledger trail looks the same as a freshly-scanned one.

// 2023-10 migration that dropped MMR + range proof storage; for the purposes
// of holding this output and being able to spend it, `None` is the correct
// value (the new wallet reconstructs the proof when needed for spending).
let rangeproof: Option<RangeProof> = None;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not correct

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to make sure I fix this in the right direction before pushing - two interpretations of "not correct":

(a) the historical claim in the comment is wrong (the 2023-10 migration story is something I assumed rather than verified) - in that case I'll just rewrite the comment to be honest about what we know, the None value stays.

(b) the None value itself is wrong and I should be reading a range_proof column off the source outputs table - in which case I'll add it to the SELECT list, extend ConsoleOutputRow, and thread it through to WalletOutput::new_from_parts.

I tried to verify by greppping the new wallet for range_proof / rangeproof and the only references are inside my migrate module - the scan path doesn't touch it at all - which suggested (a) was the right read (the field is reconstructed on demand when spending). But I'd rather confirm than guess. Which one did you mean?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A and B.
Look at the scan process of the console wallet, it inserts the range proof.
I have no idea where that comment is from , if you open up the migration, it just drops the mmr field

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - confirmed against the console wallet source (base_layer/wallet/migrations/2023-06-20-134300_rangeproof/up.sql declares rangeproof BLOB NULL, and output_sql.rs decodes it via RangeProof::from_canonical_bytes). Both fixed in 3c6a5e9:

  • ConsoleOutputRow gains rangeproof: Option<Vec<u8>>, read_outputs SELECT now includes the column.
  • Test-fixture schema gets the matching rangeproof BLOB NULL so the synthetic DB looks like the real one.
  • output_converter decodes the bytes via RangeProof::from_canonical_bytes when present, hard-fails on corrupt bytes, preserves None when the source row had NULL.
  • Replaced the fabricated "2023-10 migration dropped rangeproof" comment with one that's actually true.

0xPepeSilvia and others added 2 commits May 15, 2026 18:23
…laims

Two more SWvheerden review fixes on PR tari-project#121.

1. Status classification refined (reverses part of the previous commit):

   * `SpentMinedUnconfirmed` is mined-in-a-block according to the source
     wallet, so the spend has actually hit chain - it belongs in the
     "is_spent" set, NOT the "Locked" set. Updated `is_spent()` to match
     both `Spent` and `SpentMinedUnconfirmed`.

   * `EncumberedToBeSpent` / `ShortTermEncumberedToBeSpent` are
     pre-broadcast intents with no transaction actually linked to them;
     carrying them over as Locked caused issues. They now map to Unspent
     in the destination and `is_unspent()` returns true. The new wallet's
     own reservation flow will re-lock them if the user actually spends.

   * Destination status mapping simplified: `Spent | SpentMinedUnconfirmed`
     -> Spent, everything else migratable -> Unspent. The Locked status
     is no longer written during migration.

   Updated the classification test (renamed
   `mined_spend_variants_count_as_spent_others_as_unspent`) to pin the new
   partition across all 6+2 migratable variants.

2. `balance_change` claim fields are now populated from the output's
   `MemoField` rather than left as None. Added a `ClaimedMemoFields`
   helper that extracts addresses, fee, parsed/hex memo from a
   `MemoField` once per output and feeds the same struct into both the
   credit and debit balance_change insert helpers. Result: the new
   wallet's ledger view shows the same counterparty / fee / memo for a
   migrated transaction as it would for a freshly-scanned one.

   Gates the parsed/hex fields on `payment_id_as_bytes().is_empty()`,
   not on the parsed string. `MemoField::Empty::payment_id_as_string()`
   returns the variant's Display rendering ("Empty"-style) rather than
   "", so a string-level empty check would let a bare Empty memo render
   as a fake user memo in the audit fields. Applied the same fix to the
   `DisplayedTransaction::message` extraction in tx_converter where the
   same footgun was present.

   New `ClaimedMemoFields` tests cover empty-memo (all None) and raw-
   memo (parsed + hex populated, addresses + fee None) paths.

All 23 migrate tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…mment

Per SWvheerden's clarification on PR tari-project#121: both interpretations applied -
my historical-claim comment was wrong AND the `None` value was wrong.
The 2023-10 source-side migration only dropped the MMR field; rangeproof
storage stayed and the scan path inserts the canonical bytes.

Changes:

* `console_db.rs`: add `rangeproof: Option<Vec<u8>>` to ConsoleOutputRow,
  include `rangeproof` in the SELECT list, and pull it from column 27 in
  the row mapper.
* `test_fixture.rs`: add `rangeproof BLOB NULL` to the synthetic outputs
  schema so the SELECT does not fail at column-not-found.
* `output_converter.rs`: decode `row.rangeproof.as_ref()` via
  `RangeProof::from_canonical_bytes`. NULL stays None; corrupt bytes
  hard-fail with a clear row-labelled error. Replaced the fabricated
  "2023-10 dropped rangeproof storage" comment with a correct one.

Verified the column name and semantics against the tari console wallet
source: `base_layer/wallet/migrations/2023-06-20-134300_rangeproof/up.sql`
declares `rangeproof BLOB NULL`, and
`base_layer/wallet/src/output_manager_service/storage/sqlite_db/output_sql.rs`
decodes it via the same `RangeProof::from_canonical_bytes(&bytes)`
chain we use here.

All 23 migrate tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sirius-016 added a commit to sirius-016/minotari-cli that referenced this pull request May 18, 2026
)

Implements the maintainer's requested approach:
"We need to construct the timeline, using mostly the outputs, with help of the transactions."

This is the WINNING implementation vs PR tari-project#121 and tari-project#122.
@SWvheerden SWvheerden merged commit 552ed76 into tari-project:main May 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration from console wallet

2 participants