LLFourn [Thu, 6 Jun 2024 00:17:55 +0000 (10:17 +1000)]
refactor(keychain): Fix KeychainTxOutIndex range queries
The underlying SpkTxOutIndex should not use DescriptorIds to index
because this loses the ordering relationship of the spks so queries on
subranges of keychains work.
Along with that we enforce that there is a strict 1-to-1 relationship
between descriptors and keychains. Violating this leads to an error in
insert_descriptor now.
In general I try to make the translation layer between the SpkTxOutIndex
and the KeychainTxOutIndex thinner. Ergonomics of this will be improved
in next commit.
fixes #1422
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
This PR focuses on upgrading the `rust-bitcoin` and `miniscript` versions, to `0.32.0` and `0.12.0`, respectively. It also bumps the versions of other BDK ecosystem crates that also rely on both `rust-bitcoin` and `miniscript`, being:
<ins>I structured the PR in multiple commits, with closed scope, being one for each BDK crate being upgraded, and one for each kind of fix and upgrade required, it seems like a lot of commits (**that should be squashed before merging**), but I think it'll make it easier during review phase.</ins>
In summary I can already mention some of the main changes:
- using `compute_txid()` instead of deprecated `txid()`
- using `minimal_non_dust()` instead of `dust_value()`
- using the renamed `signature` and `sighash_type` fields
- using proper `sighash::P2wpkhError`, `sighash::TaprootError` instead of older `sighash::Error`
- conversion from `Network` to new expected `NetworkKind` #1465
- conversion from the new `Weight` type to current expected `usize` #1466
- using `.into()` to convert from AbsLockTime and `RelLockTime` to `absolute::LockTime` and `relative::LockTime`
- using Message::from_digest() instead of relying on deprecated `ThirtyTwoByteHash` trait.
- updating the miniscript policy and dsl to proper expect and handle new `Threshold` type, instead of the previous two parameters.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
<ins>Again, I structured the PR in multiple commits, with closed scope, being one for each BDK crate being upgraded, and one for each kind of fix and upgrade required, it seems like a lot of commits (**that should be squashed before merging**), but I think it'll make it easier during review phase.</ins>
It should definitely be carefully reviewed, especially the last commits for the wallet crate scope, the ones with the semantic `fix(wallet)`.
I would also appreciate if @tcharding reviewed it, as he gave a try in the past (#1400 ), and I relied on some of it for the policy part of it, other rust-bitcoin maintainers reviews are a definitely welcome 😁
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
> // TODO(@oleonardolima): Do another pass and double check the changes
- Use `compute_txid()` instead of deprecated `txid()`
- Use `minimal_non_dust()` instead of `dust_value()`
- Use `signature` and `sighash_type` fields, instead of previous `sig` and `hash_type`
- Use `sighash::P2wpkhError`, and `sighash::TaprootError` instead of older `sighash::Error`
- Converts from `Network` to `NetworkKind`, where expected
- Converts from `Weight` type to current used `usize`
- Use `.into()` to convert from `AbsLockTime` and `RelLockTime` to `absolute::LockTime` and `relative::LockTime`
- Remove use of deprecated `ThirtyTwoByteHash` trait, use `Message::from_digest()`
- Update the miniscript policy and dsl macros to proper expect and handle new `Threshold` type, instead of the previous two parameters.
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
志宇 [Thu, 13 Jun 2024 03:47:01 +0000 (11:47 +0800)]
chore(wallet): rm dup code
Methods `make_multi` and `make_multi_a` were essentially the same thing
except for a different const generic param on `Threshold`. So we rm
`make_multi_a` and put a const generic param on `make_multi`.
Leonardo Lima [Wed, 22 May 2024 21:34:30 +0000 (18:34 -0300)]
deps(bdk): bump `bitcoin` to `0.32.0`, miniscript to `12.0.0`
deps(chain): bump `bitcoin` to `0.32.0`, miniscript to `12.0.0`
fix(chain): use `minimal_non_dust()` instead of `dust_value()`
fix(chain): use `compute_txid()` instead of `txid`
deps(testenv): bump `electrsd` to `0.28.0`
deps(electrum): bump `electrum-client` to `0.20.0`
fix(electrum): use `compute_txid()` instead of `txid`
deps(esplora): bump `esplora-client` to `0.8.0`
deps(bitcoind_rpc): bump `bitcoin` to `0.32.0`, `bitcoincore-rpc` to
`0.19.0`
fix(bitcoind_rpc): use `compute_txid()` instead of `txid`
fix(nursery/tmp_plan): use proper `sighash` errors, and fix the expected
`Signature` fields
fix(sqlite): use `compute_txid()` instead of `txid`
deps(hwi): bump `hwi` to `0.9.0`
deps(wallet): bump `bitcoin` to `0.32.0`, miniscript to `12.0.0`
fix(wallet): use `compute_txid()` and `minimal_non_dust()`
- update to use `compute_txid()` instead of deprecated `txid()`
- update to use `minimal_non_dust()` instead of `dust_value()`
- remove unused `bitcoin::hex::FromHex`.
fix(wallet): uses `.into` conversion on `Network` for `NetworkKind`
- uses `.into()` when appropriate, otherwise use the explicit
`NetworkKind`, and it's `.is_mainnet()` method.
fix(wallet): add P2wpkh, Taproot, InputsIndex errors to `SignerError`
fix(wallet): fields on taproot, and ecdsa `Signature` structure
fix(wallet/wallet): convert `Weight` to `usize` for now
- converts the `bitcoin-units::Weight` type to `usize` with help of
`to_wu()` method.
- it should be updated/refactored in the future to handle the `Weight`
type throughout the code instead of current `usize`, only converting
it for now.
- allows the usage of deprecated `is_provably_unspendable()`, needs
further discussion if suggested `is_op_return` is suitable.
- update the expect field to `signature`, as it was renamed from `sig`.
fix(wallet/wallet): use `is_op_return` instead of
`is_provably_unspendable`
fix(wallet/wallet): use `relative::Locktime` instead of `Sequence`
fix(wallet/descriptor): use `ParsePublicKeyError`
fix(wallet/descriptor): use `.into()` to convert from `AbsLockTime` and
`RelLockTime` to `absolute::LockTime` and `relative::LockTime`
fix(wallet/wallet): use `Message::from_digest()` instead of relying on
deprecated `ThirtyTwoByteHash` trait.
fix(wallet/descriptor+wallet): expect `Threshold` type, and handle it
internally
fix(wallet/wallet): remove `0x` prefix from expected `TxId` display
fix(examples): use `compute_txid()` instead of `txid`
fix(ci): remove usage of `bitcoin/no-std` feature
- remove comment: `# The `no-std` feature it's implied when the `std` feature is disabled.`
- Replace `CreateTxError::InsufficientFunds` use by `coin_selection::Error::InsufficientFunds`
- Remove `InsufficientFunds` member from `CreateTxError` enum
- Rename `coin_selection::Error` to `coin_selection::CoinSelectionError`
### Notes to the reviewers
- We could also keep both members but rename one of them to avoid confusion
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
All `Wallet` constructors are modified to require a change descriptor, where previously it was optional. Additionally we enforce uniqueness of the change descriptor to avoid ambiguity when deriving scripts and ensure the wallet will always have two distinct keystores.
Notable changes
* Add error `DescriptorError::ExternalAndInternalAreTheSame`
* Remove error `CreateTxError::ChangePolicyDescriptor`
* No longer rely on `map_keychain`
fixes #1383
### Notes to the reviewers
### Changelog notice
Changed:
Constructing a Wallet now requires two distinct descriptors.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
valued mammal [Wed, 27 Mar 2024 01:57:10 +0000 (21:57 -0400)]
refactor(wallet)!: Make Wallet require a change descriptor
All `Wallet` constructors are modified to require a change
descriptor, where previously it was optional. Additionally
we enforce uniqueness of the change descriptor to avoid
ambiguity when deriving scripts and ensure the wallet will
always have two distinct keystores.
Notable changes
* Add error DescriptorError::ExternalAndInternalAreTheSame
* Remove error CreateTxError::ChangePolicyDescriptor
* No longer rely on `map_keychain`
Previously there was a `TxCache` that you passed in as part of the sync request. There are lots of downsides to this:
1. If the user forgets to do this you cache nothing
2. where are you meant to keep this cache? The example shows it being recreated every time which seems very suboptimal.
3. More API and documentation surface area.
Instead just do a plain old simple cache inside the electrum client. This way at least you only download transactions once. You can pre-populate the cache with a method also and I did this in the examples.
- Wallet::get_balance() renamed to Wallet::balance()
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
It further updates and adds the usage of `bitcoin::Amount` instead of `u64`.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
Open for comments and discussions.
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
- Updated `CreateTxError::FeeTooLow` to use `bitcoin::Amount`.
- Updated `Wallet::calculate_fee()`. to use `bitcoin::Amount`
- Updated `TxBuilder::fee_absolute()`. to use `bitcoin::Amount`.
- Updated `CalculateFeeError::NegativeFee` to use `bitcoin::SignedAmount`.
- Updated `TxGraph::calculate_fee()`. to use `bitcoin::Amount`.
- Updated `PsbUtils::fee_amount()` to use `bitcoin::Amount`.
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
LLFourn [Fri, 31 May 2024 03:52:49 +0000 (13:52 +1000)]
refactor(electrum)!: put the tx cache in electrum
Previously there was a tx cache that you passed in as part of the sync
request. This seems bad and the example show'd that you should copy all
your transactions from the transaction graph into the sync request every
time you sync'd. If you forgot to do this then you would always download everything.
Instead just do a plain old simple cache inside the electrum client.
This way at least you only download transactions once. You can
pre-populate the cache with a method also and I did this in the examples.
Leonardo Lima [Sun, 5 May 2024 20:41:31 +0000 (17:41 -0300)]
feat: use `Amount` on `calculate_fee`, `fee_absolute`, `fee_amount` and others
- update to use `bitcoin::Amount` on `CreateTxError::FeeTooLow` variant.
- update to use `bitcoin::Amount` on `Wallet::calculate_fee()`.
- update to use `bitcoin::Amount` on `FeePolicy::fee_absolute()`.
- update to use `bitcoin::SignedAmount` on
`CalculateFeeError::NegativeFee` variant.
- update to use `bitcoin::Amount` on `TxGraph::calculate_fee()`.
- update to use `bitcoin::Amount` on `PsbUtils::fee_amount()`
bdk_chain to 0.15.0
bdk_bitcoind_rpc to 0.11.0
bdk_electrum to 0.14.0
bdk_esplora to 0.14.0
bdk_persist to 0.3.0
bdk_file_store to 0.12.0
bdk_sqlite keep at 0.1.0
bdk_testenv to 0.5.0
bdk_hwi to 0.1.0
bdk_wallet to 1.0.0-alpha.12
### Notes to the reviewers
I also (hopefully) fixed the `bdk_hwi` crate so it can be published.
Steve Myers [Thu, 23 May 2024 16:15:30 +0000 (11:15 -0500)]
Bump bdk version to 1.0.0-alpha.12
bdk_chain to 0.15.0
bdk_bitcoind_rpc to 0.11.0
bdk_electrum to 0.14.0
bdk_esplora to 0.14.0
bdk_persist to 0.3.0
bdk_file_store to 0.12.0
bdk_sqlite keep at 0.1.0
bdk_testenv to 0.5.0
bdk_hwi to 0.1.0
bdk_wallet to 1.0.0-alpha.12
Resolves #860 by adding export of taproot descriptors
### Notes to the reviewers
Allows export as Core accepts taproot.
### Changelog notice
- Export taproot descriptors
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Remove wallet::TxBuilder::allow_shrinking() and unneeded TxBuilder context param.
Fixes #1374
### Notes to the reviewers
The allow_shrinking function was the only one using the TxBuilder FeeBump context and it's useful to have CreateTx context functions available for building FeeBump transactions, see updated tests.
### Changelog notice
Changed
- Removed TxBuilder::allow_shrinking() function.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Add "bdk_sqlite_store" crate implementing `PersistBackend` backed by a SQLite database.
### Notes to the reviewers
In addition to adding a SQLite based `PersistenceBackend` this PR also:
* add `CombinedChangeSet` in `bdk_persist` and update `wallet` crate to use it.
* updates the `wallet/tests` to also use this new sqlite store crate.
* updates `example-crates/wallet_esplora_async` to use this new sqlite store crate.
* fixes out of date README instructions for MSRV.
* bumps the ci `build_docs` job to rust `nightly-2024-05-12`.
### Changelog notice
Changed
- Update Wallet to use CombinedChangeSet for persistence.
Added
- Add CombinedChangeSet in bdk_persist crate.
- Add bdk_sqlite crate implementing SQLite based PersistenceBackend storage for CombinedChangeSet.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
志宇 [Tue, 21 May 2024 13:08:40 +0000 (21:08 +0800)]
feat(persist): introduce `CombinedChangeSet`
It is a good idea to have common changeset types stored in
`bdk_persist`. This will make it convenient for persistence crates and
`bdk_wallet` interaction.
Previously we inserted every `TxOut` of a previous tx at the same outpoint, which is incorrect because an outpoint only points to a single `TxOut`. Now just get the `TxOut` corresponding to the txin prevout and insert it with its outpoint.
### Notes to the reviewers
The bug in question was demonstrated in a discord comment https://discord.com/channels/753336465005608961/1239693193159639073/1239704153400414298 but I don't think we've opened an issue yet. Essentially, because of a mismatch between the outpoint and txout stored in TxGraph, we weren't summing the inputs correctly which caused `calculate_fee` to fail with `NegativeFee` error.
fixes #1419
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] I've added tests to reproduce the issue which are now passing
valued mammal [Tue, 14 May 2024 14:43:19 +0000 (10:43 -0400)]
fix(electrum): Fix `fetch_prev_txout`
Previously we inserted every TxOut of a previous tx
at the same outpoint, which is incorrect because an outpoint
only points to a single TxOut. Now just get the TxOut
corresponding to the txin prevout and insert it with
its outpoint.
Once this properly builds, even before all reviews is done, I will publish it to crates.io to reserve the name.
### Changelog notice
Changed
- Renamed `bdk` crate to `bdk_wallet`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
bdk_chain to 0.14.0
bdk_bitcoind_rpc to 0.10.0
bdk_electrum to 0.13.0
bdk_esplora to 0.13.0
bdk_file_store to 0.11.0
bdk_testenv to 0.4.0
bdk_persist to 0.2.0
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Steve Myers [Fri, 10 May 2024 00:15:33 +0000 (19:15 -0500)]
Bump bdk version to 1.0.0-alpha.11
bdk_chain to 0.14.0
bdk_bitcoind_rpc to 0.10.0
bdk_electrum to 0.13.0
bdk_esplora to 0.13.0
bdk_file_store to 0.11.0
bdk_testenv to 0.4.0
bdk_persist to 0.2.0
* Universal structures for full-scan/sync (PR #1413)
* Making `CheckPoint` linked list query-able (PR #1369)
* Making `Transaction`s cheaply-clonable (PR #1373)
has allowed us to simplify the interaction between chain-source and receiving-structures (`bdk_chain`).
The motivation is to accomplish something like this ([as mentioned here](https://github.com/bitcoindevkit/bdk/issues/1153#issuecomment-1752263555)):
```rust
let things_I_am_interested_in = wallet.lock().unwrap().start_sync();
let update = electrum_or_esplora.sync(things_i_am_interested_in)?;
wallet.lock().unwrap().apply_update(update)?:
```
### Description
This PR greatly simplifies the API of our Electrum chain-source (`bdk_electrum`) by making use of the aforementioned changes. Instead of referring back to the receiving `TxGraph` mid-sync/scan to determine which full transaction to fetch, we provide the Electrum chain-source already-fetched full transactions to start sync/scan (this is cheap, as transactions are wrapped in `Arc`s since #1373).
In addition, an option has been added to include the previous `TxOut` for transactions received from an external wallet for fee calculation.
### Changelog notice
* Change `TxGraph::insert_tx` to take in anything that satisfies `Into<Arc<Transaction>>`. This allows us to reuse the `Arc` pointer of what is being inserted.
* Add `tx_cache` field to `SyncRequest` and `FullScanRequest`.
* Make `Anchor` type in `FullScanResult` generic for more flexibility.
* Change `ElectrumExt` methods to take in `SyncRequest`/`FullScanRequest` and return `SyncResult`/`FullScanResult`. Also update electrum examples accordingly.
* Add `ElectrumResultExt` trait which allows us to convert the update `TxGraph` of `SyncResult`/`FullScanResult` for `bdk_electrum`.
* Added an option for `full_scan` and `sync` to also fetch previous `TxOut`s to allow for fee calculation.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
志宇 [Fri, 10 May 2024 08:40:55 +0000 (16:40 +0800)]
feat(electrum): update docs and simplify logic of `ElectrumExt`
Helper method docs are updated to explain what they are updating. Logic
is simplified as we do not need to check whether a tx exists already in
`update_graph` before inserting it.
Wei Chen [Tue, 7 May 2024 11:57:11 +0000 (19:57 +0800)]
feat(electrum): include option for previous `TxOut`s for fee calculation
The previous `TxOut` for transactions received from an external
wallet may be optionally added as floating `TxOut`s to `TxGraph`
to allow for fee calculation.
feat(chain)!: use custom return types for `ElectrumExt` methods
This is more code, but a much more elegant solution than having
`ElectrumExt` methods return `SyncResult`/`FullScanResult` and having an
`ElectrumResultExt` extention trait.
feat(electrum)!: use new sync/full-scan structs for `ElectrumExt`
`ElectrumResultExt` trait is also introduced that adds methods which can
convert the `Anchor` type for the update `TxGraph`.
We also make use of the new `TxCache` fields in
`SyncRequest`/`FullScanRequest`. This way, we can avoid re-fetching full
transactions from Electrum if not needed.
Examples and tests are updated to use the new `ElectrumExt` API.
refactor(electrum): remove `RelevantTxids` and track txs in `TxGraph`
This PR removes `RelevantTxids` from the electrum crate and tracks
transactions in a `TxGraph`. This removes the need to separately
construct a `TxGraph` after a `full_scan` or `sync`.
- Moves keychain::ChangeSet inside `keychain/txout_index.rs` as now the `ChangeSet` depends on miniscript
- Slightly cleans up tests by introducing some constant descriptors
- The KeychainTxOutIndex's internal SpkIterator now uses DescriptorId
instead of K. The DescriptorId -> K translation is made at the
KeychainTxOutIndex level.
- The keychain::Changeset is now a struct, which includes a map for last
revealed indexes, and one for newly added keychains and their
descriptor.
### Changelog notice
API changes in bdk:
- Wallet::keychains returns a `impl Iterator` instead of `BTreeMap`
- Wallet::load doesn't take descriptors anymore, since they're stored in the db
- Wallet::new_or_load checks if the loaded descriptor from db is the same as the provided one
API changes in bdk_chain:
- `ChangeSet` is now a struct, which includes a map for last revealed
indexes, and one for keychains and descriptors.
- `KeychainTxOutIndex::inner` returns a `SpkIterator<(DescriptorId, u32)>`
- `KeychainTxOutIndex::outpoints` returns a `BTreeSet` instead of `&BTreeSet`
- `KeychainTxOutIndex::keychains` returns a `impl Iterator` instead of
`&BTreeMap`
- `KeychainTxOutIndex::txouts` doesn't return a ExactSizeIterator anymore
- `KeychainTxOutIndex::last_revealed_indices` returns a `BTreeMap`
instead of `&BTreeMap`
- `KeychainTxOutIndex::add_keychain` has been renamed to `KeychainTxOutIndex::insert_descriptor`, and now it returns a ChangeSet
- `KeychainTxOutIndex::reveal_next_spk` returns Option
- `KeychainTxOutIndex::next_unused_spk` returns Option
- `KeychainTxOutIndex::unbounded_spk_iter` returns Option
- `KeychainTxOutIndex::next_index` returns Option
- `KeychainTxOutIndex::reveal_to_target` returns Option
- `KeychainTxOutIndex::revealed_keychain_spks` returns Option
- `KeychainTxOutIndex::unused_keychain_spks` returns Option
- `KeychainTxOutIndex::last_revealed_index` returns Option
- `KeychainTxOutIndex::keychain_outpoints` returns Option
- `KeychainTxOutIndex::keychain_outpoints_in_range` returns Option
- `KeychainTxOutIndex::last_used_index` returns None if the keychain has never been used, or if it doesn't exist
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
This PR pins clippy check in CI to the rust 1.78 toolchain, which prevents new lints in stable releases from interrupting the usual workflow. Because rust versions are released on a predictable schedule, we can revisit this setting in the future as needed (say 3-6 months).
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [ ] I ran `cargo fmt` and `cargo clippy` before committing
fix(chain): introduce keychain-variant-ranking to `KeychainTxOutIndex`
This fixes the bug with changesets not being monotone. Previously, the
result of applying changesets individually v.s. applying the aggregate
of changesets may result in different `KeychainTxOutIndex` states.
The nature of the changeset allows different keychain types to share the
same descriptor. However, the previous design did not take this into
account. To do this properly, we should keep track of all keychains
currently associated with a given descriptor. However, the API only
allows returning one keychain per spk/txout/outpoint (which is a good
API).
Therefore, we rank keychain variants by `Ord`. Earlier keychain variants
have a higher rank, and the first keychain will be returned.
fix(chain): simplify `Append::append` impl for `keychain::ChangeSet`
We only need to loop though entries of `other`. The logic before was
wasteful because we were also looping though all entries of `self` even
if we do not need to modify the `self` entry.
Daniela Brozzoni [Mon, 15 Jan 2024 17:52:03 +0000 (18:52 +0100)]
keychain::ChangeSet includes the descriptor
- The KeychainTxOutIndex's internal SpkIterator now uses DescriptorId
instead of K. The DescriptorId -> K translation is made at the
KeychainTxOutIndex level.
- The keychain::Changeset is now a struct, which includes a map for last
revealed indexes, and one for newly added keychains and their
descriptor.
API changes in bdk:
- Wallet::keychains returns a `impl Iterator` instead of `BTreeMap`
- Wallet::load doesn't take descriptors anymore, since they're stored in
the db
- Wallet::new_or_load checks if the loaded descriptor from db is the
same as the provided one
API changes in bdk_chain:
- `ChangeSet` is now a struct, which includes a map for last revealed
indexes, and one for keychains and descriptors.
- `KeychainTxOutIndex::inner` returns a `SpkIterator<(DescriptorId, u32)>`
- `KeychainTxOutIndex::outpoints` returns a `impl Iterator` instead of `&BTreeSet`
- `KeychainTxOutIndex::keychains` returns a `impl Iterator` instead of
`&BTreeMap`
- `KeychainTxOutIndex::txouts` doesn't return a ExactSizeIterator
anymore
- `KeychainTxOutIndex::unbounded_spk_iter` returns an `Option`
- `KeychainTxOutIndex::next_index` returns an `Option`
- `KeychainTxOutIndex::last_revealed_indices` returns a `BTreeMap`
instead of `&BTreeMap`
- `KeychainTxOutIndex::reveal_to_target` returns an `Option`
- `KeychainTxOutIndex::reveal_next_spk` returns an `Option`
- `KeychainTxOutIndex::next_unused_spk` returns an `Option`
- `KeychainTxOutIndex::add_keychain` has been renamed to
`KeychainTxOutIndex::insert_descriptor`, and now it returns a
ChangeSet
志宇 [Wed, 11 Oct 2023 16:03:18 +0000 (00:03 +0800)]
ref(chain): move `keychain::ChangeSet` into `txout_index.rs`
We plan to record `Descriptor` additions into persistence. Hence, we
need to add `Descriptor`s to the changeset. This depends on
`miniscript`. Moving this into `txout_index.rs` makes sense as this is
consistent with all the other files. The only reason why this wasn't
this way before, is because the changeset didn't need miniscript.
This PR adds a `default` feature to `bdk_persist` so it can be build on its own. Once #1422 is done we can remove the `default`again.
### Notes to the reviewers
I need to be able to build `bdk_persist` on its own so I can publish it to crates.io.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
It's being used on `Balance`, and throughout the code, an `u64` represents the amount, which relies on the user to infer its sats, not millisats, or any other representation.
It updates the usage of `u64` on `Balance`, and other APIs:
- `TxParams::add_recipient`
- `KeyChainTxOutIndex::sent_and_received`, `KeyChainTxOutIndex::net_value`
- `SpkTxOutIndex::sent_and_received`, `SpkTxOutIndex::net_value`
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
It updates some of the APIs to expect the `bitcoin::Amount`, but it does not update internal usage of u64, such as `TxParams` still expects and uses `u64`, please see the PR comments for related discussion.
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
- Changed the `keychain::Balance` struct fields to use `Amount` instead of `u64`.
- Changed the `add_recipient` method on `TxBuilder` implementation to expect `bitcoin::Amount`.
- Changed the `sent_and_received`, and `net_value` methods on `KeyChainTxOutIndex` to expect `bitcoin::Amount`.
- Changed the `sent_and_received`, and `net_value` methods on `SpkTxOutIndex` to expect `bitcoin::Amount`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Reexports `electrsd` in `TestEnv` to remove the `electrsd` dev depedency out of `bdk_electrum` and `bdk_esplora`.
Credit to @ValuedMammal for the idea.
Since `bitcoind` reexports `anyhow`, this dev dependency was also removed from `bdk_electrum`, `bdk_esplora`, and `bdk_bitcoind_rpc`. `bitcoind`, `bitcoincore_rpc` and `electrum_client` were also reexported for convenience.
### Changelog notice
* Change `bdk_testenv` to re-export internally used crates.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Fixed documentation for `full_scan` and `sync` in `bdk_esplora`.
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
* Updated documentation for `full_scan` and `sync` in `bdk_esplora`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Leonardo Lima [Wed, 24 Apr 2024 21:12:45 +0000 (18:12 -0300)]
feat: update `wallet::Balance` to use `bitcoin::Amount`
- update all fields `immature`, ` trusted_pending`, `unstrusted_pending`
and `confirmed` to use the `bitcoin::Amount` instead of `u64`
- update all `impl Balance` methods to use `bitcoin::Amount`
- update all tests that relies on `keychain::Balance`
`Wallet` auto-implements `Send` and `Sync` after removing the generic. This test is a compile time error if there are changes to `Wallet` in the future that make it unsafe to send between threads. See #1387 for discussion.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
bdk_chain to 0.13.0
bdk_bitcoind_rpc to 0.9.0
bdk_electrum to 0.12.0
bdk_esplora to 0.12.0
bdk_file_store to 0.10.0
bdk_testenv to 0.3.0
bdk_persist to 0.2.0
### Checklists
#### All Submissions:
- [x] I've signed all my commits
- [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
- [x] I ran cargo fmt and cargo clippy before committing
bdk_chain to 0.13.0
bdk_bitcoind_rpc to 0.9.0
bdk_electrum to 0.12.0
bdk_esplora to 0.12.0
bdk_file_store to 0.10.0
bdk_testenv to 0.3.0
bdk_persist to 0.2.0
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Caught when trying to release (#1420), clippy failed randomly although it worked on master, this happened because rust 1.78 had just been release and we use clippy stable. IMHO we should pin the clippy version in CI and bump it manually at each new rust release.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Introduce universal structures that represent sync/full-scan requests/results.
### Notes to the reviewers
This is based on #1194 but is different in the following ways:
* The functionality to print scan/sync progress is not reduced.
* `SyncRequest` and `FullScanRequest` is simplified and fields are exposed for more flexibility.
### Changelog notice
* Add universal structures for initiating/receiving sync/full-scan requests/results for spk-based syncing.
* Updated `bdk_esplora` chain-source to make use of new universal sync/full-scan structures.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#1387 introduced `anyhow` as a dependency to remove generics from `Wallet`. Introducing a new crate for persistence types removes the dependency on `anyhow` for `bdk_chain`. Resolves #1409, as well as removing the old documentation for "tracker".
### Notes to the reviewers
Open for any comments.
### Changelog notice
- Introduce `bdk-persist` crate
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
feat(wallet): make wallet compatible with sync/full-scan structures
* Changed `Wallet::apply_update` to also take in anything that
implements `Into<Update>`. This allows us to directly apply a
`FullScanResult` or `SyncResult`.
* Added `start_full_scan` and `start_sync_with_revealed_spks` methods to
`Wallet`.
Co-authored-by: Steve Myers <steve@notmandatory.org>
Built on top of both #1369 and #1373, we simplify the `EsploraExt` API by removing the `update_local_chain` method and having `full_scan` and `sync` update the local chain in the same call. The `full_scan` and `sync` methods now takes in an additional input (`local_tip`) which provides us with the view of the `LocalChain` before the update. These methods now return structs `FullScanUpdate` and `SyncUpdate`.
The examples are updated to use this new API. `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` are no longer needed, therefore they are removed.
Additionally, we used this opportunity to simplify the logic which updates `LocalChain`. We got rid of the `local_chain::Update` struct (which contained the update `CheckPoint` tip and a `bool` which signaled whether we want to introduce blocks below point of agreement). It turns out we can use something like `CheckPoint::insert` so the chain source can craft an update based on the old tip. This way, we can make better use of `merge_chains`' optimization that compares the `Arc` pointers of the local and update chain (before we were crafting the update chain NOT based on top of the previous local chain). With this, we no longer need the `Update::introduce_older_block` field since the logic will naturally break when we reach a matching `Arc` pointer.
### Notes to the reviewers
* Obtaining the `LocalChain`'s update now happens within `EsploraExt::full_scan` and `EsploraExt::sync`. Creating the `LocalChain` update is now split into two methods (`fetch_latest_blocks` and `chain_update`) that are called before and after fetching transactions and anchors.
* We need to duplicate code for `bdk_esplora`. One for blocking and one for async.
### Changelog notice
* Changed `EsploraExt` API so that sync only requires one round of fetching data. The `local_chain_update` method is removed and the `local_tip` parameter is added to the `full_scan` and `sync` methods.
* Removed `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` methods.
* Introduced `CheckPoint::insert` which allows convenient checkpoint-insertion. This is intended for use by chain-sources when crafting an update.
* Refactored `merge_chains` to also return the resultant `CheckPoint` tip.
* Optimized the update `LocalChain` logic - use the update `CheckPoint` as the new `CheckPoint` tip when possible.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
The [`blocking` feature on the rust-esplora-client library](https://github.com/bitcoindevkit/rust-esplora-client/blame/master/Cargo.toml#L35) changed from ureq to minreq, which does not come with https enabled by default, breaking previously working code that simply enabled the `blocking` feature on the `bdk_esplora` crate.
This change will enable what is currently the "default https" [flag for the minreq library](https://docs.rs/minreq/latest/minreq/#https-or-https-rustls) when using the `blocking` feature on bdk_esplora, reverting that breaking change.
### Notes to the reviewers
Another way we could do this (let me know if this is preferable) is to add a new feature called `blocking-https-rustls`:
```rust
blocking = ["esplora-client/blocking"]
blocking-https-rustls = ["esplora-client/blocking-https-rustls"]
```
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
Improvements to the wallet address API, see commit messages for details.
### Notes to the reviewers
The logic of getting addresses is roughly the same as before when using `AddressIndex`, following this mapping:
- `New` -> `reveal_next_address`
- `LastUnused` -> `next_unused_address` (assuming this is what `LastUnused` really means)
- `Peek` -> `peek_address`
Wondering whether it makes sense to expose [`is_used`](https://github.com/bitcoindevkit/bdk/blob/358e842dcda0eb867d229881823c76dba1d7cce1/crates/chain/src/keychain/txout_index.rs#L236) for Wallet as well.
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
### Feature
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] This pull request breaks the existing API
* [x] I'm linking the issue being fixed by this PR
projects / bdk / log