feat(chain,wallet)!: Change persist-traits to be "safer"
Previously, `Persist{Async}With::persist` can be directly called as a
method on the type (i.e. `Wallet`). However, the `db: Db` (which is an
input) may not be initialized. We want a design which makes it harder
for the caller to make this mistake.
We change `Persist{Async}With::persist` to be an "associated function"
which takes two inputs: `db: &mut Db` and `changeset`. However, the
implementer cannot take directly from `Self` (as it's no longer an
input). So we introduce a new trait `Staged` which defines the staged
changeset type and a method that gives us a `&mut` of the staged
changes.
feat!: Rework sqlite, changesets, persistence and wallet-construction
Rework sqlite: Instead of only supported one schema (defined in
`bdk_sqlite`), we have a schema per changeset type for more flexiblity.
* rm `bdk_sqlite` crate (as we don't need `bdk_sqlite::Store` anymore).
* add `sqlite` feature on `bdk_chain` which adds methods on each
changeset type for initializing tables, loading the changeset and
writing.
Rework changesets: Some callers may want to use `KeychainTxOutIndex`
where `K` may change per descriptor on every run. So we only want to
persist the last revealed indices by `DescriptorId` (which uniquely-ish
identifies the descriptor).
* rm `keychain_added` field from `keychain_txout`'s changeset.
* Add `keychain_added` to `CombinedChangeSet` (which is renamed to
`WalletChangeSet`).
Rework persistence: add back some safety and convenience when persisting
our types. Working with changeset directly (as we were doing before) can
be cumbersome.
* Intoduce `struct Persisted<T>` which wraps a type `T` which stores
staged changes to it. This adds safety when creating and or loading
`T` from db.
* `struct Persisted<T>` methods, `create`, `load` and `persist`, are
avaliable if `trait PersistWith<Db>` is implemented for `T`. `Db`
represents the database connection and `PersistWith` should be
implemented per database-type.
* For async, we have `trait PersistedAsyncWith<Db>`.
* `Wallet` has impls of `PersistedWith<rusqlite::Connection>`,
`PersistedWith<rusqlite::Transaction>` and
`PersistedWith<bdk_file_store::Store>` by default.
Rework wallet-construction: Before, we had multiple methods for loading
and creating with different input-counts so it would be unwieldly to add
more parameters in the future. This also makes it difficult to impl
`PersistWith` (which has a single method for `load` that takes in
`PersistWith::LoadParams` and a single method for `create` that takes in
`PersistWith::CreateParams`).
* Introduce a builder pattern when constructing a `Wallet`. For loading
from persistence or `ChangeSet`, we have `LoadParams`. For creating a
new wallet, we have `CreateParams`.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
Fixes #980.
### Description
This PR is the first step in reworking `bdk_electrum` to use merkle proofs. When we fetch a transaction, we now also obtain the merkle proof and block header for verification. We then insert an anchor only after validation that the transaction exists in a confirmed block. The loop logic that previously existed in `full_scan` to account for re-orgs has also been removed as part of this rework.
This is a breaking change because `graph_update`s now provide the full `ConfirmationTimeHeightAnchor` type. This removes the need for the `ElectrumFullScanResult` and `ElectrumSyncResult` structs that existed only to provide the option for converting the anchor type from `ConfirmationHeightAnchor` into `ConfirmationTimeHeightAnchor`.
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
* `ConfirmationTimeHeightAnchor` and `ConfirmationHeightAnchor` have been removed.
* `ConfirmationBlockTime` has been introduced as a new anchor type.
* `bdk_electrum`'s `full_scan` and `sync` now return `graph_update`s with `ConfirmationBlockTime`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
Both `bdk_electrum` and `bdk_esplora` now report the exact block
that the transaction is in, which removes the need for having the
old `ConfirmationTimeHeightAnchor` and `ConfirmationHeightAnchor`.
This PR introduces a new, simpler anchor type that can be modified
to support additional data in the future.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
`cc` version 1.0.106 raised `msrv` to 1.67.
The previous working version 1.0.105 was pinned for CI to continue working.
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
* Pinned cc dependency version to build with rust 1.63.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Simplifies `public_descriptor` function by using `get_descriptor` and removes `get_descriptor_for_keychain`.
### Notes to the reviewers
Tested with `cargo test --all-features`.
### Changelog notice
- Simplifies `public_descriptor` function and removes `get_descriptor_for_keychain`
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
Fixes #1467.
### Description
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
Renames the `Append` trait to `Merge`.
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
* Rename `bdk_chain::Append` to `bdk_chain::Merge`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Resumes from the work in https://github.com/bitcoindevkit/bdk/pull/556.
Add custom sorting function for inputs and outputs through `TxOrdering::Custom` and deprecates `TxOrdering::Bip69Lexicographic`.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
I tried consider all discussions in https://github.com/bitcoindevkit/bdk/issues/534 while implementing some changes to the original PR. I created a summary of the considerations I had while implementing this:
##### Why use smart pointers?
The size of enums and structs should be known at compilation time. A struct whose fields implements some kind of trait cannot be specified without using a smart pointer because the size of the implementations of the trait cannot be known beforehand.
##### Why `Arc` or `Rc` instead of `Box`?
The majority of the useful smart pointers that I know (`Arc`, `Box`, `Rc`) for this case implement `Drop` which rules out the implementation of `Copy`, making harder to manipulate a simple enum like `TxOrdering`. `Clone` can be used instead, implemented by `Arc` and `Rc`, but not implemented by `Box`.
##### Why `Arc` instead of `Rc`?
Multi threading I guess.
##### Why using a type alias like `TxVecSort`?
cargo-clippy was accusing a too complex type if using the whole type inlined in the struct inside the enum.
##### Why `Fn` and not `FnMut`?
`FnMut` is not allowed inside `Arc`. I think this is due to the `&mut self` ocupies the first parameter of the `call` method when desugared (https://rustyyato.github.io/rust/syntactic/sugar/2019/01/17/Closures-Magic-Functions.html), which doesn't respects `Arc` limitation of not having mutable references to data stored inside `Arc`:
Quoting the [docs](https://doc.rust-lang.org/std/sync/struct.Arc.html):
> you cannot generally obtain a mutable reference to something inside an `Arc`.
`FnOnce` > `FnMut` > `Fn`, where `>` stands for "is supertrait of", so, `Fn` can be used everywhere `FnMut` is expected.
##### Why not `&'a dyn FnMut`?
It needs to include a lifetime parameter in `TxOrdering`, which will force the addition of a lifetime parameter in `TxParams`, which will require the addition of a lifetime parameter in a lot of places more. **Which one is preferable?**
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
- Adds new `TxOrdering` variant: `TxOrdering::Custom`. A structure that stores the ordering functions to sort the inputs and outputs of a transaction.
- Deprecates `TxOrdering::Bip69Lexicographic`.
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [ ] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [ ] I've added docs for the new feature
BIP 69 proposed a deterministic way to sort transaction inputs and
outputs with the idea of enhancing privacy. It was later discovered
there was no such enhancement but rather a decrement in privacy due to
this sorting.
To avoid the promotion of bad practices, the
TxOrdering::Bip69Lexicographic variant which implemented this BIP for
BDK is removed with this change.
Notice that you can still produce a BIP 69 compliant transaction
defining order functions for TxOrdering::Custom.
Signed-off-by: Steve Myers <steve@notmandatory.org>
FadedCoder [Sun, 27 Feb 2022 05:49:53 +0000 (11:19 +0530)]
feat(wallet): add TxOrdering::Custom
The deterministic sorting of transaction inputs and outputs proposed in
BIP 69 doesn't improve the privacy of transactions as originally
intended. In the search of not spreading bad practices but still provide
flexibility for possible use cases depending on particular order of the
inputs/outpus of a transaction, a new TxOrdering variant has been added
to allow the implementation of these orders through the definition of
comparison functions.
Signed-off-by: Steve Myers <steve@notmandatory.org>
The PR changes the type of last_seen to `Option<u64>` for `txs` member of `TxGraph`.
This fixes an issue where unbroadcast and otherwise non-canonical transactions were returned from methods `list_chain_txs` and `Wallet::transactions` because every new tx inserted had a last_seen of 0 making it appear unconfirmed.
fixes #1446
fixes #1396
### Notes to the reviewers
### Changelog notice
Changed
- Member `last_seen_unconfirmed` of `TxNode` is changed to `Option<u64>`
- Renamed `TxGraph` method `list_chain_txs` to `list_canonical_txs`
- Changed `Wallet::insert_tx` to take a single `tx: Transaction` as parameter
Added
- Add method `txs_with_no_anchor_or_last_seen` for `TxGraph`
- Add method `unbroadcast_transactions` for `Wallet`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
valued mammal [Thu, 23 May 2024 21:33:45 +0000 (17:33 -0400)]
fix(tx_graph)!: Change tx_last_seen to `Option<u64>`
Also fixup `test_list_owned_txouts` to check that the right
outputs, utxos, and balance are returned at different local
chain heights.
This fixes an issue where unbroadcast and otherwise non-canonical
transactions were returned from methods `list_chain_txs` and
`Wallet::transactions` because every tx inserted had a last_seen
of 0 making it appear unconfirmed.
Note this commit changes the way `Balance` is represented due to
new logic in `try_get_chain_position` that no longer considers
txs with non-canonical anchors. Before this change, a tx anchored
to a block that is reorged out had a permanent effect on the
pending balance, and now only txs with a last_seen time or an
anchor confirmed in the best chain will return a `ChainPosition`.
Rename `DescriptorId` to `KeychainId` and `descriptor_id()` to `keychain_id()`.
Calculate keychain ids as the hash of the spk derived from its descriptor as index 0.
Added docs to `Wallet` and `KeychainTxOutIndex::insert_descriptor()` explaining that it's the users responsibility not to use wildcard and non-wildcard descriptors that can derive the same script pubkey. I also recommended for `Wallet` that legacy non-wildcard descriptors be added to a temporary `Wallet` and swept into a modern wildcard descriptor.
### Notes to the reviewers
fixes #1483
### Changelog notice
changed
- Renamed DescriptorId to KeychainId, DescriptorExt::descriptor_id() to keychain_id().
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
In `rust-bitcoin` the `blockdata` module is a code organisation thing, it should never have been public. One day those guys would like to remove it, so as not to be a PITA for `bdk` when they do lets remove all usage of `blockdata::` now.
### Changelog notice
Internal change only, no externally visible changes.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Tobin C. Harding [Wed, 26 Jun 2024 04:33:00 +0000 (14:33 +1000)]
Remove usage of blockdata:: from bitcoin paths
In `rust-bitcoin` the `blockdata` module is a code organisation thing,
it should never have been public. One day those guys would like to
remove it, so as not to be a PITA for `bdk` when they do lets remove all
usage of `blockdata::` now.
Internal change only, no externally visible changes.
fixes #1466
depends on #1448
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
This PR is a follow-up on top of #1448, and should be rebased and merged after it, it uses the rust-bitcoin `Weight` type instead of the current `usize` usage.
NOTE: ~~It also adds a new `MiniscriptError` variant, and remove the `.unwrap()` usage.~~ As suggested I'll address this on another issue #1485, trying to reproduce the error first.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
It should be ready to review after #1448 gets merged.
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
- Change `WeightedUtxo` `satisfaction_weight` has been changed to use `Weight` type.
- Change `TxBuilder` methods `add_foreign_utxo` and `add_foreign_utxo_with_sequence` to expect `Weight` as `satisfaction_weight` type.
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
This PR does some cleanup of the `bdk_wallet` signer module most notably by removing the internal trait `ComputeSighash` and replacing old code for computing the sighash (for legacy and segwit context) with a single method [`Psbt::sighash_ecdsa`](https://docs.rs/bitcoin/0.31.2/bitcoin/psbt/struct.Psbt.html#method.sighash_ecdsa). The logic for computing the taproot sighash is unchanged and extracted to a new helper function `compute_tap_sighash`.
- [x] Unimplement `ComputeSighash`
- [x] Try de-duplicating code by using `Psbt::sighash_ecdsa`. see https://github.com/bitcoindevkit/bdk/pull/1023#discussion_r1263140218
- Not done in this PR: Consider removing unused `SignerError` variants
fixes #1038
### Notes to the reviewers
### Changelog notice
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
valued mammal [Fri, 24 May 2024 23:20:31 +0000 (19:20 -0400)]
ref(signer): Use `Psbt::sighash_ecdsa` for computing sighashes
- Change param `hash` to `&Message` in `sign_psbt_ecdsa`
- Remove unused methods `compute_legacy_sighash`,
and `compute_segwitv0_sighash`.
- Match on `self.ctx` when signing for `SignerWrapper<PrivateKey>`
valued mammal [Mon, 20 May 2024 11:01:05 +0000 (07:01 -0400)]
fix(wallet)!: Rework `Wallet::insert_tx` to no longer insert anchors
since we'd be lacking context that should normally occur during
sync with a chain source. The logic for inserting a graph
anchor from a `ConfirmationTime` is moved to the wallet common
test module in order to simulate receiving new txs and
confirming them.
The `rand` dependency was imported explicitly, but `rand` is also implicitly used through the `rand-std` feature flag on `bitcoin`.
### Notes to he reviewers
**Updated:**
`rand` was used primarily in two parts of `bdk`. Particularly in signing and in building a transaction.
Signing:
- Used implicitly in [`sign_schnorr`](https://docs.rs/bitcoin/latest/bitcoin/key/struct.Secp256k1.html#method.sign_schnorr), but nowhere else within `signer`.
Transaction ordering:
- Used to shuffle the inputs and outputs of a transaction, the default
- Used in the single random draw __as a fallback__ to branch and bound during coin selection. Branch and bound is the default coin selection option.
See conversation for proposed solutions.
### Changelog notice
- Remove the `rand` dependency from `bdk`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Rather than comingle various `SignOptions` with the finalization step, we simply clear all fields when finalizing as per the PSBT spec in BIPs 174 and 371 which is more in line with user expectations.
### Notes to the reviewers
I chose to re-implement some parts of [`finalize_input`](https://docs.rs/miniscript/latest/src/miniscript/psbt/finalizer.rs.html#434) since it's fairly straightforward, see also https://github.com/bitcoindevkit/bdk/issues/1461#issuecomment-2171983426. I had to refactor some wallet tests but didn't go out of my way to add additional tests.
closes #1461
### Changelog notice
- Removed field `remove_partial_sigs` from `SignOptions`
- Removed field `remove_taproot_extras` from `SignOptions`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
valued mammal [Tue, 11 Jun 2024 21:56:31 +0000 (17:56 -0400)]
fix(wallet)!: Simplify `SignOptions` and improve finalization logic
Rather than comingle various `SignOptions` with the finalization
step, we simply clear all fields when finalizing as per the PSBT
spec in BIPs 174 and 371 which is more in line with user
expectations.
bdk_chain to 0.16.0
bdk_bitcoind_rpc to 0.12.0
bdk_electrum to 0.15.0
bdk_esplora to 0.15.0
bdk_file_store to 0.13.0
bdk_sqlite keep at 0.2.0
bdk_testenv to 0.6.0
bdk_hwi to 0.3.0
Steve Myers [Fri, 14 Jun 2024 02:20:05 +0000 (21:20 -0500)]
Bump bdk version to 1.0.0-alpha.13
bdk_chain to 0.16.0
bdk_bitcoind_rpc to 0.12.0
bdk_electrum to 0.15.0
bdk_esplora to 0.15.0
bdk_file_store to 0.13.0
bdk_sqlite keep at 0.2.0
bdk_testenv to 0.6.0
bdk_hwi to 0.3.0
@LLFourn suggested these changes which greatly simplifies the amount of code we have to maintain and removes the `async-trait` dependency. We remove `PersistBackend`, `PersistBackendAsync`, `StageExt` and `StageExtAsync` completely. Instead, we introduce `Wallet::take_staged(&mut self) -> Option<ChangeSet>`.
The original intention to keep a staging area (`StageExt`, `StageExtAsync`) is to enforce:
1. The caller will not persist an empty changeset.
2. The caller does not take the staged changeset if the database (`PersistBackend`) fails.
We achieve `1.` by returning `None` if the staged changeset is empty.
`2.` is not too important. The caller can try figure out what to do with the changeset if persisting to db fails.
### Notes to the reviewers
I added a `take` convenience method to the `Append` trait. I thought it would be handy for the caller to do a staging area with this.
### Changelog notice
* Remove `persist` submodule from `bdk_chain`.
* Change `Wallet` to outsource it's persistence logic by introducing `Wallet::take_staged`.
* Add `take` convenience method to `Append` trait.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
~* [ ] I've added tests for the new feature~
* [x] I've added docs for the new feature
志宇 [Fri, 14 Jun 2024 12:42:25 +0000 (20:42 +0800)]
feat!: rm `persist` submodule
Remove `PersistBackend`, `PersistBackendAsync`, `StageExt` and
`StageExtAsync`. Remove `async` feature flag and dependency. Update
examples and wallet.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
It fixes the typo on the `expect()` message introduced on #1426, noticed at https://github.com/bitcoindevkit/bdk/pull/1426#discussion_r1626761825
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Sorry to submit another refactor PR for the persist related stuff, but I think it's worth revisiting. My primary motivations are:
1. remove `db` from `Wallet` so users have the ability to use `async` storage crates, for example using `sqlx`. I updated docs and examples to let users know they are responsible for persisting changes.
2. remove the `anyhow` dependency everywhere (except as a dev test dependency). It really doesn't belong in a lib and by removing persistence from `Wallet` it isn't needed.
3. remove the `bdk_persist` crate and revert back to the original design with generic error types. I kept the `Debug` and `Display` constrains on persist errors so they could still be used with the `anyhow!` macro.
### Notes to the reviewers
I also replaced/renamed old `Persist` with `StagedPersist` struct inspired by #1453, it is only used in examples. The `Wallet` handles it's own staging.
### Changelog notice
Changed
- Removed `db` from `Wallet`, users are now responsible for persisting changes, see docs and examples.
- Removed the `bdk_persist` crate and moved logic back to `bdk_chain::persist` module.
- Renamed `PersistBackend` trait to `Persist`
- Replaced `Persist` struct with `StagedPersist`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
This reverts part of the changes in #1203. There the `SpkTxOutIndex<(K,u32)>` was changed to `SpkTxOutIndex<(DescriptorId, u32>)`. This led to a complicated translation logic in `KeychainTxOutIndex` (where the API is based on `K`) to transform calls to it to calls to the underlying `SpkTxOutIndex` (which now indexes by `DescriptorId`). The translation layer was broken when it came to translating range queries from the `KeychainTxOutIndex`. My solution was just to revert this part of the change and remove the need for a translation layer (almost) altogether. A thin translation layer remains to ensure that un-revealed spks are filtered out before being returned from the `KeychainTxOutIndex` methods.
I feel like this PR could be extended to include a bunch of ergonomics improvements that are easier to implement now. But I think that's the point of https://github.com/bitcoindevkit/bdk/pull/1451 so I held off and should probably go and scope creep that one instead.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
The previous commit b9c5b9d08b040faf6c6b2d9b3745918031555b72 added
IndexSpk. This goes further and adds `Indexed` and `KeychainIndexed`
type alises (IndexSpk is Indexed<ScriptBuf>) and attempts to standardize
the structure of return types more generally.
LLFourn [Thu, 6 Jun 2024 00:17:55 +0000 (10:17 +1000)]
refactor(keychain): Fix KeychainTxOutIndex range queries
The underlying SpkTxOutIndex should not use DescriptorIds to index
because this loses the ordering relationship of the spks so queries on
subranges of keychains work.
Along with that we enforce that there is a strict 1-to-1 relationship
between descriptors and keychains. Violating this leads to an error in
insert_descriptor now.
In general I try to make the translation layer between the SpkTxOutIndex
and the KeychainTxOutIndex thinner. Ergonomics of this will be improved
in next commit.
fixes #1422
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
This PR focuses on upgrading the `rust-bitcoin` and `miniscript` versions, to `0.32.0` and `0.12.0`, respectively. It also bumps the versions of other BDK ecosystem crates that also rely on both `rust-bitcoin` and `miniscript`, being:
<ins>I structured the PR in multiple commits, with closed scope, being one for each BDK crate being upgraded, and one for each kind of fix and upgrade required, it seems like a lot of commits (**that should be squashed before merging**), but I think it'll make it easier during review phase.</ins>
In summary I can already mention some of the main changes:
- using `compute_txid()` instead of deprecated `txid()`
- using `minimal_non_dust()` instead of `dust_value()`
- using the renamed `signature` and `sighash_type` fields
- using proper `sighash::P2wpkhError`, `sighash::TaprootError` instead of older `sighash::Error`
- conversion from `Network` to new expected `NetworkKind` #1465
- conversion from the new `Weight` type to current expected `usize` #1466
- using `.into()` to convert from AbsLockTime and `RelLockTime` to `absolute::LockTime` and `relative::LockTime`
- using Message::from_digest() instead of relying on deprecated `ThirtyTwoByteHash` trait.
- updating the miniscript policy and dsl to proper expect and handle new `Threshold` type, instead of the previous two parameters.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
<ins>Again, I structured the PR in multiple commits, with closed scope, being one for each BDK crate being upgraded, and one for each kind of fix and upgrade required, it seems like a lot of commits (**that should be squashed before merging**), but I think it'll make it easier during review phase.</ins>
It should definitely be carefully reviewed, especially the last commits for the wallet crate scope, the ones with the semantic `fix(wallet)`.
I would also appreciate if @tcharding reviewed it, as he gave a try in the past (#1400 ), and I relied on some of it for the policy part of it, other rust-bitcoin maintainers reviews are a definitely welcome 😁
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
> // TODO(@oleonardolima): Do another pass and double check the changes
- Use `compute_txid()` instead of deprecated `txid()`
- Use `minimal_non_dust()` instead of `dust_value()`
- Use `signature` and `sighash_type` fields, instead of previous `sig` and `hash_type`
- Use `sighash::P2wpkhError`, and `sighash::TaprootError` instead of older `sighash::Error`
- Converts from `Network` to `NetworkKind`, where expected
- Converts from `Weight` type to current used `usize`
- Use `.into()` to convert from `AbsLockTime` and `RelLockTime` to `absolute::LockTime` and `relative::LockTime`
- Remove use of deprecated `ThirtyTwoByteHash` trait, use `Message::from_digest()`
- Update the miniscript policy and dsl macros to proper expect and handle new `Threshold` type, instead of the previous two parameters.
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
志宇 [Thu, 13 Jun 2024 03:47:01 +0000 (11:47 +0800)]
chore(wallet): rm dup code
Methods `make_multi` and `make_multi_a` were essentially the same thing
except for a different const generic param on `Threshold`. So we rm
`make_multi_a` and put a const generic param on `make_multi`.
Leonardo Lima [Wed, 22 May 2024 21:34:30 +0000 (18:34 -0300)]
deps(bdk): bump `bitcoin` to `0.32.0`, miniscript to `12.0.0`
deps(chain): bump `bitcoin` to `0.32.0`, miniscript to `12.0.0`
fix(chain): use `minimal_non_dust()` instead of `dust_value()`
fix(chain): use `compute_txid()` instead of `txid`
deps(testenv): bump `electrsd` to `0.28.0`
deps(electrum): bump `electrum-client` to `0.20.0`
fix(electrum): use `compute_txid()` instead of `txid`
deps(esplora): bump `esplora-client` to `0.8.0`
deps(bitcoind_rpc): bump `bitcoin` to `0.32.0`, `bitcoincore-rpc` to
`0.19.0`
fix(bitcoind_rpc): use `compute_txid()` instead of `txid`
fix(nursery/tmp_plan): use proper `sighash` errors, and fix the expected
`Signature` fields
fix(sqlite): use `compute_txid()` instead of `txid`
deps(hwi): bump `hwi` to `0.9.0`
deps(wallet): bump `bitcoin` to `0.32.0`, miniscript to `12.0.0`
fix(wallet): use `compute_txid()` and `minimal_non_dust()`
- update to use `compute_txid()` instead of deprecated `txid()`
- update to use `minimal_non_dust()` instead of `dust_value()`
- remove unused `bitcoin::hex::FromHex`.
fix(wallet): uses `.into` conversion on `Network` for `NetworkKind`
- uses `.into()` when appropriate, otherwise use the explicit
`NetworkKind`, and it's `.is_mainnet()` method.
fix(wallet): add P2wpkh, Taproot, InputsIndex errors to `SignerError`
fix(wallet): fields on taproot, and ecdsa `Signature` structure
fix(wallet/wallet): convert `Weight` to `usize` for now
- converts the `bitcoin-units::Weight` type to `usize` with help of
`to_wu()` method.
- it should be updated/refactored in the future to handle the `Weight`
type throughout the code instead of current `usize`, only converting
it for now.
- allows the usage of deprecated `is_provably_unspendable()`, needs
further discussion if suggested `is_op_return` is suitable.
- update the expect field to `signature`, as it was renamed from `sig`.
fix(wallet/wallet): use `is_op_return` instead of
`is_provably_unspendable`
fix(wallet/wallet): use `relative::Locktime` instead of `Sequence`
fix(wallet/descriptor): use `ParsePublicKeyError`
fix(wallet/descriptor): use `.into()` to convert from `AbsLockTime` and
`RelLockTime` to `absolute::LockTime` and `relative::LockTime`
fix(wallet/wallet): use `Message::from_digest()` instead of relying on
deprecated `ThirtyTwoByteHash` trait.
fix(wallet/descriptor+wallet): expect `Threshold` type, and handle it
internally
fix(wallet/wallet): remove `0x` prefix from expected `TxId` display
fix(examples): use `compute_txid()` instead of `txid`
fix(ci): remove usage of `bitcoin/no-std` feature
- remove comment: `# The `no-std` feature it's implied when the `std` feature is disabled.`
- Replace `CreateTxError::InsufficientFunds` use by `coin_selection::Error::InsufficientFunds`
- Remove `InsufficientFunds` member from `CreateTxError` enum
- Rename `coin_selection::Error` to `coin_selection::CoinSelectionError`
### Notes to the reviewers
- We could also keep both members but rename one of them to avoid confusion
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
All `Wallet` constructors are modified to require a change descriptor, where previously it was optional. Additionally we enforce uniqueness of the change descriptor to avoid ambiguity when deriving scripts and ensure the wallet will always have two distinct keystores.
Notable changes
* Add error `DescriptorError::ExternalAndInternalAreTheSame`
* Remove error `CreateTxError::ChangePolicyDescriptor`
* No longer rely on `map_keychain`
fixes #1383
### Notes to the reviewers
### Changelog notice
Changed:
Constructing a Wallet now requires two distinct descriptors.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
valued mammal [Wed, 27 Mar 2024 01:57:10 +0000 (21:57 -0400)]
refactor(wallet)!: Make Wallet require a change descriptor
All `Wallet` constructors are modified to require a change
descriptor, where previously it was optional. Additionally
we enforce uniqueness of the change descriptor to avoid
ambiguity when deriving scripts and ensure the wallet will
always have two distinct keystores.
Notable changes
* Add error DescriptorError::ExternalAndInternalAreTheSame
* Remove error CreateTxError::ChangePolicyDescriptor
* No longer rely on `map_keychain`
Previously there was a `TxCache` that you passed in as part of the sync request. There are lots of downsides to this:
1. If the user forgets to do this you cache nothing
2. where are you meant to keep this cache? The example shows it being recreated every time which seems very suboptimal.
3. More API and documentation surface area.
Instead just do a plain old simple cache inside the electrum client. This way at least you only download transactions once. You can pre-populate the cache with a method also and I did this in the examples.
- Wallet::get_balance() renamed to Wallet::balance()
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
It further updates and adds the usage of `bitcoin::Amount` instead of `u64`.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
Open for comments and discussions.
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
- Updated `CreateTxError::FeeTooLow` to use `bitcoin::Amount`.
- Updated `Wallet::calculate_fee()`. to use `bitcoin::Amount`
- Updated `TxBuilder::fee_absolute()`. to use `bitcoin::Amount`.
- Updated `CalculateFeeError::NegativeFee` to use `bitcoin::SignedAmount`.
- Updated `TxGraph::calculate_fee()`. to use `bitcoin::Amount`.
- Updated `PsbUtils::fee_amount()` to use `bitcoin::Amount`.
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
LLFourn [Fri, 31 May 2024 03:52:49 +0000 (13:52 +1000)]
refactor(electrum)!: put the tx cache in electrum
Previously there was a tx cache that you passed in as part of the sync
request. This seems bad and the example show'd that you should copy all
your transactions from the transaction graph into the sync request every
time you sync'd. If you forgot to do this then you would always download everything.
Instead just do a plain old simple cache inside the electrum client.
This way at least you only download transactions once. You can
pre-populate the cache with a method also and I did this in the examples.
Leonardo Lima [Sun, 5 May 2024 20:41:31 +0000 (17:41 -0300)]
feat: use `Amount` on `calculate_fee`, `fee_absolute`, `fee_amount` and others
- update to use `bitcoin::Amount` on `CreateTxError::FeeTooLow` variant.
- update to use `bitcoin::Amount` on `Wallet::calculate_fee()`.
- update to use `bitcoin::Amount` on `FeePolicy::fee_absolute()`.
- update to use `bitcoin::SignedAmount` on
`CalculateFeeError::NegativeFee` variant.
- update to use `bitcoin::Amount` on `TxGraph::calculate_fee()`.
- update to use `bitcoin::Amount` on `PsbUtils::fee_amount()`
projects / bdk / log