We would previously calculate the fee amount in two steps:
1. Add the weight of the empty transaction
2. Add the weight of each output
That's unnecessary: you can just use the weight of the transaction
*after* the output addition. This is clearer, but also avoids a
rare bug: if there are many outputs, adding them would cause the
"number of outputs" transaction parameter lenght to increase, and we
wouldn't notice it.
This might still happen when adding the drain output - this
commit also adds a comment as a reminder.
Currently SqliteDatabase::new takes a String as path,
with this change, it now accepts any type that implements
AsRef<Path>.
### Notes to the reviewers
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
The former way to compute and create change was inside `create_tx`, just after
performing coin selection.
It blocked the opportunity to have an "ensemble" algorithm to decide between
multiple coin selection algorithms based on a metric, like Waste.
Now, change is not created inside `coin_select` but the change amount and the
possibility to create change is decided inside the `coin_select` method. In
this way, change is associated with the coin selection algorithm that generated
it, and a method to decide between them can be implemented.
Fixes #147.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
The former way to compute and create change was inside `create_tx`, just after
performing coin selection.
It blocked the opportunity to have an "ensemble" algorithm to decide between
multiple coin selection algorithms based on a metric, like Waste.
Now, change isn't created inside `coin_select` but the change amount and the
possibility to create change is decided inside the `coin_select` method. In
this way, change is associated with the coin selection algorithm that generated
it, and a method to decide between them can be implemented.
^ However, only for electrum-based `Blockchain` implementations. For RPC and Compact Block Filters, syncing works differently, and so are the bugs - I have created a separate ticket for this (#677).
### Description
Previously, electrum-based blockchain implementations only synced for `scriptPubKey`s that are already cached in `Database`.
This PR introduces a feedback mechanism, that uses `stop_gap` and the difference between "current index" and "last active index" to determine whether we need to cache more `scriptPubKeys`.
The `WalletSync::wallet_setup` trait now may return an `Error::MissingCachedScripts` error which contains the number of extra `scriptPubKey`s to cache, in order to satisfy `stop_gap` for the next call.
`Wallet::sync` now calls `WalletSync` in a loop, caching in-between subsequent calls (if needed).
#### Notes to reviewers
1. The caveat to this solution is that it is not the most efficient. Every call to `WalletSync::wallet_setup` starts polling the Electrum-based server for `scriptPubKey`s starting from index 0.
However, I feel like this solution is the least "destructive" to the API of `Blockchain`. Also, once the `bdk_core` sync logic is integration, we can select specific ranges of `scriptPubKey`s to sync.
2. Also note that this PR only fixes the issue for electrum-based `Blockchain` implementations (currently `blockchain::electrum` and `blockchain::esplora` only).
3. Another thing to note is that, although `Database` assumes 1-2 keychains, the current `WalletSync` "feedback" only returns one number (which is interpreted as the larger "missing count" of the two keychains). This is done for simplicity, and because we are planning to only have one keychain per database in the future.
4. Please have a read of https://github.com/bitcoindevkit/bdk/pull/672#issuecomment-1186929465 for additional context.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Fix wallet sync not finding coins of addresses which are not cached
Previously, electrum-based blockchain implementations only synced for
`scriptPubKey`s that are already cached in `Database`.
This PR introduces a feedback mechanism, that uses `stop_gap` and the
difference between "current index" and "last active index" to determine
whether we need to cache more `scriptPubKeys`.
The `WalletSync::wallet_setup` trait now may return an
`Error::MissingCachedScripts` error which contains the number of extra
`scriptPubKey`s to cache, in order to satisfy `stop_gap` for the next call.
`Wallet::sync` now calls `WalletSync` in a loop, cacheing inbetween
subsequent calls (if needed).
This test is to ensure there are no regressions when we later change
internal logic of `Wallet`. A single descriptor wallet should always get
a new address with `AddressIndex::New` even if we alternate grabbing
internal/external keychains.
I thought of adding this during work on #647
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
### Description
Seems like `doc(include = "../README.md")` doesn't include the readme file as doc for the dummy struct. This might be due to a difference in Rust edition used back then or something.
Fixes #637
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
`get_checksum_bytes()` returns a descriptor checksum as `[u8; 8]` instead of `String`, potentially improving performance and memory usage.
In addition to this, since descriptors only use characters that fit within a UTF-8 8-bit code unit ([US-ASCII](https://www.charset.org/charsets/us-ascii)), there is no need to use the `char` type (which is 4 bytes). This can also potentially bring in some performance and memory-usage benefits.
### Notes to the reviewers
This is useful because we will be using descriptor checksums for indexing operations in the near future (multi-descriptor wallets #486 ).
Refer to comments by @afilini :
* https://github.com/bitcoindevkit/bdk/pull/647#discussion_r921184366
* https://github.com/bitcoindevkit/bdk/pull/647#discussion_r921914696
* https://github.com/bitcoindevkit/bdk/pull/654#discussion_r921980876
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
Introduce `get_checksum_bytes` method and improvements
`get_checksum_bytes` returns a descriptor checksum as `[u8; 8]` instead
of `String`, potentially improving performance and memory usage.
In addition to this, since descriptors only use charaters that fit
within a UTF-8 8-bit code unit, there is no need to use the `char` type
(which is 4 bytes). This can also potentially bring in some performance
and memory-usage benefits.
Seems like `doc(include = "../README.md")` doesn't include the readme file as docs for the dummy struct. This might be due to a difference in Rust edition used back then or something
📢 Release 0.20.0 is out! Highlights include bug fixes for the ElectrumBlockchain and descriptor templates, discourage fee sniping in tx building, and new tx signing options. A big thanks to our past and latest new contributors. For all changes see: https://github.com/bitcoindevkit/bdk/releases/tag/v0.20.0
Usually we don't have any prefix except for methods that can *add* to a list or replace the list entirely (e.g. `add_recipients` vs `set_recipients`)
I missed this during review of #611
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
It is critical to ensure `Wallet::get_address` with `AddressIndex::new` always returns a new and unused address.
This bug seems to be Electrum-specific. The fix is to check address index updates to ensure that newly suggested indexes are not smaller than indexes already in database.
### Notes to the reviewers
I have written new tests in `/testutils/blockchain_tests.rs` that tests all `Blockchain` implementations.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
~* [ ] This pull request breaks the existing API~
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
This bug seems to be Electrum-specific. The fix is to check the
proposed changes against the current state of the database. Ensure
newly suggested indexes are not smaller than indexes already in
database.
Changes:
* Check index updates before they are applied to database during
Electrum Blockchain sync (Thank you @rajarshimaitra for providing
an elegant solution).
> address validators are supposed to be used for a slightly different thing, which is when you ask the hardware wallet to independently generate the address for a derivation index and then you compare what you see on your computer/phone with what the hardware wallet is displaying
> in the case of change addresses i agree that it's not as important (because as you said the device can just refuse to sign) but for consistency we implemented it for both external and internal addresses
> more broadly, they can be thought of as a way to get a callback every time an address is generated, which may also be useful for other things (for example when i was working on a green-compatible client written in bdk i used that feature to ping the server every time a new address was generated, because that's required in their protocol)
> that said, i think currently pretty much nobody uses them and i am myself moving away from the concept that "everything needs to happen inside bdk": currently my mindset is targeted more towards reducing complexity by breaking down individual parts and wrapping them or making them "extensible" in some way
> that is to say: if you want to verify addresses in your hardware wallet you don't necessarily need bdk to do it for you (actually, you would still have to implement the callback manually), you can just call bdk to get a new addr and then ping the device yourself. and this would allow us to reduce complexity and delete some code
> actually, here's an idea: unless somebody here is opposed to this, i can make a pr to deprecate address validators in the next (0.20) release. if after that again nobody complains we can completely remove them and point users towards different strategies to achieve the same goal
### Checklists
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
* [x] I've updated `CHANGELOG.md`
This test is to ensure there are no regressions when we later change
internal logic of `Wallet`. A single descriptor wallet should always get
a new address with `AddressIndex::New` even if we alternate grabbing
internal/external keychains.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
This PR is to add 2 keys(`try_finalize` and `remove_partial_sigs`) in `SignOptions`. See this issue for detail https://github.com/bitcoindevkit/bdk/issues/612
### Notes to the reviewers
~I found the negative naming of these 2 new keys `do_not_finalize` and `do_not_remove_partial_sigs` are a bit confusing(like most negative named paremeter/variable). Should we actually change it back to positive naming(`do_finalize` and `do_remove_partial_sigs`)?~
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
### Description
This PR create a new trait `blockchain::GetBlockHash` with a `get_block_hash` method which returns a block hash given the block height. This has been implemented for all blockchain backends.
Fixes #603
### Notes to the reviewers
I haven't updated the `CHANGELOG.md` and docs. Am I suppose to update it for this change?
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [ ] I've added docs for the new feature
* [ ] I've updated `CHANGELOG.md`
Vladimir Fomene [Thu, 16 Jun 2022 19:42:02 +0000 (20:42 +0100)]
Get block hash by its height
Create blockchain::GetBlockHash trait
with a method to get block hash given
a block height. Then, implement this
trait for all backends (Electrum, RPC
, Esplora, CBF). Referenced in issue 603.
#144 is describing a bug that doesn't seem to happen in BDK master anymore (BDK not respecting BIP125 rule 2). This PR just adds a test to check that the bug is fixed.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Daniela Brozzoni [Thu, 30 Jun 2022 18:31:38 +0000 (20:31 +0200)]
Remove wrong TODO comment in build_fee_bump
The proposed solution is bad for privacy as well.
Let's call the initial change output, which is normally shrink when you
fee bump, change#1, and the extra output aforementioned change#2 (as,
in this case, it's going to be a change output as well). If you add change#2
you might not revel change#1, but you're still revealing change#2.
You're not improving your privacy, and you're wasting money in fees.
With this PR we start considering how many confirmations a coinbase has. If it's not mature yet, we don't use it for building transactions.
Fixes #413
### Notes to the reviewers
This PR is based on #611, review that one before reviewing this 😄
007c5a78335a3e9f6c9c28a077793c2ba34bbb4e adds a coinbase parameter to `populate_test_db`, to specify if you want the db to be populated with immature coins. This is useful for `test_spend_coinbase`, but that's probably going to be the only use case.
I don't think it's a big deal to have a test function take an almost_always_useless parameter - it's not an exposed API, anyways. But, if you can come up with a different way of implementing `test_spend_coinbase` that doesn't require 007c5a78335a3e9f6c9c28a077793c2ba34bbb4e, even better! I looked for it for a while, but other than duplicating the whole `populate_test_db` code, which made the test way harder to comprehend, I didn't find any other way.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
`Wallet::sync` hangs indefinitely when syncing with Electrum with `stop_gap` set as 0.
The culprit is having `chunk_size` set as `stop_gap`. A zero value results in syncing not being able to progress.
Fixes #651
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
~* [ ] This pull request breaks the existing API~
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
This is a continuation of the #651 fix. We should also check whether the
same bug affects esplora as noted by @afilini. To achieve this, I've
introduced a `ConfigurableBlockchainTester` trait that can test multiple
blockchain implementations.
* Introduce `ConfigurableBlockchainTester` trait.
* Use the aforementioned trait to also test esplora.
* Change the electrum test to also use the new trait.
* Fix some complaints by clippy in ureq.rs file (why is CI not seeing
this?).
* Refactor some code.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
This PR is to remove Database::flush. See this issue for detail https://github.com/bitcoindevkit/bdk/issues/567
### Notes to the reviewers
The 2nd commit is a small refactoring of adding a new private ivec_to_u32 to avoid too much code duplication. Please let me know if it's ok to include this in this PR or I should make it into a separate PR
Currently existing test cases are shared across for all Databaes implementation so I am not sure if we should add specific test cases for keyvalue(Tree) for this auto-flush behaviour?(and I feel like it's more a implementation detail). Please let me know how should I proceed for test case in this PR
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
### Description
Before this commit, you could create a transaction with `drain_to` set
without specifying recipients, nor `drain_wallet`, nor `utxos`. What would
happen is that BDK would pick one input from the wallet and send
that one to `drain_to`, which is quite weird.
This PR restricts the usage of `drain_to`: if you want to use it as a
change output, you need to set recipients as well. If you want to send
a specific utxo to the `drain_to` address, you specify it through
`add_utxos`. If you want to drain the whole wallet, you set
`drain_wallet`. In any other case, if `drain_to` is set, we return a
`NoRecipients` error.
Fixes #620
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API - kinda?
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Daniela Brozzoni [Wed, 25 May 2022 17:54:40 +0000 (18:54 +0100)]
populate_test_db accepts a `coinbase` param
Allows user to ask for a test db populated with clean coins
from coinbases. This is useful for testing the wallet behaviour
when some inputs are coinbases.
### Description
By default bdk sets the transaction's nLockTime to current_height
to prevent fee sniping.
current_height can be provided by the user through TxParams; if the user
didn't provide it, we use the last sync height, or 0 if we never synced.
If you want to know more about fee sniping: https://bitcoinops.org/en/topics/fee-sniping/
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
I'm not sure if this is needed or helpful, but this PR adds comments to describe how the `sent` and `received` fields of `TransactionDetails` are calculated.
I wasn't sure how it was done until I looked deeper into the codebase (but maybe I am too much of a beginner and this is common sense for most).
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Daniela Brozzoni [Wed, 30 Mar 2022 14:29:31 +0000 (16:29 +0200)]
Discourage fee sniping with nLockTime
By default bdk sets the transaction's nLockTime to current_height
to discourage fee sniping.
current_height can be provided by the user through TxParams; if the user
didn't provide it, we use the last sync height, or 0 if we never synced.
Before this commit, you could create a transaction with `drain_to` set
without specifying recipients, nor `drain_wallet`, nor `utxos`. What would
happen is that BDK would pick one input from the wallet and send
that one to `drain_to`, which is quite weird.
This PR restricts the usage of `drain_to`: if you want to use it as a
change output, you need to set recipients as well. If you want to send
a specific utxo to the `drain_to` address, you specify it through
`add_utxos`. If you want to drain the whole wallet, you set
`drain_wallet`. In any other case, if `drain_to` is set, we return a
`NoRecipients` error.
### Description
This PR fixes the CI, which is currently failing after a Github Actions update.
The MSRV is bumped to 1.56.1 (from 1.56.0), since that's `reqwest`'s current MSRV.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Daniela Brozzoni [Tue, 21 Jun 2022 14:20:28 +0000 (16:20 +0200)]
Bump MSRV from 1.56.0 to 1.56.1
In this way we can continue using reqwest v0.11, whose MSRV is now
1.56.1
The only difference between v1.56.0 and v1.56.1 is a bug fix for
CVE-2021-42574.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
just a small typo fix
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Not much to say besides that I bumped some version which helps us to resolve some dependency hell :)
### Notes to the reviewers
`ahash` was previously fixed because of an incompatibility with the defined MSRV. The MSRV has been bumped to 1.56 so we can update the dependency.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
* [ ] I've updated `CHANGELOG.md`
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
Only use the old `importmulti` with Core versions that don't support descriptor-based (sqlite) wallets.
Add an extra feature to test against Core 0.20 called `test-rpc-legacy`.
This also makes us compatible with Core 23.0 and is thus a replacement for #613, which actually looking back at it was adding support for 23.0 but probably breaking older wallets by adding the extra argument to `createwallet`.
I believe #613 should now only focus on getting the tests to work against 23.0, which is still important but not such a high priority as being compatible with the latest version of Core.
Also fixes #598
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] I've added tests to reproduce the issue which are now passing
This is a work-in-progress PR to update BDK to rust-bitcoin `0.28` which introduces taproot support and a few other improvements. While updating we also introduce taproot support in BDK.
High level list of subtasks for this PR:
- [x] Update rust-bitcoin and rust-miniscript
- [x] Stop using deprecated structs
- [x] Add taproot metadata to psbts
- [x] Produce schnorr signatures
- [x] Finalize taproot txs
- [x] Support `tr()` descriptors in the `descriptor!()` macro
- [x] Write a lot of tests
- [x] Interoperability with other wallets (Core + ?)
- [x] Signing/finalizing a psbt made by core
- [x] Producing a psbt that core can sign and finalize
- [x] Creating psbts
- [x] Verify the metadata are correct
- [x] Verify sighashes are applied correctly
- [x] Create a tx with a foreign taproot and non-taproot utxo
- [x] Signing psbts
- [x] Signing for a key spend
- [x] Signing for a script spend
- [x] Signing with a single (wif) key
- [x] Signing with an xprv (with and without knowing the utxo being spent in the db)
- [x] Signing with weird sighashes
- [x] Policy module
- [x] Simple key spend
- [x] More complex tap tree with a few keys
- [x] Verify both `contribution` and `satisfaction` of a PSBT input
- [x] Wallet module
- [x] Generate addresses
Fixes #63
### Notes to the reviewers
#### Milestone
I'm adding this to the `0.19` milestone because now that rust-bitcoin and rust-miniscript have been released we should not waiting too long to release a version of BDK that supports the new libraries.
#### API Breaks
Since this is an API-break because of the new version of rust-bitcoin and rust-miniscript, I'm also taking the chance to update a few things in our lib that I had been thinking about for a while.
One example is the signer interface, which had that weird `sign_whole_tx()` method. This has now been removed, and the `Signer` trait replaced with `TransactionSigner` and `InputSigner`. I'm also starting to think that the signer should not only look at the psbt to figure out what to do, but ideally it should also receive some information about the descriptor (for example, the type) to simplify the code.
One option is to add an extra parameter, but that would probably only be used by our internal signers and not much else (for example, if you ask an hardware wallet to sign, it will probably already know what kind of wallet you have).
Another option is to wrap `PrivateKey` and `DescriptorXKey<ExtendedPrivKey>` which are the two internal signers we support with a struct that contains metadata about the descriptor, and then implement the signer traits on that struct. We could construct this in `Wallet::new()`, after miniscript parses the descriptor.
#### MSRV Bump
Due to the update of `rust-electrum-client`, which in turn depends on an updated `webpki`, we will have to bump our MSRV beacuse 1.46 is not supported by the new `webpki` version.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added docs for the new feature
* [ ] I've updated `CHANGELOG.md`
Alekos Filini [Thu, 12 May 2022 15:28:41 +0000 (17:28 +0200)]
Attach a context to our software signers
This allows the signer to know the signing context precisely without
relying on heuristics on the psbt fields.
Due to the context being static, we still have to look at the PSBT when
producing taproot signatures to determine the set of leaf hashes that
the key can sign for.
This PR fixes #591 by:
1. Add sqlite `MIGRATIONS` statements to remove duplicate utxos and add unique utxos index on txid and vout.
2. Do an upsert (if insert fails update) instead of an insert in `set_utxo()`.
3. Update database::test::test_utxo to also verify `set_utxo()` doesn't insert duplicate utxos.
### Notes to the reviewers
I verified the updated `test_utxo` fails as expected before my fix and passes after the fix. I tested the new migrations using the below `bdk-cli` command and a manually updated sqlite db with duplicate utxos.
```shell
cargo run --no-default-features --features cli,sqlite-db,esplora-ureq -- wallet -w test1 --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
```
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
* [ ] I've updated `CHANGELOG.md`
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
danielabrozzoni:
utACK 35feb107ed5969720ce54a6aa76b7b2176f6c7c1 - Code looks good, but I didn't do any local test to see if the db gets wiped
Also explicitly match all the individual variants to ensure a similar problem
doesn't happen again.
Fixes #609
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
unpinning dependency tokio to just 1
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
* [x] I've updated `CHANGELOG.md`
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
Upgrade all our dependencies to work with the new release of rust-bitcoin
### Notes to the reviewers
The commits in this pr were originally part of #593
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
projects / bdk / log