feat(electrum)!: use new sync/full-scan structs for `ElectrumExt`
`ElectrumResultExt` trait is also introduced that adds methods which can
convert the `Anchor` type for the update `TxGraph`.
We also make use of the new `TxCache` fields in
`SyncRequest`/`FullScanRequest`. This way, we can avoid re-fetching full
transactions from Electrum if not needed.
Examples and tests are updated to use the new `ElectrumExt` API.
refactor(electrum): remove `RelevantTxids` and track txs in `TxGraph`
This PR removes `RelevantTxids` from the electrum crate and tracks
transactions in a `TxGraph`. This removes the need to separately
construct a `TxGraph` after a `full_scan` or `sync`.
- Moves keychain::ChangeSet inside `keychain/txout_index.rs` as now the `ChangeSet` depends on miniscript
- Slightly cleans up tests by introducing some constant descriptors
- The KeychainTxOutIndex's internal SpkIterator now uses DescriptorId
instead of K. The DescriptorId -> K translation is made at the
KeychainTxOutIndex level.
- The keychain::Changeset is now a struct, which includes a map for last
revealed indexes, and one for newly added keychains and their
descriptor.
### Changelog notice
API changes in bdk:
- Wallet::keychains returns a `impl Iterator` instead of `BTreeMap`
- Wallet::load doesn't take descriptors anymore, since they're stored in the db
- Wallet::new_or_load checks if the loaded descriptor from db is the same as the provided one
API changes in bdk_chain:
- `ChangeSet` is now a struct, which includes a map for last revealed
indexes, and one for keychains and descriptors.
- `KeychainTxOutIndex::inner` returns a `SpkIterator<(DescriptorId, u32)>`
- `KeychainTxOutIndex::outpoints` returns a `BTreeSet` instead of `&BTreeSet`
- `KeychainTxOutIndex::keychains` returns a `impl Iterator` instead of
`&BTreeMap`
- `KeychainTxOutIndex::txouts` doesn't return a ExactSizeIterator anymore
- `KeychainTxOutIndex::last_revealed_indices` returns a `BTreeMap`
instead of `&BTreeMap`
- `KeychainTxOutIndex::add_keychain` has been renamed to `KeychainTxOutIndex::insert_descriptor`, and now it returns a ChangeSet
- `KeychainTxOutIndex::reveal_next_spk` returns Option
- `KeychainTxOutIndex::next_unused_spk` returns Option
- `KeychainTxOutIndex::unbounded_spk_iter` returns Option
- `KeychainTxOutIndex::next_index` returns Option
- `KeychainTxOutIndex::reveal_to_target` returns Option
- `KeychainTxOutIndex::revealed_keychain_spks` returns Option
- `KeychainTxOutIndex::unused_keychain_spks` returns Option
- `KeychainTxOutIndex::last_revealed_index` returns Option
- `KeychainTxOutIndex::keychain_outpoints` returns Option
- `KeychainTxOutIndex::keychain_outpoints_in_range` returns Option
- `KeychainTxOutIndex::last_used_index` returns None if the keychain has never been used, or if it doesn't exist
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
This PR pins clippy check in CI to the rust 1.78 toolchain, which prevents new lints in stable releases from interrupting the usual workflow. Because rust versions are released on a predictable schedule, we can revisit this setting in the future as needed (say 3-6 months).
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [ ] I ran `cargo fmt` and `cargo clippy` before committing
fix(chain): introduce keychain-variant-ranking to `KeychainTxOutIndex`
This fixes the bug with changesets not being monotone. Previously, the
result of applying changesets individually v.s. applying the aggregate
of changesets may result in different `KeychainTxOutIndex` states.
The nature of the changeset allows different keychain types to share the
same descriptor. However, the previous design did not take this into
account. To do this properly, we should keep track of all keychains
currently associated with a given descriptor. However, the API only
allows returning one keychain per spk/txout/outpoint (which is a good
API).
Therefore, we rank keychain variants by `Ord`. Earlier keychain variants
have a higher rank, and the first keychain will be returned.
fix(chain): simplify `Append::append` impl for `keychain::ChangeSet`
We only need to loop though entries of `other`. The logic before was
wasteful because we were also looping though all entries of `self` even
if we do not need to modify the `self` entry.
Daniela Brozzoni [Mon, 15 Jan 2024 17:52:03 +0000 (18:52 +0100)]
keychain::ChangeSet includes the descriptor
- The KeychainTxOutIndex's internal SpkIterator now uses DescriptorId
instead of K. The DescriptorId -> K translation is made at the
KeychainTxOutIndex level.
- The keychain::Changeset is now a struct, which includes a map for last
revealed indexes, and one for newly added keychains and their
descriptor.
API changes in bdk:
- Wallet::keychains returns a `impl Iterator` instead of `BTreeMap`
- Wallet::load doesn't take descriptors anymore, since they're stored in
the db
- Wallet::new_or_load checks if the loaded descriptor from db is the
same as the provided one
API changes in bdk_chain:
- `ChangeSet` is now a struct, which includes a map for last revealed
indexes, and one for keychains and descriptors.
- `KeychainTxOutIndex::inner` returns a `SpkIterator<(DescriptorId, u32)>`
- `KeychainTxOutIndex::outpoints` returns a `impl Iterator` instead of `&BTreeSet`
- `KeychainTxOutIndex::keychains` returns a `impl Iterator` instead of
`&BTreeMap`
- `KeychainTxOutIndex::txouts` doesn't return a ExactSizeIterator
anymore
- `KeychainTxOutIndex::unbounded_spk_iter` returns an `Option`
- `KeychainTxOutIndex::next_index` returns an `Option`
- `KeychainTxOutIndex::last_revealed_indices` returns a `BTreeMap`
instead of `&BTreeMap`
- `KeychainTxOutIndex::reveal_to_target` returns an `Option`
- `KeychainTxOutIndex::reveal_next_spk` returns an `Option`
- `KeychainTxOutIndex::next_unused_spk` returns an `Option`
- `KeychainTxOutIndex::add_keychain` has been renamed to
`KeychainTxOutIndex::insert_descriptor`, and now it returns a
ChangeSet
志宇 [Wed, 11 Oct 2023 16:03:18 +0000 (00:03 +0800)]
ref(chain): move `keychain::ChangeSet` into `txout_index.rs`
We plan to record `Descriptor` additions into persistence. Hence, we
need to add `Descriptor`s to the changeset. This depends on
`miniscript`. Moving this into `txout_index.rs` makes sense as this is
consistent with all the other files. The only reason why this wasn't
this way before, is because the changeset didn't need miniscript.
This PR adds a `default` feature to `bdk_persist` so it can be build on its own. Once #1422 is done we can remove the `default`again.
### Notes to the reviewers
I need to be able to build `bdk_persist` on its own so I can publish it to crates.io.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
It's being used on `Balance`, and throughout the code, an `u64` represents the amount, which relies on the user to infer its sats, not millisats, or any other representation.
It updates the usage of `u64` on `Balance`, and other APIs:
- `TxParams::add_recipient`
- `KeyChainTxOutIndex::sent_and_received`, `KeyChainTxOutIndex::net_value`
- `SpkTxOutIndex::sent_and_received`, `SpkTxOutIndex::net_value`
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
It updates some of the APIs to expect the `bitcoin::Amount`, but it does not update internal usage of u64, such as `TxParams` still expects and uses `u64`, please see the PR comments for related discussion.
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
- Changed the `keychain::Balance` struct fields to use `Amount` instead of `u64`.
- Changed the `add_recipient` method on `TxBuilder` implementation to expect `bitcoin::Amount`.
- Changed the `sent_and_received`, and `net_value` methods on `KeyChainTxOutIndex` to expect `bitcoin::Amount`.
- Changed the `sent_and_received`, and `net_value` methods on `SpkTxOutIndex` to expect `bitcoin::Amount`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Reexports `electrsd` in `TestEnv` to remove the `electrsd` dev depedency out of `bdk_electrum` and `bdk_esplora`.
Credit to @ValuedMammal for the idea.
Since `bitcoind` reexports `anyhow`, this dev dependency was also removed from `bdk_electrum`, `bdk_esplora`, and `bdk_bitcoind_rpc`. `bitcoind`, `bitcoincore_rpc` and `electrum_client` were also reexported for convenience.
### Changelog notice
* Change `bdk_testenv` to re-export internally used crates.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Fixed documentation for `full_scan` and `sync` in `bdk_esplora`.
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
* Updated documentation for `full_scan` and `sync` in `bdk_esplora`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Leonardo Lima [Wed, 24 Apr 2024 21:12:45 +0000 (18:12 -0300)]
feat: update `wallet::Balance` to use `bitcoin::Amount`
- update all fields `immature`, ` trusted_pending`, `unstrusted_pending`
and `confirmed` to use the `bitcoin::Amount` instead of `u64`
- update all `impl Balance` methods to use `bitcoin::Amount`
- update all tests that relies on `keychain::Balance`
`Wallet` auto-implements `Send` and `Sync` after removing the generic. This test is a compile time error if there are changes to `Wallet` in the future that make it unsafe to send between threads. See #1387 for discussion.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
bdk_chain to 0.13.0
bdk_bitcoind_rpc to 0.9.0
bdk_electrum to 0.12.0
bdk_esplora to 0.12.0
bdk_file_store to 0.10.0
bdk_testenv to 0.3.0
bdk_persist to 0.2.0
### Checklists
#### All Submissions:
- [x] I've signed all my commits
- [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
- [x] I ran cargo fmt and cargo clippy before committing
bdk_chain to 0.13.0
bdk_bitcoind_rpc to 0.9.0
bdk_electrum to 0.12.0
bdk_esplora to 0.12.0
bdk_file_store to 0.10.0
bdk_testenv to 0.3.0
bdk_persist to 0.2.0
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Caught when trying to release (#1420), clippy failed randomly although it worked on master, this happened because rust 1.78 had just been release and we use clippy stable. IMHO we should pin the clippy version in CI and bump it manually at each new rust release.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Introduce universal structures that represent sync/full-scan requests/results.
### Notes to the reviewers
This is based on #1194 but is different in the following ways:
* The functionality to print scan/sync progress is not reduced.
* `SyncRequest` and `FullScanRequest` is simplified and fields are exposed for more flexibility.
### Changelog notice
* Add universal structures for initiating/receiving sync/full-scan requests/results for spk-based syncing.
* Updated `bdk_esplora` chain-source to make use of new universal sync/full-scan structures.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#1387 introduced `anyhow` as a dependency to remove generics from `Wallet`. Introducing a new crate for persistence types removes the dependency on `anyhow` for `bdk_chain`. Resolves #1409, as well as removing the old documentation for "tracker".
### Notes to the reviewers
Open for any comments.
### Changelog notice
- Introduce `bdk-persist` crate
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
feat(wallet): make wallet compatible with sync/full-scan structures
* Changed `Wallet::apply_update` to also take in anything that
implements `Into<Update>`. This allows us to directly apply a
`FullScanResult` or `SyncResult`.
* Added `start_full_scan` and `start_sync_with_revealed_spks` methods to
`Wallet`.
Co-authored-by: Steve Myers <steve@notmandatory.org>
Built on top of both #1369 and #1373, we simplify the `EsploraExt` API by removing the `update_local_chain` method and having `full_scan` and `sync` update the local chain in the same call. The `full_scan` and `sync` methods now takes in an additional input (`local_tip`) which provides us with the view of the `LocalChain` before the update. These methods now return structs `FullScanUpdate` and `SyncUpdate`.
The examples are updated to use this new API. `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` are no longer needed, therefore they are removed.
Additionally, we used this opportunity to simplify the logic which updates `LocalChain`. We got rid of the `local_chain::Update` struct (which contained the update `CheckPoint` tip and a `bool` which signaled whether we want to introduce blocks below point of agreement). It turns out we can use something like `CheckPoint::insert` so the chain source can craft an update based on the old tip. This way, we can make better use of `merge_chains`' optimization that compares the `Arc` pointers of the local and update chain (before we were crafting the update chain NOT based on top of the previous local chain). With this, we no longer need the `Update::introduce_older_block` field since the logic will naturally break when we reach a matching `Arc` pointer.
### Notes to the reviewers
* Obtaining the `LocalChain`'s update now happens within `EsploraExt::full_scan` and `EsploraExt::sync`. Creating the `LocalChain` update is now split into two methods (`fetch_latest_blocks` and `chain_update`) that are called before and after fetching transactions and anchors.
* We need to duplicate code for `bdk_esplora`. One for blocking and one for async.
### Changelog notice
* Changed `EsploraExt` API so that sync only requires one round of fetching data. The `local_chain_update` method is removed and the `local_tip` parameter is added to the `full_scan` and `sync` methods.
* Removed `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` methods.
* Introduced `CheckPoint::insert` which allows convenient checkpoint-insertion. This is intended for use by chain-sources when crafting an update.
* Refactored `merge_chains` to also return the resultant `CheckPoint` tip.
* Optimized the update `LocalChain` logic - use the update `CheckPoint` as the new `CheckPoint` tip when possible.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
The [`blocking` feature on the rust-esplora-client library](https://github.com/bitcoindevkit/rust-esplora-client/blame/master/Cargo.toml#L35) changed from ureq to minreq, which does not come with https enabled by default, breaking previously working code that simply enabled the `blocking` feature on the `bdk_esplora` crate.
This change will enable what is currently the "default https" [flag for the minreq library](https://docs.rs/minreq/latest/minreq/#https-or-https-rustls) when using the `blocking` feature on bdk_esplora, reverting that breaking change.
### Notes to the reviewers
Another way we could do this (let me know if this is preferable) is to add a new feature called `blocking-https-rustls`:
```rust
blocking = ["esplora-client/blocking"]
blocking-https-rustls = ["esplora-client/blocking-https-rustls"]
```
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
Improvements to the wallet address API, see commit messages for details.
### Notes to the reviewers
The logic of getting addresses is roughly the same as before when using `AddressIndex`, following this mapping:
- `New` -> `reveal_next_address`
- `LastUnused` -> `next_unused_address` (assuming this is what `LastUnused` really means)
- `Peek` -> `peek_address`
Wondering whether it makes sense to expose [`is_used`](https://github.com/bitcoindevkit/bdk/blob/358e842dcda0eb867d229881823c76dba1d7cce1/crates/chain/src/keychain/txout_index.rs#L236) for Wallet as well.
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
### Feature
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
* [x] This pull request breaks the existing API
* [x] I'm linking the issue being fixed by this PR
Introduce a new API for getting addresses from the Wallet that
reflects a similiar interface as the underlying indexer
`KeychainTxOutIndex` in preparation for removing `AddressIndex` enum.
Before this change, the only way to get an address was via the methods
`try_get{_internal}_address` which required a `&mut` reference to the
wallet, matching on the desired AddressIndex variant. This is too
restrictive since for example peeking or listing unused addresses
shouldn't change the state of the wallet. Hence we provide separate
methods for each use case which makes for a more efficient API.
The `PersistenceBackend` uses generics to describe errors returned while applying the change set to the persistence layer. This change removes generics wherever possible and introduces a new public error enum. Removing the generics from `PersistenceBackend` errors is the first step towards #1363
*Update*: I proceeded with removing the generics from `Wallet` by introducing a `Box<dyn PersistenceBackend>` .
### Notes to the reviewers
This one sort of blew up in the number of changes due to the use of generics for most of the `Wallet` error variants. The generics were only used for the persistence errors, so I removed the generics from higher level errors whenever possible. The error variants of `PersistenceBackend` may also be more expressive, but I will level that up for discussion and make any changes required.
### Changelog notice
- Changed `PersistenceBackend` errors to depend on the `anyhow` crate.
- Remove the generic `T` from `Wallet`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
KeychainTxOut index should try and avoid "all" kind of queries. There may be subranges of interest. If the user wants "all" they can just query "..".
The ideas is that KeychainTxOutIndex should be designed to be able to incorporate many unrelated keychains that can be managed in the same index. We should be able to see the "net_value" of a transaction to a specific subrange. e.g. imagine a collaborative custody service that manages all their user descriptors inside the same `KeychainTxOutIndex`. One user in their service may pay another so when you are analyzing how much a transaction is spending for a particular user you need to do analyze a particular sub-range.
### Notes to the reviewers
- I didn't change `unused_spks` to follow this rule because I want to delete that method some time in the future. `unused_spks` is being used in the examples for syncing but it shouldn't be (the discussion as to why will probably surface in #1194).
- I haven't applied this reasoning to the methods that return `BTreeMap`s e.g. `all_unbounded_spk_iters`. It probably should be but I haven't made up my mind yet.
This probably belongs after #1194
### Changelog notice
- `KeychainTxOutIndex` methods modified to take ranges of keychains instead.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
Previously, we would update the `TxGraph` and `KeychainTxOutIndex`
first, then create a second update for `LocalChain`. This required
locking the receiving structures 3 times (instead of twice, which
is optimal).
This PR eliminates this requirement by making use of the new `query`
method of `CheckPoint`.
bdk_chain to 0.12.0
bdk_bitcoind_rpc to 0.8.0
bdk_electrum to 0.11.0
bdk_esplora to 0.11.0
bdk_file_store to 0.9.0
bdk_testenv to 0.2.0
fixes #1399
### Notes to the reviewers
I also removed unneeded version for bdk_testenv in dev-dependencies, see https://github.com/bitcoindevkit/bdk/pull/1177#discussion_r1545945973.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
This is based on #1369. It made sense for me to introduce the `CheckPoint::range` test as a proptest. I think we should also start introducing it for other methods too.
### Changelog notice
* Added proptest for `CheckPoint::range`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
A method `update_last_seen_unconfirmed` is added for `TxGraph` that allows inserting a `seen_at` time for all unanchored transactions.
fixes #1336
### Changelog notice
Changed:
- Methods `into_tx_graph` and `into_confirmation_time_tx_graph`for `RelevantTxids` are changed to no longer accept a `seen_at` parameter.
Added:
- Added method `update_last_seen_unconfirmed` for `TxGraph`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Tobin C. Harding [Mon, 16 Oct 2023 08:51:53 +0000 (19:51 +1100)]
Upgrade miniscript/bitcoin dependency
Upgrade:
- bitcoin to v0.31.0
- miniscript to v11.0.0
Note: The bitcoin upgrade includes improvements to the
`Transaction::weight()` function, it appears those guys did good, we
no longer need to add the 2 additional weight units "just in case".
fix(electrum)!: Remove `seen_at` param from `into_tx_graph`
and `into_confirmation_time_tx_graph`, since now it makes
sense to use `TxGraph::update_last_seen_unconfirmed`.
Also, use `update_last_seen_unconfirmed` in examples for
electrum/esplora. We show how to update the last seen
time for transactions by calling `update_last_seen_unconfirmed`
on the graph update returned from a blockchain source, passing
in the current time, before applying it to another `TxGraph`.
That accepts a `u64` as param representing the latest timestamp
and internally calls `insert_seen_at` for all transactions in
graph that aren't yet anchored in a confirmed block.
These methods allow us to query for checkpoints contained within the linked list by height and height range. This is useful to determine checkpoints to fetch for chain sources without having to refer back to the `LocalChain`.
Currently this is not implemented efficiently, but in the future, we will change `CheckPoint` to use a skip list structure.
### Notes to the reviewers
Please refer to the conversation in #1354 for more details. In summary, both `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` are problematic in their own ways. Additionally, it's a better API for chain sources if we only need to lock our receiving structures twice (once to obtain initial state and once for applying the update). Instead of relying on methods such as `EsploraExt::update_local_chain` to get relevant checkpoints, we can use these query methods instead. This allows up to get rid of `::missing_heights`-esc methods and remove the need to lock receiving structures in the middle of the sync.
### Changelog notice
* Added `get` and `range` methods to `CheckPoint` (and in turn, `LocalChain`). This simulates an API where we have implemented a skip list of checkpoints (to implement in the future). This is a better API because we can query for any height or height range with just a checkpoint tip instead of relying on a separate checkpoint index (which needs to live in `LocalChain`).
* Changed `LocalChain` to have a faster `Eq` implementation. We now maintain an xor value of all checkpoint block hashes. We compare this xor value to determine whether two chains are equal.
* Added `PartialEq` implementation for `CheckPoint` and `local_chain::Update`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
chore(chain)!: rm `From<LocalChain> for BTreeMap<u32, BlockHash>`
I don't think this was ever used. The only possible usecase I can think
of is for tests, but I don't think that is a strong enough incentive for
us to keep this.
feat(chain): add `get` and `range` methods to `CheckPoint`
These methods allow us to query for checkpoints contained within the
linked list by height and height range. This is useful to determine
checkpoints to fetch for chain sources without having to refer back to
the `LocalChain`.
Currently this is not implemented efficiently, but in the future, we
will change `CheckPoint` to use a skip list structure.
### Description
Continuation of #1115.
Closes #1036.
* Change deprecated `max_satisfaction_weight` to `max_weight_to_satisfy`
* Remove `#[allow(deprecated)]` flags
### Notes to the reviewers
I've changed all `max_satisfaction_weight()` to `max_weight_to_satisfy()` in `Wallet.get_available_utxo()` and `Wallet.build_fee_bump()`. Checking the docs on the `miniscript` crate for `max_weight_to_satisfy` has the following note:
We are testing if the underlying descriptor `is.segwit()` or `.is_taproot`,
then adding 4WU if true or leaving as it is otherwise.
Another thing, we are not testing in BDK tests for legacy (pre-segwit) descriptors.
Should I also add them to this PR?
### Changelog notice
### Fixed
Replace the deprecated `max_satisfaction_weight` from `rust-miniscript` to `max_weight_to_satisfy`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
This PR makes `TxGraph` store transactions as `Arc<Transaction>`.
`Arc<Transaction>` can be shared between the chain-source and receiving structures. This allows the chain-source to keep the already-fetched transactions (save bandwith) and have a shared pointer to the transaction (save memory).
Our current logic to avoid re-fetching transactions is to refer back to the receiving structures. However, this means an additional round of locking our receiving structures.
### Notes to the reviewers
This will make more sense once I update the esplora/electrum chain sources to make use of both #1369 and this PR. The result would be chain sources which would only require locking the receiving structures twice (once for initiating the update, and once for applying the update).
### Changelog notice
* Changed `TxGraph` to store transactions as `Arc<Transaction>`. This allows chain-sources to cheaply keep a copy of already-fetched transactions.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
Jose Storopoli [Sun, 10 Sep 2023 09:53:56 +0000 (05:53 -0400)]
fix: remove deprecated `max_satisfaction_weight
- Change deprecated `max_satisfaction_weight` to `max_weight_to_satisfy`
- Remove `#[allow(deprecated)]` flags
- updates the calculations in TXIN_BASE_WEIGHT and P2WPKH_SATISFACTION_SIZE
bdk version to 1.0.0-alpha.8
bdk_bitcoind_rpc to 0.7.0
bdk_electrum to 0.10.0
bdk_esplora to 0.10.0
bdk_file_store to 0.8.0
bdk_hwi to 0.2.0
fixes #1388
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
This will make the `bdk_testenv` crate actually publishable.
### Changelog notice
* Fix cargo manifest of `bdk_testenv`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
- changes the code implementation to "the maximum number of consecutive unused addresses" in esplora async and blocking extensions.
- for all purposes treats `stop_gap = 0` as `stop_gap = 1`.
- renames `past_gap_limit` to `gap_limit_reached` to indicate we want to break once the gap limit is reached and not go further in `full_scan`, as suggested in https://github.com/bitcoindevkit/bdk/issues/1227#issuecomment-1859040463
- change the tests according to the new implementation.
- add notes on what `stop_gap` means and links to convergent definition in other Bitcoin-related software.
Closes #1227.
### Notes to the reviewers
We can iterate over the wording and presentation of the `stop_gap` definition
and details.
### Changelog notice
- BREAKING: change `stop_gap` definition and effects in `full_scan`
to reflect the common definitions in most Bitcoin-related software.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Jose Storopoli [Tue, 26 Mar 2024 15:44:03 +0000 (12:44 -0300)]
fix: define and document `stop_gap`
- changes the code implementation to "the maximum number of consecutive unused addresses"
in esplora async and blocking extjensions.
- treat `stop_gap = 0` as `stop_gap = 1` for all purposes.
- renames `past_gap_limit` to `gap_limit_reached` to indicate we want to break once the gap
limit is reached and not go further in `full_scan`, as suggested in
https://github.com/bitcoindevkit/bdk/issues/1227#issuecomment-1859040463
- change the tests according to the new implementation.
- add notes on what `stop_gap` means and links to convergent definition in other
Bitcoin-related software.
志宇 [Thu, 14 Mar 2024 06:20:51 +0000 (14:20 +0800)]
feat(chain)!: wrap `TxGraph` txs with `Arc`
Wrapping transactions as `Arc<Transaction>` allows us to share
transactions cheaply between the chain-source and receiving structures.
Therefore the chain-source can keep already-fetched transactions (save
bandwidth) and have a shared pointer to the transactions (save memory).
This is better than the current way we do things, which is to refer back
to the receiving structures mid-sync.
`TestEnv` is extracted into its own crate with `electrsd` support added so that `TestEnv` can also serve `esplora` and `electrum`.
The tests in the `esplora` crate have also been updated to use `TestEnv`.
The `tx_can_become_unconfirmed_after_reorg()` test in `test_electrum` suggests that the electrum issue where a reorged tx would be stuck at confirmed does not exist anymore.
### Notes to the reviewers
The code for `tx_can_become_unconfirmed_after_reorg()` was adapted from the same test in `bitcoind_rpc/test_emitter`. This electrum version of the test requires extra review, as I am uncertain if I used the API efficiently.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
projects / bdk / log