Before, the `chain_update` function might have panicked if the local checkpoint was not on the same network as the remote server. Now if we have iterated all of the blocks of the local CP and do not find a point of agreement, then we return early with a `esplora_client::Error::HeaderHashNotFound`.
cc bitcoindevkit/bdk_wallet#30
### Notes to the reviewers
### Changelog notice
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo +nightly fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
- Add benchmark `reindex_tx_graph`. Run with `cargo bench -p bdk_chain --bench indexer`.
- Add unit test to `keychain_txout` module to test behavior of spk-cache.
- Fixup a few doc comments.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo +nightly fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
This PR covers a wide range on tests for excluded bounds and the `SpkTxOutIndex::outputs_in_range`. This PR fixes bitcoindevkit/bdk_wallet#58
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Replaces #1908, originally authored by @Keerthi421.
Fixes #1891.
### Description
This PR optimizes `sync`/`full_scan` performance by batching and caching key RPC calls to slash network round-trips and eliminate redundant work.
Key improvements:
* Gather all `blockchain.transaction.get_merkle` calls into a single `batch_call` request.
* Use `batch_script_get_history` instead of many individual `script_get_history` calls.
* Use `batch_block_header` to fetch all needed block headers in one call rather than repeatedly calling `block_header`.
* Introduce a cache of transaction anchors to skip re-validating already confirmed transactions.
#### Anchor Caching Performance Improvements
Results suggest a significant speed up with a warmed up cache. Tested on local Electrum server with:
```
$ cargo bench -p bdk_electrum --bench test_sync
```
Results before this PR (https://github.com/LagginTimes/bdk/tree/1957-master-branch):
```
sync_with_electrum time: [1.3702 s 1.3732 s 1.3852 s]
```
Results after this PR:
```
sync_with_electrum time: [851.31 ms 853.26 ms 856.23 ms]
```
#### Batch Call Performance Improvements
No persisted data was carried over between runs, so each test started with cold caches and measured only raw batching performance. Tested with`example_electrum` out of https://github.com/LagginTimes/bdk/tree/example_electrum_timing with the following parameters:
Results after this PR (using this PR's [`bdk_electrum_client.rs`](https://github.com/bitcoindevkit/bdk/blob/70495e2010541acbb5d62f9b5692de20924ac53f/crates/electrum/src/bdk_electrum_client.rs)):
```
FULL_SCAN TIME: 2.594050112s
```
### Changelog notice
* Add transaction anchor cache to prevent redundant network calls.
* Batch Merkle proof, script history, and header requests.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
valued mammal [Thu, 5 Jun 2025 17:17:21 +0000 (13:17 -0400)]
fix(esplora): `chain_update` errors if no point of connection
Before, the `chain_update` function would hit a panic if the
local checkpoint was not on the same network as the remote
server. Now if we have iterated all of the blocks of the
`local_cp` and do not find a "point of agreement", then we
return early with a `esplora_client::Error::HeaderHashNotFound`.
志宇 [Wed, 25 Jun 2025 23:34:50 +0000 (23:34 +0000)]
test(electrum): Improve benchmark
* Actually use different spks
* Do not benchmark applying updates (only fetching/contructing)
* Have two benches: One with cache, one without.
* Remove `black_box`.
Automated update to Github CI workflow `cont_integration.yml` by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo +nightly fmt` and `cargo clippy` before committing
Users with large wallet and/or complex descriptors may experience slow startup of `KeychainTxOutIndex`. This PR addresses this problem by providing the option to persist derived spks so that they no longer need to be re-derived on startup.
The `IndexedTxGraph` API has been changed for better ergonomics.
Compared to #1960, this is a more longterm solution that does not depend on multi-threading logic.
### Changelog notice
```md
Changed
- `KeychainTxOutIndex::new` to take in an additional parameter `persist_spks` to control whether derived spks are cached and persisted across restarts. The default of `persist_spks` is false.
- `KeychainTxOutIndex` methods (`lookahead_to_target, `next_unused_spk`, `reveal_next_spk`) now return changesets as they may derive spks to be persisted.
- The `InsertDescriptorError` type now wraps descriptors in `Box` to reduce enum size.
Added
- `spk_cache` field to `indexer::keychain_txout::ChangeSet` which persists derived spks.
- `IndexedTxGraph::from_changeset` - allows constructing from `indexed_tx_graph::ChangeSet` and only indexing when ready.
- `IndexedTxGraph::reindex` method.
Fixed
- `KeychainTxOutIndex::reveal_to_target` so that it actually returns `None` if the `keychain` does not exist.
```
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo +nightly fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
Adds missing persistence for `first_seen`, which was not included in #1950.
### Changelog notice
- Adds `first_seen` column to the `bdk_txs` table via schema v3 migration.
- Updates `from_sqlite()` and `persist_to_sqlite()` to handle `first_seen`.
- Updates the v0-to-v3 migration test to verify compatibility with older schemas.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo +nightly fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
`is_empty` methods are helpful in various contexts so I added them.
### Changelog notice
```md
Added
- `is_empty` methods to `TxUpdate`, `SyncResponse` and `FullScanResponse`
```
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
志宇 [Fri, 23 May 2025 08:27:55 +0000 (18:27 +1000)]
feat(chain)!: Clean up ergonomics of `IndexedTxGraph`
* `new` is now intended to construct a fresh indexed-tx-graph
* `from_changeset` is added for constructing indexed-tx-graph from a
previously persisted changeset
* added `reindex` for calling after indexer mutations that require it
* reintroduce `Default` impl
志宇 [Fri, 23 May 2025 05:44:35 +0000 (15:44 +1000)]
feat(chain): `KeychainTxOutIndex`: Make spk cache optional
Also added staging changes to `ChangeSet::spk_cache`. This way, we can
avoid returning `ChangeSet`s for `apply_changeset` and
`insert_descriptor`.
* `KeychainTxOutIndex::new` now takes in an additional `use_spk_cache`
parameter.
* Fixed `reveal_to_target` method to actually return `None` if the
keychain does not exist.
* When merging changesets, assert that spk of a given descriptor id &
derivation index does not get changed.
* When reading spk from cache, check the spk by deriving it.
This PR adds basic formatting configuration for uniform comments. Note that the nightly toolchain is necessary.
### Changelog:
- Create rustfmt.toml with this config:
```
comment_width = 100
format_code_in_doc_comments = true
wrap_comments = true
```
- Apply rustfmt.toml.
- Update the fmt CI job to use the nightly toolchain.
- Update PR template to use nightly toolchain.
Note: I had to add #[rustfmt::skip] to test_extract_satisfaction_timelock() because the base64 PSBT would get wrapped due to the slashes.
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo +nightly fmt` and `cargo clippy` before committing
ci: add zizmor github actions security analysis workflow and fix possible vulnerabilities
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Update `rust-version` to 1.86.0 and address some clippy lints, in particular [`clippy::doc_overindented_list_items`](https://rust-lang.github.io/rust-clippy/master/index.html#doc_overindented_list_items).
fix #1940
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
It's a minor documentation fix, as reported during the audit and referenced in bitcoindevkit/bdk_wallet#59.
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
### Notes to the reviewers
I'm unsure if anything else / any other explanation should be included in the documentation. Let me know if you think more context should be added to it.
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
This PR solves issue #1947 by implementing `first_seen` tracking.
* Added `first_seen` field to `TxGraph` and `ChangeSet` so that `first_seen` timestamp can be added when inserting a new seen-at using `insert_seen_at`.
* `first_seen` added to `TxNode` as a way to retrieve the first-seen timestamp for a transaction.
* `first_seen` added to `ChainPosition::Unconfirmed` to order unconfirmed transactions by `first_seen`.
* New tests have been added for the above described functionalities.
### Changelog notice
* Add tracking first-seen timestamps of transactions
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
Work for this began as part of #1811, based on this [comment](https://github.com/bitcoindevkit/bdk/pull/1811#discussion_r1945941691).
`Emitter::mempool` now returns a `MempoolEvent` which provides the data for tracking `evicted_at`:
```
pub struct MempoolEvent {
/// Unemitted transactions or transactions with ancestors that are unseen by the receiver.
///
/// To understand the second condition, consider a receiver which filters transactions based on
/// whether it alters the UTXO set of tracked script pubkeys. If an emitted mempool transaction
/// spends a tracked UTXO which is confirmed at height `h`, but the receiver has only seen up to
/// block of height `h-1`, we want to re-emit this transaction until the receiver has seen the
/// block at height `h`.
pub new_txs: Vec<(Transaction, u64)>,
/// [`Txid`]s of all transactions that have been evicted from mempool.
pub evicted_txids: HashSet<Txid>,
/// The latest timestamp of when a transaction entered the mempool.
///
/// This is useful for setting the timestamp for evicted transactions.
pub latest_update_time: u64,
}
```
### Changelog notice
* Change `Emitter::mempool` to return `MempoolEvent`s which contain mempool-eviction data.
* Change `Emitter::client` to have more relaxed generic bounds. `C: Deref, C::Target: RpcApi` are the new bounds.
* Add conversion impls for `CanonicalTx` to `Txid`/`Arc<Transaction>`.
* Add `ChainPosition::is_unconfirmed` method.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
feat(bitcoind_rpc)!: Reduce friction of `Emitter` API.
* Change signature of `Emitter::new` so that `expected_mempool_txids`
can be more easily constructed from `TxGraph` methods.
* Change generic bounds of `C` within `Emitter<C>` to be `C: DeRef,
C::Target: RpcApi`. This allows the caller to have `Arc<Client>` as
`C` and does not force to caller to hold a lifetimed reference.
Add the ability to modify the canonicalization algorithm. Right now, the only modifier is `assume_canonical` which takes in a `Vec` (ordered list) of txids and superimposes it on the canonicalization algorithm. Txs later in the list (higher index) have a higher priority (in case of conflicts).
### Notes to the reviewers
None at the moment.
### Changelog notice
* Added `CanonicalizationParams` to allow the caller to modify the canonicalization algorithm. This in a new parameter on `CanonicalIter::new`.
* Changed `TxGraph::insert_tx` to allow for updating a transaction's witness field. This is useful for initially introducing an unsigned tx and adding witnesses later on.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
志宇 [Thu, 23 Jan 2025 04:16:46 +0000 (15:16 +1100)]
feat(chain)!: Add ability to modify canonicalization algorithm
Introduce `CanonicalizationParams` which is passed in to
`CanonicalIter::new`.
`CanonicalizationParams::assume_canonical` is the only field right now.
This contains a list of txids that we assume to be canonical,
superceding any other canonicalization rules.
Bumps [Swatinem/rust-cache](https://github.com/swatinem/rust-cache) from 2.7.7 to 2.7.8.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/swatinem/rust-cache/releases">Swatinem/rust-cache's releases</a>.</em></p>
<blockquote>
<h2>v2.7.8</h2>
<h2>What's Changed</h2>
<ul>
<li>Include CPU arch in the cache key for arm64 Linux runners by <a href="https://github.com/rhysd"><code>@rhysd</code></a> in <a href="https://redirect.github.com/Swatinem/rust-cache/pull/228">Swatinem/rust-cache#228</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/Swatinem/rust-cache/compare/v2.7.7...v2.7.8">https://github.com/Swatinem/rust-cache/compare/v2.7.7...v2.7.8</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md">Swatinem/rust-cache's changelog</a>.</em></p>
<blockquote>
<h2>2.7.8</h2>
<ul>
<li>Include CPU arch in the cache key</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/Swatinem/rust-cache/commit/9d47c6ad4b02e050fd481d890b2ea34778fd09d6"><code>9d47c6a</code></a> 2.7.8</li>
<li><a href="https://github.com/Swatinem/rust-cache/commit/27b8ea9368cf428f0bfe41b0876b1a7e809d9844"><code>27b8ea9</code></a> Include CPU arch in the cache key (<a href="https://redirect.github.com/swatinem/rust-cache/issues/228">#228</a>)</li>
<li>See full diff in <a href="https://github.com/swatinem/rust-cache/compare/v2.7.7...v2.7.8">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
When we initially created the canonicalization algorithm, we didn't expect callers to insert invalid transactions into the graph. However, user error happens and we should handle it correctly.
Before this PR, when inserting transactions that double-spend themselves (where two or more different inputs of the transactions conflict), the canonicalization result will have inconsistencies.
### Notes to the reviewers
Logic changes are all contained in `CanonicalIter::mark_canonical`. `mark_canonical` will detect whether the `tx` being passed in double spends itself. If so, we abort and undo all changes made so far.
There is a slight <2% degradation in performance with this change (except in two cases where there is a performance improvement of ~10%).
* Fix canonicalization mess-up when transactions that conflict with itself are inserted.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
~~* [ ] This pull request breaks the existing API~~
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
志宇 [Fri, 28 Mar 2025 06:06:26 +0000 (17:06 +1100)]
fix(chain): Correctly handle txs that double spend themselves
Detect self-double-spends in `CanonicalIter::mark_canonical`. Disregard
the tx that self-double-spends and all it's attached meta data (such as
timestamps and anchors).
Add new test cases on `test_tx_conflict_handling` for transactions that
self-double-spend.
### Description
* Added a `from_changeset()` function to `TxGraph` implementation
* This function initializes a new `TxGraph` and applies the given `ChangeSet`
* The method allows constructing a `TxGraph` directly from a `ChangeSet`,
simplifying graph creation.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
Removes the `bdk_wallet` crate from the workspace of this repo now that it's been moved to it's own repo.
### Notes to the reviewers
I did a bit of extra cleanup on the README file. I replaced the crates list with a table that also includes badges for the crate version and docs. I also renamed the `example-crates` folder to simply `examples`, a nit of mine that I already fixed in the new `bdk_wallet` repo.
See: bitcoindevkit/bdk_wallet#18
### Changelog notice
For the changelog for the next version of `bdk_wallet`:
- The `bdk_wallet` crate has been removed from the `bdk` repository and move to it's own `bdk_wallet` repository.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
feat(chain): Add method for constructing `TxGraph` from a `ChangeSet`
* Added a `from_changeset()` function to `TxGraph` implementation
* This function initializes a new `TxGraph` and applies the given `ChangeSet`
* The method allows constructing a `TxGraph` directly from a `ChangeSet`,
simplifying graph creation.
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
The PR aims to add a `--feerate` option to the psbt `New` command.
Also in this PR:
- bump `bdk_coin_select` to 0.4
- fix collecting `Assets` and enable support for more descriptor types
### Changelog notice
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Automated update to Github CI workflow `cont_integration.yml` by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action
I removed the nightly_docs workflow since it's not needed. The latest published bdk crate docs can be found on docs.rs.
### Notes to the reviewers
We early in the project life we published nightly docs since the API was changing often and releases were infrequent. But now that the api is more stable and we make regular releases nightly docs aren't needed.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
The publish_docs workflow pushes changes to this repository: "bitcoindevkit/bitcoindevkit.org" which requires valid credentials. By setting the persist-credentials to false in #1778, the credentials required are not made available.
### Notes to the reviewers
<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
of the PR were done in a specific way -->
To fix the issue I added a write permission to the publish_jobs, which will allow it to push changes without the credentials.
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
This PR bumps the `check-wasm` job runner image to `ubuntu-24.04`. It was previously running `ubuntu-20.04`, which is set to be unsupported in a 3 weeks (see https://github.com/actions/runner-images/issues/11101); `clang` gets bumped to `clang-14` because of this.
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
This PR allows the receiving structures (`bdk_chain`, `bdk_wallet`) to detect and evict incoming transactions that are double spent (cancelled).
We add a new field to `TxUpdate` (`TxUpdate::evicted_ats`), which in turn, updates the `last_evicted` timestamps that are tracked/persisted by `TxGraph`. This is similar to how `TxUpdate::seen_ats` update the `last_seen` timestamp in `TxGraph`. Transactions with a `last_evicted` timestamp higher than their `last_seen` timestamp are considered evicted.
`SpkWithExpectedTxids` is introduced in `SpkClient` to track expected `Txid`s for each `spk`. During a sync, if any `Txid`s from `SpkWithExpectedTxids` are not in the current history of an `spk` obtained from the chain source, those `Txid`s are considered missing. Support for `SpkWithExpectedTxids` has been added to both `bdk_electrum` and `bdk_esplora` chain source crates.
The canonicalization algorithm is updated to disregard transactions with a `last_evicted` timestamp greater than or equal to their `last_seen` timestamp, except in cases where transitivity rules apply.
### Notes to the reviewers
This PR does not fix #1740 for block-by-block chain source (such as `bdk_bitcoind_rpc`). This work is done in a separate PR (#1857).
### Changelog notice
* Add `TxUpdate::evicted_ats` which tracks transactions that have been replaced and are no longer present in mempool.
* Change `TxGraph` to track `last_evicted` timestamps. This is when a transaction is last marked as missing from the mempool.
* The canonicalization algorithm now disregards transactions with a `last_evicted` timestamp greater than or equal to it's `last_seen` timestamp, except when a canonical descendant exists due to rules of transitivity.
* Add `SpkWithExpectedTxids` in `spk_client` which keeps track of expected `Txid`s for each `spk`.
* Change `bdk_electrum` and `bdk_esplora` to understand `SpkWithExpectedTxids`.
* Add `SyncRequestBuilder::expected_txids_of_spk` method which adds an association between `txid`s and `spk`s.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
志宇 [Fri, 24 Jan 2025 08:45:00 +0000 (19:45 +1100)]
feat(chain)!: Change `TxGraph` to understand evicted-at & last-evicted
The evicted-at and last-evicted timestamp informs the `TxGraph` when the
transaction was last deemed as missing (evicted) from the mempool.
The canonicalization algorithm is changed to disregard transactions with
a last-evicted timestamp greater or equal to it's last-seen timestamp.
The exception is when we have a canonical descendant due to rules of
transitivity.
Update rusqlite_impl to persist `last_evicted`.
Also update docs:
* Remove duplicate paragraphs about `ChangeSet`s.
* Add "Canonicalization" section which expands on methods that require
canonicalization and the associated data types used in the
canonicalization algorithm.
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
- Pin `minreq` to `2.13.2` because `2.13.3` uses the unstable `std::sync::LazyLock` (https://github.com/bitcoindevkit/bdk/pull/1886#issuecomment-2714736611).
- Pin `base64ct` to `1.6.0` for MSRV.
Required for #1885 (`ubuntu-24.04` on all CI workflows).
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
fixes #1873 based on solution proposed by @praveenperera .
### Notes to the reviewers
This is not a breaking change since it's only changing the internal `_marker` field's type.
### Changelog notice
- Fix PersistedWallet to be Send + Sync, even when used with a !Sync persister type such as rusqlite::Connection.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
Steve Myers [Thu, 6 Mar 2025 03:12:02 +0000 (21:12 -0600)]
fix(wallet): allow PersistedWallet<P> to be Send + Sync even if P is !Sync
The goal of this change is to ensure that PersistWallet<P> remains Send and Sync, even if the functions implemented on it
have a &mut P parameter and P is not Sync.
The reason to change PersistWallet<P>'s _marker field type from PhatonData<P> to PhantomData<fn(&mut P)> is to tell the
Rust compiler that this struct is a "consumer" of &mut P, but it does not own P and therefore should not be tied to its
lifetime.
projects / bdk / log