From: merge-script <hello@evanlinjin.me>
Date: Thu, 3 Jul 2025 03:39:18 +0000 (+0000)
Subject: Merge bitcoindevkit/bdk#1957: feat(electrum): optimize merkle proof validation with... 
X-Git-Tag: bitcoind_rpc-0.21.0~17
X-Git-Url: http://internal-gitweb-vhost/script/%22https:/database/scripts/struct.FeeRate.html?a=commitdiff_plain;h=1039d3cc8f2daa468e8246b272fc28e28087db68;p=bdk

Merge bitcoindevkit/bdk#1957: feat(electrum): optimize merkle proof validation with batching

156cbab67f4ff91276f9f03749944f4c46210f7f test(electrum): Improve benchmark (志宇)
4ea5ea6c490adb0652ad068bc816c2434c51da35 feat(electrum): batch `transaction.get_merkle` calls via `batch_call` (Wei Chen)
ec4fd971c81e7d3bcf8fbdac15c6df63b934cd70 feat(electrum): batched `Header`s and `script_get_history` (Wei Chen)
f21a21d8c8dfc5b84957c854c1f1daee59bcc620 test(electrum): add `criterion` benchmark for `sync` (Wei Chen)
b57768dd2bd25d4fc9d6acc3990b04fa8594e7b1 fix(electrum): improve tx validation and gap limit scanning (keerthi)
7a18cad68be6bda33deaeb50ec9a5307d1ce8f6d feat(electrum): optimize merkle proof validation with batching (Wei Chen)

Pull request description:

  Replaces #1908, originally authored by @Keerthi421.
  Fixes #1891.

  ### Description

  This PR optimizes `sync`/`full_scan` performance by batching and caching key RPC calls to slash network round-trips and eliminate redundant work.

  Key improvements:

  * Gather all `blockchain.transaction.get_merkle` calls into a single `batch_call` request.
  * Use `batch_script_get_history` instead of many individual `script_get_history` calls.
  * Use `batch_block_header` to fetch all needed block headers in one call rather than repeatedly calling `block_header`.
  * Introduce a cache of transaction anchors to skip re-validating already confirmed transactions.

  #### Anchor Caching Performance Improvements

  Results suggest a significant speed up with a warmed up cache. Tested on local Electrum server with:
  ```
  $ cargo bench -p bdk_electrum --bench test_sync
  ```

  Results before this PR (https://github.com/LagginTimes/bdk/tree/1957-master-branch):

  ```
  sync_with_electrum      time:   [1.3702 s 1.3732 s 1.3852 s]
  ```

  Results after this PR:

  ```
  sync_with_electrum      time:   [851.31 ms 853.26 ms 856.23 ms]
  ```
  #### Batch Call Performance Improvements

  No persisted data was carried over between runs, so each test started with cold caches and measured only raw batching performance. Tested with`example_electrum` out of https://github.com/LagginTimes/bdk/tree/example_electrum_timing with the following parameters:

  ```
  $ example_electrum init "tr([62f3f3af/86'/1'/0']tpubDD4Kse29e47rSP5paSuNPhWnGMcdEDAuiG42LEd5yaRDN2CFApWiLTAzxQSLS7MpvxrpxvRJBVcjhVPRk7gec4iWfwvLrEhns1LA4h7i3c2/0/*)#cn4sudyq"
  $ example_electrum scan tcp://signet-electrumx.wakiyamap.dev:50001
  ```

  Results before this PR:

  ```
  FULL_SCAN TIME: 8.145874476s
  ```

  Results after this PR (using this PR's [`bdk_electrum_client.rs`](https://github.com/bitcoindevkit/bdk/blob/70495e2010541acbb5d62f9b5692de20924ac53f/crates/electrum/src/bdk_electrum_client.rs)):

  ```
  FULL_SCAN TIME: 2.594050112s
  ```

  ### Changelog notice

  * Add transaction anchor cache to prevent redundant network calls.
  * Batch Merkle proof, script history, and header requests.

  ### Checklists

  #### All Submissions:

  * [x] I've signed all my commits
  * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
  * [x] I ran `cargo fmt` and `cargo clippy` before committing

  #### New Features:

  * [x] I've added tests for the new feature
  * [x] I've added docs for the new feature

  #### Bugfixes:

  * [ ] This pull request breaks the existing API
  * [x] I've added tests to reproduce the issue which are now passing
  * [x] I'm linking the issue being fixed by this PR

ACKs for top commit:
  oleonardolima:
    tACK 156cbab67f4ff91276f9f03749944f4c46210f7f
  evanlinjin:
    ACK 156cbab67f4ff91276f9f03749944f4c46210f7f

Tree-SHA512: dc7dc1d7de938223cc03293d8bb8ae12c8799c7ec8ba8c7faec5cf2076c96a1b1e50b406cbcc90cbd6cbe7a311c0c11dd036691c03ed067c469a26260903993b
---

1039d3cc8f2daa468e8246b272fc28e28087db68