From: Steve Myers <steve@notmandatory.org>
Date: Tue, 18 Feb 2025 16:40:07 +0000 (-0600)
Subject: Merge bitcoindevkit/bdk#1831: Fix out of memory issue caused by many txs.
X-Git-Tag: v0.30.2~1
X-Git-Url: http://internal-gitweb-vhost/?a=commitdiff_plain;h=128a6c89813be24b70b25d2c006e9136d51f20fd;p=bdk

Merge bitcoindevkit/bdk#1831: Fix out of memory issue caused by many txs.

6f8eb034f8d84bb043fad6e5ffd34ea9fc5c0bf0 ci: pin msrv dep version for rustls@0.23.23 (Steve Myers)
7530b22b6fa59832776a224fa087db1ddde9e589 Address out of memory issue caused by many txs. (Joey Guinta)

Pull request description:

  ### Description
  Partially addresses #1827.

  For some wallets there exists a pathological case where we may try to fetch many thousands of transactions at once, which creates enormous memory pressure. By chunking the batch into more reasonably sized sub-queries, we allow time for memory to be freed.

  For Bitkey, we have a user in which the bdk wallet tries to sync ~45000 transactions at once. See the below screenshots the impact of this change via basic memory profiling in our Android app:

  Before:
  <img width="458" alt="Screenshot 2025-02-13 at 10 26 31 AM" src="https://github.com/user-attachments/assets/9cc0ca4d-e8ba-4298-9a10-efc98664f2fb" />

  After:
  <img width="638" alt="Screenshot 2025-02-13 at 10 26 53 AM" src="https://github.com/user-attachments/assets/eb98d332-e838-4489-ab4b-6d3f5bd7b2ab" />

  ### Notes to the reviewers
  I am a Rust noob, so please feel free to suggest alternative implementations.

  Note that https://github.com/bitcoindevkit/bdk/pull/1828 is also needed to successfully sync wallets of this size.

  ### Changelog notice

  Fix out of memory issue caused by many batch fetching many txs

  ### Checklists

  #### All Submissions:

  * [x] I've signed all my commits
  * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
  * [x] I ran `cargo fmt` and `cargo clippy` before committing

  #### Bugfixes:

  * [ ] This pull request breaks the existing API
  * [ ] I've added tests to reproduce the issue which are now passing
  * [x] I'm linking the issue being fixed by this PR

ACKs for top commit:
  evanlinjin:
    Untested ACK 6f8eb034f8d84bb043fad6e5ffd34ea9fc5c0bf0
  notmandatory:
    ACK 6f8eb034f8d84bb043fad6e5ffd34ea9fc5c0bf0

Tree-SHA512: fb2a5247195c0dcc175406ae4a9c372734df52478e4696228b01337abf6d922cbd2e43f5121f07ba171655aa022bd74d1c7cd1642f015bbc13c3422875ea694d
---

128a6c89813be24b70b25d2c006e9136d51f20fd