From: thunderbiscuit Date: Tue, 9 Jan 2024 20:37:58 +0000 (-0500) Subject: Move all tutorials to blog X-Git-Url: http://internal-gitweb-vhost/script/%22https:/database/scripts/trait.ByteDecoder.html?a=commitdiff_plain;h=9079966456fea2558aaf0bac972942510c2d467a;p=bitcoindevkit.org Move all tutorials to blog --- diff --git a/docs/.vuepress/config.js b/docs/.vuepress/config.js index db6926cf4b..49f1d7fd4c 100644 --- a/docs/.vuepress/config.js +++ b/docs/.vuepress/config.js @@ -101,10 +101,6 @@ module.exports = { text: 'Docs', link: '/getting-started/' }, - { - text: 'Tutorials', - link: '/tutorials/hello-world' - }, { text: 'Case Studies', link: '/case-studies' @@ -117,15 +113,6 @@ module.exports = { text: 'Blog', link: '/blog/' }, - { - text: 'Discord', - link: discordUrl - }, - { - text: 'GitHub', - link: githubUrl, - rel: 'noopener noreferrer' - } ], sidebar: { '/_blog/': blogSidebar, diff --git a/docs/_blog/Bitcoin_Core_RPC_Demo.md b/docs/_blog/Bitcoin_Core_RPC_Demo.md new file mode 100644 index 0000000000..9987d2e26f --- /dev/null +++ b/docs/_blog/Bitcoin_Core_RPC_Demo.md @@ -0,0 +1,666 @@ +--- +title: "BDK wallet with Bitcoin core RPC " +description: "Tutorial showing usage of Bitcoin core backend with BDK wallet" +authors: + - Rajarshi Maitra +date: "2021-08-21" +tags: ["tutorial", "BDK", "Bitcoin Core", "RPC", "Wallet"] +hidden: true +draft: false +--- + +## Introduction +BDK wallet developer library can be used to easily deploy wallets with various kinds of blockchain backend support, like [`electrum`](https://github.com/romanz/electrs), [`esplora`](https://github.com/Blockstream/esplora), `compact-filters` ([BIP157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki)) etc. With the latest release of BDK [`v0.10.0`](https://github.com/bitcoindevkit/bdk/releases/tag/v0.10.0), BDK now supports Bitcoin Core as a blockchain backend. BDK talks with Bitcoin Core using rust-bitcoin's [bitcoincore-rpc](https://github.com/rust-bitcoin/rust-bitcoincore-rpc) library. + +This allows wallet devs to quickly deploy their wallet that can talk to a bitcoin full node (home raspi nodes) out of the box. Wallet devs don't need to worry about connecting to a full node with correct RPC calls, all of that is handled by BDK under the hood. All they need is to identify the full node's RPC IP address and the correct RPC credentials. + +In this tutorial we will see how to write a very simplistic wallet code that can connect to a bitcoin core node and maintain its balance and make transactions. + +Unlike other tutorials, we will not use `bdk-cli` tools, but instead write rust code directly using `BDK` devkit. In the end we will end up with our own simple bitcoin wallet. + +## Prerequisite +To run with this tutorial you would need to have a bitcoin core node running in regtest mode. Get the bitcoin core binary either from the [bitcoin core repo](https://bitcoincore.org/bin/bitcoin-core-0.21.1/) or [build from source](https://github.com/bitcoin/bitcoin/blob/v0.21.1/doc/build-unix.md). + +Then configure the node with a following `bitcoin.conf` file +```txt +regtest=1 +fallbackfee=0.0001 +server=1 +txindex=1 +rpcuser=admin +rpcpassword=password +``` + +Apart from that, you would need to install rust in your system. Grab the installation one-liner from [here](https://www.rust-lang.org/tools/install). + +## Setting Up +Create a new cargo binary repository. +```shell +mkdir ~/tutorial +cd tutorial +cargo new bdk-example +cd bdk-example +``` +This will create a new project folder named `bdk-example` with `src/main.rs` and a `cargo.toml`. +```shell +$ tree -L 3 . +. +├── Cargo.toml +└── src + └── main.rs + +1 directory, 2 files +``` +Opening `main.rs` you will see some predefined code like this + +``` rust +fn main() { + println!("Hello, world!"); +} +``` +Try running `cargo run` and if everything is set, you should see "Hello, world!" printed in your terminal +```shell +$ cargo run + Compiling bdk-example v0.1.0 (/home/raj/github-repo/tutorial/bdk-example) + Finished dev [unoptimized + debuginfo] target(s) in 0.95s + Running `target/debug/bdk-example` +Hello, world! +``` +Of course we will not use the given `println!()` statement, but we will put our main code in the `main()` function. + +`cargo new` will also produce a skeleton `Cargo.toml` file like this +```toml +[package] +name = "bdk-example" +version = "0.1.0" +edition = "2018" + +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + +[dependencies] +``` + +## Setting dependencies +Once the rust binary is compiled and running, we now need to specify the dependencies we need to work on our library. + +Remember that BDK provides almost everything we would need to build a wallet out of the box. So we don't need any more dependencies apart from BDK. We will use another small rust crate called [`dirs_next`](https://crates.io/crates/dirs-next) to find our home directory and store wallet files in a subfolder there. + +Add the dependencies into `Cargo.toml` like below +```toml +[package] +name = "bdk-example" +version = "0.1.0" +edition = "2018" + +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + +[dependencies] +bdk = { version = "^0.10", default-features = false, features = ["all-keys", "key-value-db", "rpc"]} +dirs-next = "2.0" +``` +We disabled the default BDK feature (which specifies blockchain backend as an electrum server) and we requested the following features: + - **all-keys**: Adds BIP39 key derivation capabilities + - **key-value-db**: Adds a persistence storage capability + - **rpc**: Adds the RPC blockchain backend capability. + +Now that we have the dependencies added, we can import them in the `main.rs` file to use in our code. +Add the following imports at the start of `main.rs` + +```rust +use bdk::bitcoin::Network; +use bdk::bitcoin::secp256k1::Secp256k1; +use bdk::bitcoin::util::bip32::{DerivationPath, KeySource}; +use bdk::bitcoin::Amount; +use bdk::bitcoincore_rpc::{Auth as rpc_auth, Client, RpcApi}; + +use bdk::blockchain::rpc::{Auth, RpcBlockchain, RpcConfig, wallet_name_from_descriptor}; +use bdk::blockchain::{ConfigurableBlockchain, NoopProgress}; + +use bdk::keys::bip39::{Mnemonic, Language, MnemonicType}; +use bdk::keys::{GeneratedKey, GeneratableKey, ExtendedKey, DerivableKey, DescriptorKey}; +use bdk::keys::DescriptorKey::Secret; + +use bdk::miniscript::miniscript::Segwitv0; + +use bdk::Wallet; +use bdk::wallet::{AddressIndex, signer::SignOptions}; + +use bdk::sled; + +use std::str::FromStr; +``` +With this we are now ready to add our wallet code. + +## Getting Descriptors + +BDK is a descriptor based wallet library. That means when we specify our wallet key-chain we need to tell BDK about it in the format of a descriptor. You can read up on descriptors more [here](https://bitcoindevkit.org/descriptors/). A descriptor string looks like this +`"wpkh([b8b575c2/84'/1'/0'/0]tprv8icWtRzy9CWgFxpGMLSdAeE4wWyz39XGc6SwykeTo13tYm14JkVVQAf7jz8WDDarCgNJrG3aEPJEqchDWeJdiaWpS3FwbLB9SzsN57V7qxB/*)"`. + +This describes a SegwitV0 descriptor of a key derived at path `m/84'/1'/0'/0`. If you already have a descriptor from other sources, you can use that. Otherwise, BDK has your back. BDK can be used to generate a fresh master key with mnemonic, and then derive child keys from it given a specific path. Putting the key in a descriptor is as simple as wrapping it with a `wpkh()` string. + +We will use a dedicated function that will create fresh receive and change descriptors from BDK for our purpose. It will also generate the mnemonic word list for later regenerating the wallet. But we will ignore that for our scope. + +Add a function named `get-descriptor()` below the `main()` function as shown +```rust +fn main() { + ... +} + +// generate fresh descriptor strings and return them via (receive, change) tuple +fn get_descriptors() -> (String, String) { + // Create a new secp context + let secp = Secp256k1::new(); + + // You can also set a password to unlock the mnemonic + let password = Some("random password".to_string()); + + // Generate a fresh mnemonic, and from there a privatekey + let mnemonic: GeneratedKey<_, Segwitv0> = + Mnemonic::generate((MnemonicType::Words12, Language::English)).unwrap(); + let mnemonic = mnemonic.into_key(); + let xkey: ExtendedKey = (mnemonic, password).into_extended_key().unwrap(); + let xprv = xkey.into_xprv(Network::Regtest).unwrap(); + + // Create derived privkey from the above master privkey + // We use the following derivation paths for receive and change keys + // receive: "m/84h/1h/0h/0" + // change: "m/84h/1h/0h/1" + let mut keys = Vec::new(); + + for path in ["m/84h/1h/0h/0", "m/84h/1h/0h/1"] { + let deriv_path: DerivationPath = DerivationPath::from_str(path).unwrap(); + let derived_xprv = &xprv.derive_priv(&secp, &deriv_path).unwrap(); + let origin: KeySource = (xprv.fingerprint(&secp), deriv_path); + let derived_xprv_desc_key: DescriptorKey = + derived_xprv.into_descriptor_key(Some(origin), DerivationPath::default()).unwrap(); + + // Wrap the derived key with the wpkh() string to produce a descriptor string + if let Secret(key, _, _) = derived_xprv_desc_key { + let mut desc = "wpkh(".to_string(); + desc.push_str(&key.to_string()); + desc.push_str(")"); + keys.push(desc); + } + } + + // Return the keys as a tuple + (keys[0].clone(), keys[1].clone()) +} +``` + +To check that the above added function is working as expected, call it in the main function and print the descriptors +``` rust +use ... + +fn main() { + let (receive_desc, change_desc) = get_descriptors(); + println!("recv: {:#?}, \nchng: {:#?}", receive_desc, change_desc); +} + +fn get_descriptors() -> (String, String) { + ... +} +``` +Running the binary should produce the following result +```shell +$ cargo run +recv: "wpkh([89df6a67/84'/1'/0'/0]tprv8iSRXyLtTKJN9qt1jyPVqwhDMEaYztXunPaRQznaH1z8gj8e2g7RnF2ZoHP56VEXwMn76AiV1Je6nJmZbFistwAQCrRGmSrsoKfdqfTDNA1/*)", +chng: "wpkh([89df6a67/84'/1'/0'/1]tprv8iSRXyLtTKJNCECQxBJ19cgx2ueS7mC7GNq7VqTWY3RNPMBY7DfTb9HUnXpJqa14jCJNRmi4yGxfoTVS4WLBXDkvTLq4vujeAD9NfDtSxGP/*)" +``` +Voila! Now we have nice descriptors strings handy to use for our BDK wallet construction. + +## Talking to Bitcoin Core Programmatically +Like all other tutorials we will use two wallets to send coins back and forth. A Bitcoin Core wallet accessible via `bitcoin-cli` command line tools, and a BDK wallet maintained by BDK library. + +But unlike other tutorials, we won't be using `bitcoin-cli` to talk to the Core wallet (we can, but let's spice things up). Instead, we will use the `bitcoin-rpc` library, to talk with our core node listening at `127.0.0.1:18443`, from inside our main function. This will allow us to write code, that will handle both the core and BDK wallet, from inside of the same function, and we won't have to switch terminals! + +Remember we imported `use bdk::bitcoincore_rpc::{Auth as rpc_auth, Client, RpcApi};`? Thats exactly for this purpose. + +Start the `bitcoind` node. + +you should see bitcoind listening at port 18443 +```shell +$ sudo netstat -nptl | grep 18443 +tcp 0 0 0.0.0.0:18443 0.0.0.0:* LISTEN 135532/bitcoind +``` + +Lets create a core rpc interface in our main function. +```rust +fn main() { + ... + + // Create a RPC interface + let rpc_auth = rpc_auth::UserPass( + "admin".to_string(), + "password".to_string() + ); + let core_rpc = Client::new("http://127.0.0.1:18443/wallet/test".to_string(), rpc_auth).unwrap(); + println!("{:#?}", core_rpc.get_blockchain_info().unwrap()); +} +``` +We have provided our RPC authentication `username` and `password` (same as provided in `bitcoin.conf` file). +We have provided the RPC address of our local bitcoin node, with the path to a wallet file, named `test`. And then asked the rpc client to give us the current blockchain info. +If everything goes well, running `cargo run` you should see an output like below: +```shell +$ cargo run +... +GetBlockchainInfoResult { + chain: "regtest", + blocks: 0, + headers: 0, + best_block_hash: 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206, + difficulty: 0.00000000046565423739069247, + median_time: 1296688602, + verification_progress: 1.0, + initial_block_download: true, + ... +``` +Thats it. Now we can programmatically talk to our core node. + +## Get some balance in core wallet. +We have told our rpc client that we would use a wallet named `test`. But currently, our core node doesn't have such a wallet. So let's create the wallet and fund it with some test coins. +```rust +fn main() { + ... + + // Create the test wallet + core_rpc.create_wallet("test", None, None, None, None).unwrap(); + + // Get a new address + let core_address = core_rpc.get_new_address(None, None).unwrap(); + + // Generate 101 blocks and use the above address as coinbase + core_rpc.generate_to_address(101, &core_address).unwrap(); + + // fetch the new balance + let core_balance = core_rpc.get_balance(None, None).unwrap(); + + // Show balance + println!("core balance: {:#?}", core_balance); +} +``` +This will create a wallet in bitcoin core named `test`. generate 101 blocks and use a new address from the wallet as coinbase wallet. Because required coinbase maturity in bitcoin is 100 blocks, by generating 101 blocks, we will have the balance of the first coinbase block reward available for use. +The last `println!()` statement will show the new updated balance as 50 BTC. +```shell +$ cargo run +... +core balance: Amount(50.00000000 BTC) +``` +Great! We now have 50 regtest BTC to play with. + +## Setup the BDK wallet +Now that we are done setting up the core wallet. The last remaining step is to setup the BDK wallet. For this we will use the previous descriptor generation function and write code as below. + +**Note**: You might want to comment out the previous code in `main()`, as running them again will create more coins in core, which isn't an issue, but might be confusing. + +```rust +fn main() { + ... + + // Get receive and change descriptor + let (receive_desc, change_desc) = get_descriptors(); + + // Use deterministic wallet name derived from descriptor + let wallet_name = wallet_name_from_descriptor( + &receive_desc, + Some(&change_desc), + Network::Regtest, + &Secp256k1::new() + ).unwrap(); + + // Create the datadir to store wallet data + let mut datadir = dirs_next::home_dir().unwrap(); + datadir.push(".bdk-example"); + let database = sled::open(datadir).unwrap(); + let db_tree = database.open_tree(wallet_name.clone()).unwrap(); + + // Set RPC username, password and url + let auth = Auth::UserPass { + username: "admin".to_string(), + password: "password".to_string() + }; + let mut rpc_url = "http://".to_string(); + rpc_url.push_str("127.0.0.1:18443"); + + // Setup the RPC configuration + let rpc_config = RpcConfig { + url: rpc_url, + auth, + network: Network::Regtest, + wallet_name, + skip_blocks: None, + }; + + // Use the above configuration to create a RPC blockchain backend + let blockchain = RpcBlockchain::from_config(&rpc_config).unwrap(); + + // Combine everything and finally create the BDK wallet structure + let wallet = Wallet::new(&receive_desc, Some(&change_desc), Network::Regtest, db_tree, blockchain).unwrap(); + + // Sync the wallet + wallet.sync(NoopProgress, None).unwrap(); + + // Fetch a fresh address to receive coins + let address = wallet.get_address(AddressIndex::New).unwrap().address; + + println!("bdk address: {:#?}", address); +} +``` +That's a lot of code. They are divided into logical sections. Let's discuss each step one by one. + - First we used our previous `get_descriptors()` function to generate two descriptor strings. One for generating receive addresses and one for change addresses. + - Then we used a special function from BDK called `wallet_name_from_descriptor()` to derive a name of the wallet from our descriptors. This allows us to have wallet names deterministically linked with descriptors. So in future if we use a different descriptor, the wallet will automatically have a different name. This allows us to not mix wallet names with same descriptor, and given the descriptors we can always determine what was the name we used. It is recommended to derive wallet names like this while using a core backend. Note that this wallet will be created inside the core node. So just like we accessed the `test` wallet, we could also access this wallet. + - Then we created a data directory at path `/home/username/.bdk-example`. We use `dirs_next` to find our home path, and then appended that with `.bdk-example`. All the BDK wallet files will be created and maintained in that directory. In the Database we instructed BDK to create a new `Tree` with `wallet_name`, so given a descriptor, BDK will always know which DB Tree to refer (`Tree` is a `sled` specific term). + - Then like we did previously, we created the rpc username/password authentication, and specified the rpc url. Note that we cannot use the same `rpc_auth` we used before for `core_rpc` as BDK auth and bitcoin-rpc auth are slightly separate structures. + - We combined all this information and created an `RpcConfig` structure. + - We used the rpc configuration to create a `RpcBlockchain` structure. + - Finally we used the Descriptors, Database, and Blockchain to create our final BDK `wallet` structure. + +Now that we have our wallet cooked, in the end, we instructed it to sync with the bitcoin core backend, and fetch us a new address. + +If all goes well, you should see an address printed in the terminal. + +```shell +cargo run + Finished dev [unoptimized + debuginfo] target(s) in 2.99s + Running `target/debug/bdk-example` +bdk address: bcrt1q9vkmujggvzs0rd4z6069v3v0jucje7ua7ap308 +``` + +## Sending Sats Around + +Now that we have covered all the groundwork, we have all we need to send coins back and forth between core and BDK wallet. + +We will keep things simple here and make the following actions + - Send 10 BTC from Core to BDK + - Send back 5 BTC from BDK to Core + - Display balance of two wallets + +In the last line of previous section we got a new address from BDK wallet. We will start from there. Without further discussion lets jump straight into code. + +```rust +fn main() { + ... + + // Fetch a fresh address to receive coins + let address = wallet.get_address(AddressIndex::New).unwrap().address; + + // Send 10 BTC from Core to BDK + core_rpc.send_to_address(&address, Amount::from_btc(10.0).unwrap(), None, None, None, None, None, None).unwrap(); + + // Confirm transaction by generating some blocks + core_rpc.generate_to_address(1, &core_address).unwrap(); + + // Sync the BDK wallet + wallet.sync(NoopProgress, None).unwrap(); + + // Create a transaction builder + let mut tx_builder = wallet.build_tx(); + + // Set recipient of the transaction + tx_builder.set_recipients(vec!((core_address.script_pubkey(), 500000000))); + + // Finalise the transaction and extract PSBT + let (mut psbt, _) = tx_builder.finish().unwrap(); + + // Set signing option + let signopt = SignOptions { + assume_height: None, + ..Default::default() + }; + + // Sign the above psbt with signing option + wallet.sign(&mut psbt, signopt).unwrap(); + + // Extract the final transaction + let tx = psbt.extract_tx(); + + // Broadcast the transaction + wallet.broadcast(tx).unwrap(); + + // Confirm transaction by generating some blocks + core_rpc.generate_to_address(1, &core_address).unwrap(); + + // Sync the BDK wallet + wallet.sync(NoopProgress, None).unwrap(); + + // Fetch and display wallet balances + let core_balance = core_rpc.get_balance(None, None).unwrap(); + let bdk_balance = Amount::from_sat(wallet.get_balance().unwrap()); + println!("core wallet balance: {:#?}", core_balance); + println!("BDK wallet balance: {:#?}", bdk_balance); +} +``` + +The above code segment is mostly straightforward. The only new thing added is `wallet.build_tx()` which returns a `TxBuilder`. BDK allows us to have very fine grained control of cooking up transactions. Almost everything that is possible to do with a Bitcoin transaction can be done in BDK. Here we have a very simple vanilla transaction with no added magic. To get full list of capabilities that `TxBuilder` supports scour its implementation [here](https://github.com/bitcoindevkit/bdk/blob/38d1d0b0e29d38cd370c740d798d96a3c9fcaa1f/src/wallet/tx_builder.rs#L123-L153). + +Finally to step through what we did above: + - We asked core wallet to send 10 BTC to bdk wallet address. + - We confirmed the transaction, and synced the wallet. + - We asked BDK to create a transaction sending 5 BTC to core wallet address. + - We signed and broadcast the transaction. BDK will use the same core node to broadcast the transaction to network. + - We confirmed the transaction by mining a block, and synced the wallet. + - We fetched and displayed balance of both core and BDK wallet. + +If all goes well, you should see the final updated balance as below: +```shell +$ cargo run + Compiling bdk-example v0.1.0 (/home/raj/github-repo/bdk-example/bdk-example) + Finished dev [unoptimized + debuginfo] target(s) in 3.57s + Running `target/debug/bdk-example` +core wallet balance: Amount(144.99998590 BTC) +BDK wallet balance: Amount(4.99999859 BTC) +``` +Voila! We have ~145 BTC (150 - 5) in core wallet and 5 BTC (10 - 5) in BDK wallet. The slight deficiency in the amount are due to transaction fees. Because we are using regtest, the fee is some standard value hardcoded in core node. + +Check out the data directory where BDK has created the wallet data files. + +```shell +$ ls ~/.bdk-example/ +blobs conf db snap.0000000000023CAB +``` + +And finally, this is what the final `main.rs` file looks like. + +```rust +use bdk::bitcoin::Network; +use bdk::bitcoin::secp256k1::Secp256k1; +use bdk::bitcoin::util::bip32::{DerivationPath, KeySource}; +use bdk::bitcoin::Amount; +use bdk::bitcoincore_rpc::{Auth as rpc_auth, Client, RpcApi}; + +use bdk::blockchain::rpc::{Auth, RpcBlockchain, RpcConfig, wallet_name_from_descriptor}; +use bdk::blockchain::{ConfigurableBlockchain, NoopProgress}; + +use bdk::keys::bip39::{Mnemonic, Language, MnemonicType}; +use bdk::keys::{GeneratedKey, GeneratableKey, ExtendedKey, DerivableKey, DescriptorKey}; +use bdk::keys::DescriptorKey::Secret; + +use bdk::miniscript::miniscript::Segwitv0; + +use bdk::Wallet; +use bdk::wallet::{AddressIndex, signer::SignOptions}; + +use bdk::sled; + +use std::str::FromStr; + +fn main() { + // Create a RPC interface + let rpc_auth = rpc_auth::UserPass( + "admin".to_string(), + "password".to_string() + ); + let core_rpc = Client::new("http://127.0.0.1:18443/wallet/test".to_string(), rpc_auth).unwrap(); + + // Create the test wallet + core_rpc.create_wallet("test", None, None, None, None).unwrap(); + + // Get a new address + let core_address = core_rpc.get_new_address(None, None).unwrap(); + + // Generate 101 blocks and use the above address as coinbase + core_rpc.generate_to_address(101, &core_address).unwrap(); + + // Get receive and change descriptor + let (receive_desc, change_desc) = get_descriptors(); + + // Use deterministic wallet name derived from descriptor + let wallet_name = wallet_name_from_descriptor( + &receive_desc, + Some(&change_desc), + Network::Regtest, + &Secp256k1::new() + ).unwrap(); + + // Create the datadir to store wallet data + let mut datadir = dirs_next::home_dir().unwrap(); + datadir.push(".bdk-example"); + let database = sled::open(datadir).unwrap(); + let db_tree = database.open_tree(wallet_name.clone()).unwrap(); + + // Set RPC username and password + let auth = Auth::UserPass { + username: "admin".to_string(), + password: "password".to_string() + }; + + // Set RPC url + let mut rpc_url = "http://".to_string(); + rpc_url.push_str("127.0.0.1:18443"); + + // Setup the RPC configuration + let rpc_config = RpcConfig { + url: rpc_url, + auth, + network: Network::Regtest, + wallet_name, + skip_blocks: None, + }; + + // Use the above configuration to create a RPC blockchain backend + let blockchain = RpcBlockchain::from_config(&rpc_config).unwrap(); + + // Combine everything and finally create the BDK wallet structure + let wallet = Wallet::new(&receive_desc, Some(&change_desc), Network::Regtest, db_tree, blockchain).unwrap(); + + // Sync the wallet + wallet.sync(NoopProgress, None).unwrap(); + + // Fetch a fresh address to receive coins + let address = wallet.get_address(AddressIndex::New).unwrap().address; + + // Send 10 BTC from Core to BDK + core_rpc.send_to_address(&address, Amount::from_btc(10.0).unwrap(), None, None, None, None, None, None).unwrap(); + + // Confirm transaction by generating some blocks + core_rpc.generate_to_address(1, &core_address).unwrap(); + + // Sync the BDK wallet + wallet.sync(NoopProgress, None).unwrap(); + + // Create a transaction builder + let mut tx_builder = wallet.build_tx(); + + // Set recipient of the transaction + tx_builder.set_recipients(vec!((core_address.script_pubkey(), 500000000))); + + // Finalise the transaction and extract PSBT + let (mut psbt, _) = tx_builder.finish().unwrap(); + + // Set signing option + let signopt = SignOptions { + assume_height: None, + ..Default::default() + }; + + // Sign the above psbt with signing option + wallet.sign(&mut psbt, signopt).unwrap(); + + // Extract the final transaction + let tx = psbt.extract_tx(); + + // Broadcast the transaction + wallet.broadcast(tx).unwrap(); + + // Confirm transaction by generating some blocks + core_rpc.generate_to_address(1, &core_address).unwrap(); + + // Sync the BDK wallet + wallet.sync(NoopProgress, None).unwrap(); + + // Fetch and display wallet balances + let core_balance = core_rpc.get_balance(None, None).unwrap(); + let bdk_balance = Amount::from_sat(wallet.get_balance().unwrap()); + println!("core wallet balance: {:#?}", core_balance); + println!("BDK wallet balance: {:#?}", bdk_balance); +} + +// generate fresh descriptor strings and return them via (receive, change) tupple +fn get_descriptors() -> (String, String) { + // Create a new secp context + let secp = Secp256k1::new(); + + // You can also set a password to unlock the mnemonic + let password = Some("random password".to_string()); + + // Generate a fresh menmonic, and from their, a fresh private key xprv + let mnemonic: GeneratedKey<_, Segwitv0> = + Mnemonic::generate((MnemonicType::Words12, Language::English)).unwrap(); + let mnemonic = mnemonic.into_key(); + let xkey: ExtendedKey = (mnemonic, password).into_extended_key().unwrap(); + let xprv = xkey.into_xprv(Network::Regtest).unwrap(); + + // Derive our descriptors to use + // We use the following paths for recieve and change descriptor + // recieve: "m/84h/1h/0h/0" + // change: "m/84h/1h/0h/1" + let mut keys = Vec::new(); + + for path in ["m/84h/1h/0h/0", "m/84h/1h/0h/1"] { + let deriv_path: DerivationPath = DerivationPath::from_str(path).unwrap(); + let derived_xprv = &xprv.derive_priv(&secp, &deriv_path).unwrap(); + let origin: KeySource = (xprv.fingerprint(&secp), deriv_path); + let derived_xprv_desc_key: DescriptorKey = + derived_xprv.into_descriptor_key(Some(origin), DerivationPath::default()).unwrap(); + + // Wrap the derived key with the wpkh() string to produce a descriptor string + if let Secret(key, _, _) = derived_xprv_desc_key { + let mut desc = "wpkh(".to_string(); + desc.push_str(&key.to_string()); + desc.push_str(")"); + keys.push(desc); + } + } + + // Return the keys as a tupple + (keys[0].clone(), keys[1].clone()) +} +``` + +## Conclusion +In this tutorial we saw some very basic BDK wallet functionality with a bitcoin core backend as the source and sync of blockchain data. This is just tip of the iceberg of BDK capabilities. BDK allows flexibility in all the dimensions of a bitcoin wallet, that is key chain, blockchain backend and database management. With all that power, we just implemented a trustless, non-custodial, private bitcoin wallet, backed by a bitcoin full node, with less than 200 lines of code (including lots of comments). + +BDK thus allows wallet devs, to only focus on stuff that they care about, writing wallet logic. All the backend stuff like blockchain, key management, and databases are abstracted away under the hood. + +To find and explore more about the BDK capabilities and how it can fit your development need refer the following resources. + + - [source code](https://github.com/bitcoindevkit/bdk) + - [dev docs](https://docs.rs/bdk/latest/bdk/) + - [community](https://discord.com/invite/d7NkDKm) + + + + + + + + + + + + + + + + + diff --git a/docs/_blog/bdk-cli_basics_multisig_2of3.md b/docs/_blog/bdk-cli_basics_multisig_2of3.md new file mode 100644 index 0000000000..89ced84766 --- /dev/null +++ b/docs/_blog/bdk-cli_basics_multisig_2of3.md @@ -0,0 +1,272 @@ +--- +title: "bdk-cli basics multi-sig 2 of 3 tutorial" +description: "Tutorial using command-line to create a 2 of 3 multi-sig Wallet and Spend" +authors: + - waterst0ne +date: "2022-10-17" +tags: ["tutorial", "bdk-cli","multi-sig"] +hidden: false +draft: false +--- + +## 2-of-3 Multi-Signature Descriptor Wallet using bdk-cli + +## Overview of the tutorial +- The purpose of this tutorial is to continue learning `bdk-cli` as our tool to manage a 2 of 3 multi-signature wallet. +- Generate a receive address with a spending Policy of 2 out of 3 escrow aka multi-signature. +- Intro to more complex but standard policies to create custom encumberances aka custom spending conditions for transactions. + +Note that to complete this tutorial, you'll need to enable the `compiler` and `electrum` flags when installing or building bdk-cli, for example by installing using: +```shell +cargo install bdk-cli --features=compiler,electrum +``` + +## Step 1: Generate the XPRVs (Extended-Keys) and Save to environment variables + +> Create three private keys and each in their own environment variable + +:arrow_forward: `export XPRV_00=$(bdk-cli key generate | jq -r '.xprv')` + +:arrow_forward: `export XPRV_01=$(bdk-cli key generate | jq -r '.xprv')` + +:arrow_forward: `export XPRV_02=$(bdk-cli key generate | jq -r '.xprv')` + + +![](https://i.imgur.com/FwgUdwK.gif) + +### 1a: Verify XPRV environment variables are Active + +:arrow_forward: `env | grep XPRV` + +![](https://i.imgur.com/ZerGPbO.gif) + + +## Step 2: Generate XPUBs (Extended Public Keys) & Save to environment variables + +> Generate the three individual Public Keys aka XPUBs using our Private key and descriptor path. + +:arrow_forward: `export XPUB_00=$(bdk-cli key derive --xprv $XPRV_00 --path "m/84'/1'/0'/0" | jq -r ".xpub")` + +:arrow_forward: `export XPUB_01=$(bdk-cli key derive --xprv $XPRV_01 --path "m/84'/1'/0'/0" | jq -r ".xpub")` + +:arrow_forward: `export XPUB_02=$(bdk-cli key derive --xprv $XPRV_02 --path "m/84'/1'/0'/0" | jq -r ".xpub")` + + +![](https://i.imgur.com/xT3KRh4.gif) + +### 2a: Verify XPUB environment variables + +:arrow_forward: `env | grep XPUB` + +![](https://i.imgur.com/SzAip9E.gif) + +*** +## Step 3: Create Single-Wallet Descriptors + +> Create the wallet Descriptor for each wallet + +:arrow_forward: `export DESCRIPTOR_00="$XPRV_00/84h/1h/0h/0/*"` + +:arrow_forward: `export DESCRIPTOR_01="$XPRV_01/84h/1h/0h/0/*"` + +:arrow_forward: `export DESCRIPTOR_02="$XPRV_02/84h/1h/0h/0/*"` + + +![](https://i.imgur.com/mFrWt6b.png) + + +## Step 4: Create Multi-Sig-Descriptor Wallets +> This is how you create the 2-of-3 multi-sig output descriptor. You will need (one PrivateKey and two Xpubs) It consists of using the `compiler` function to parse `policy` to `mini-script` . + +- When creating the descriptor the order matters so be aware of that when following tutorial if you are for any reason changing the order of the policy. +#### Multi-Sig-Wallet 0 +- [ ] :arrow_forward: `export MULTI_DESCRIPTOR_00=$(bdk-cli compile "thresh(2,pk($DESCRIPTOR_00),pk($XPUB_01),pk($XPUB_02))" | jq -r '.descriptor')` + +#### Multi-Sig-Wallet 1 +- [ ] :arrow_forward: `export MULTI_DESCRIPTOR_01=$(bdk-cli compile "thresh(2,pk($XPUB_00),pk($DESCRIPTOR_01),pk($XPUB_02))" | jq -r '.descriptor')` + +#### Multi-Sig-Wallet 2 +- [ ] :arrow_forward: `export MULTI_DESCRIPTOR_02=$(bdk-cli compile "thresh(2,pk($XPUB_00),pk($XPUB_01),pk($DESCRIPTOR_02))" | jq -r '.descriptor')` + +![](https://i.imgur.com/Yb8RmFS.gif) + +#### multi-sig 2 of 3 policy gets compiled to miniscript +```shell +# policy +thresh(2,pk(XPRV_A),pk(XPUB_B),pk(XPUB_C)) + +# miniscript +wsh(multi(2,XPRV_KEY,PUBKEY_B,XPUB_C)) +``` + + +*** + + +### 4a: Verify Multi-Sig-Descriptor environment variables are active + +:arrow_forward: `env | grep MULTI` + + +![](https://i.imgur.com/aAgtlsi.gif) + +*** + +## Step 5: Generate Receive Address by using Multi-Sig-Descriptor Wallets + +:arrow_forward: `bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 get_new_address` + +:arrow_forward: `bdk-cli wallet --wallet wallet_name_msd01 --descriptor $MULTI_DESCRIPTOR_01 get_new_address` + +:arrow_forward: `bdk-cli wallet --wallet wallet_name_msd02 --descriptor $MULTI_DESCRIPTOR_02 get_new_address` + +![](https://i.imgur.com/w1fxPSn.gif) + + +:red_circle: Did you generate the same address for all three? Good! Else, something might be incorrect. + +## Step 6: Send Testnet Bitcoin to the newly created receive-address + +[Bitcoin Testnet Faucet link:1](https://testnet-faucet.mempool.co) +[Bitcoin Testnet Faucet link:2](https://bitcoinfaucet.uo1.net) + +## Step 7: Sync one of the Multi-Sig Wallets + +:arrow_forward: ` bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 sync` + +![](https://i.imgur.com/GuefgeI.gif) + + +## Step 8: Check Balance Multi-Sig Wallets + + +:arrow_forward: ` bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 get_balance` + +![](https://i.imgur.com/zNciCqF.gif) + + +- Every wallet has access to sync and view balance. + +## Step 9: Check Multi-Sig Policies on Descriptor Wallet +:arrow_forward:` bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 policies` + +The output below confirms the command was successful. +```shell +{ + "external": { + "contribution": { + "conditions": { + "0": [ + {} + ] + }, + "items": [ + 0 + ], + "m": 2, + "n": 3, + "sorted": false, + "type": "PARTIAL" + }, + "id": "seaxtqqn", + "keys": [ + { + "fingerprint": "7cdf2d46" + }, + { + "fingerprint": "fc7870cd" + }, + { + "fingerprint": "26b03333" + } + ], + "satisfaction": { + "items": [], + "m": 2, + "n": 3, + "sorted": false, + "type": "PARTIAL" + }, + "threshold": 2, + "type": "MULTISIG" + }, + "internal": null +} + + +``` + +### SpendingPolicyRequired for complex descriptors + +```shell +--external_policy "{\"seaxtqqn\": [0,1]}" + <-rootnode-> +``` + +> Save the "id": We will need to use this ''id'' later. + +More info on [external policies here](https://bitcoindevkit.org/bdk-cli/interface/) + +## Step 10: Create a Transaction (PSBT) +- 1st Create a PSBT using the first wallet +- 2nd Sign the PSBT with the first wallet +- 3rd Sign PSBT with the second wallet +- Broadcast PSBT + +### Export UNSIGNED_PSBT to environment variable +:arrow_forward: `export UNSIGNED_PSBT=$(bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 create_tx --send_all --to mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt:0 --external_policy "{\"CHANGE_ID_HERE\": [0,1]}" | jq -r '.psbt')` + +### Verify UNSIGNED_PSBT environment variable +:arrow_forward: `env | grep UNSIGNED` +![](https://i.imgur.com/djHaRDq.gif) + +## Step 11: SIGN the Transaction + +### 1st Wallet Signs the transaction + +:arrow_forward: `bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 sign --psbt $UNSIGNED_PSBT` + +:arrow_forward: `export ONESIG_PSBT=$(bdk-cli wallet --wallet wallet_name_msd00 --descriptor $MULTI_DESCRIPTOR_00 sign --psbt $UNSIGNED_PSBT | jq -r '.psbt')` + +:arrow_forward:`env | grep ONESIG` + +``` +{ + "is_finalized": false, + "psbt": "cHNidP8BAFUBAAAAAdYCtva/7Rkt+fgFu3mxAdaPh4uTbgBL3HmYZgcEKWygAAAAAAD/////AQqGAQAAAAAAGXapFDRKD0jKFQ7CuQOBdmC5tosTpnAmiKwAAAAAAAEA6gIAAAAAAQFLyGFJFK884DGBM1WgskRZ6gKp/7oZ+Z30u0+wF3pZYAEAAAAA/v///wKghgEAAAAAACIAINHcOQLE6GpJ3J+FOzn/be+HApxW8sZtGqfA3TBW+NYX91hoOAAAAAAWABTPQDZx2wYYIn+ug2pZBmWBn0Tu/gJHMEQCIHu6GmRMDgPZyTx+klFMA9VujR3qDA/Y08kSkRvOaChjAiBAtExtGAYLuQ/DDJzCqLlNZ1bMB3MV+nxsLfTdI9YcYwEhA0b8lz+kt0xHfR/tjUKOc2Nt2L61pDd5vJ/lsKi8pw9MmFUjAAEBK6CGAQAAAAAAIgAg0dw5AsToakncn4U7Of9t74cCnFbyxm0ap8DdMFb41hciAgIjUCIdnyr6rDtuNhVNt4ZBDcvYLawfoJbzbPyxc/WNDUgwRQIhAJdILr7G3UzYylyr2fA13MFsz/jG4+iZlKeEkX79d082AiA99UF0/uFyXBVNUmuGaxdHL7wlhzqfbgGLMREN0z/O6QEBBWlSIQIjUCIdnyr6rDtuNhVNt4ZBDcvYLawfoJbzbPyxc/WNDSEDzsDXexRPSxeXiLJoS0i2fQlOoOGHmo+Dhaeaq3oHV6YhAjGKA2Dqg+QeMICBAifYslQF2WrehLEQ0iEOpp/+eQ0NU64iBgIjUCIdnyr6rDtuNhVNt4ZBDcvYLawfoJbzbPyxc/WNDRh83y1GVAAAgAEAAIAAAACAAAAAAAAAAAAiBgIxigNg6oPkHjCAgQIn2LJUBdlq3oSxENIhDqaf/nkNDRgmsDMzVAAAgAEAAIAAAACAAAAAAAAAAAAiBgPOwNd7FE9LF5eIsmhLSLZ9CU6g4Yeaj4OFp5qregdXphj8eHDNVAAAgAEAAIAAAACAAAAAAAAAAAAAAA==" +} +``` + +![](https://i.imgur.com/0w4sK5y.gif) + +### 2nd Wallet Signs the transaction +:arrow_forward: `bdk-cli wallet --wallet wallet_name_msd01 --descriptor $MULTI_DESCRIPTOR_01 sign --psbt $ONESIG_PSBT` + +:arrow_forward: `export SECONDSIG_PSBT=$(bdk-cli wallet --wallet wallet_name_msd01 --descriptor $MULTI_DESCRIPTOR_01 sign --psbt $ONESIG_PSBT | jq -r '.psbt')` + + +:arrow_forward:`env | grep SECONDSIG` + +``` +{ + "is_finalized": true, + "psbt": "cHNidP8BAFUBAAAAAdYCtva/7Rkt+fgFu3mxAdaPh4uTbgBL3HmYZgcEKWygAAAAAAD/////AQqGAQAAAAAAGXapFDRKD0jKFQ7CuQOBdmC5tosTpnAmiKwAAAAAAAEA6gIAAAAAAQFLyGFJFK884DGBM1WgskRZ6gKp/7oZ+Z30u0+wF3pZYAEAAAAA/v///wKghgEAAAAAACIAINHcOQLE6GpJ3J+FOzn/be+HApxW8sZtGqfA3TBW+NYX91hoOAAAAAAWABTPQDZx2wYYIn+ug2pZBmWBn0Tu/gJHMEQCIHu6GmRMDgPZyTx+klFMA9VujR3qDA/Y08kSkRvOaChjAiBAtExtGAYLuQ/DDJzCqLlNZ1bMB3MV+nxsLfTdI9YcYwEhA0b8lz+kt0xHfR/tjUKOc2Nt2L61pDd5vJ/lsKi8pw9MmFUjAAEBK6CGAQAAAAAAIgAg0dw5AsToakncn4U7Of9t74cCnFbyxm0ap8DdMFb41hciAgIjUCIdnyr6rDtuNhVNt4ZBDcvYLawfoJbzbPyxc/WNDUgwRQIhAJdILr7G3UzYylyr2fA13MFsz/jG4+iZlKeEkX79d082AiA99UF0/uFyXBVNUmuGaxdHL7wlhzqfbgGLMREN0z/O6QEiAgPOwNd7FE9LF5eIsmhLSLZ9CU6g4Yeaj4OFp5qregdXpkgwRQIhAO2aRERcublhAzToshkZRMg2I8GaE7mM2ECr0vYyuscmAiB5KK4ETlvrLqL0QbcRbGqrSwIa9lVuOqP3f5qCnGRMaQEBBWlSIQIjUCIdnyr6rDtuNhVNt4ZBDcvYLawfoJbzbPyxc/WNDSEDzsDXexRPSxeXiLJoS0i2fQlOoOGHmo+Dhaeaq3oHV6YhAjGKA2Dqg+QeMICBAifYslQF2WrehLEQ0iEOpp/+eQ0NU64iBgIjUCIdnyr6rDtuNhVNt4ZBDcvYLawfoJbzbPyxc/WNDRh83y1GVAAAgAEAAIAAAACAAAAAAAAAAAAiBgIxigNg6oPkHjCAgQIn2LJUBdlq3oSxENIhDqaf/nkNDRgmsDMzVAAAgAEAAIAAAACAAAAAAAAAAAAiBgPOwNd7FE9LF5eIsmhLSLZ9CU6g4Yeaj4OFp5qregdXphj8eHDNVAAAgAEAAIAAAACAAAAAAAAAAAABBwABCP3+AAQASDBFAiEAl0guvsbdTNjKXKvZ8DXcwWzP+Mbj6JmUp4SRfv13TzYCID31QXT+4XJcFU1Sa4ZrF0cvvCWHOp9uAYsxEQ3TP87pAUgwRQIhAO2aRERcublhAzToshkZRMg2I8GaE7mM2ECr0vYyuscmAiB5KK4ETlvrLqL0QbcRbGqrSwIa9lVuOqP3f5qCnGRMaQFpUiECI1AiHZ8q+qw7bjYVTbeGQQ3L2C2sH6CW82z8sXP1jQ0hA87A13sUT0sXl4iyaEtItn0JTqDhh5qPg4Wnmqt6B1emIQIxigNg6oPkHjCAgQIn2LJUBdlq3oSxENIhDqaf/nkNDVOuAAA=" +} +``` + +![](https://i.imgur.com/OdLHnJ3.gif) + +## Step 12: Broadcast Transaction + +:arrow_forward: `bdk-cli wallet --wallet wallet_name_msd01 --descriptor $MULTI_DESCRIPTOR_01 broadcast --psbt $SECONDSIG_PSBT` + +``` +{ + "txid": "61da2451874a483aa8d1d0787c7680d157639f284840de8885098cac43f6cc2f" +} +``` + +![](https://i.imgur.com/M7s0Fd6.gif) + +### Verify Transaction +Verify transcation in the memory pool on testnet [Mempool-testnet!](https://mempool.space/testnet) diff --git a/docs/_blog/bdk_cli_basics.md b/docs/_blog/bdk_cli_basics.md new file mode 100644 index 0000000000..5873d88ec7 --- /dev/null +++ b/docs/_blog/bdk_cli_basics.md @@ -0,0 +1,357 @@ +--- +title: "Command Line introduction to Bitcoin Wallet Development using bdk-cli" +description: "Intro to bdk-cli and wallet dev" +authors: + - waterst0ne +date: "2022-09-22" +tags: ["bdk-cli", "basics", "novice"] +--- +## Tutorial Goals +- The goal for this tutorial is to introduce you to [bdk-cli](https://github.com/bitcoindevkit/bdk-cli), a powerful command-line program. You will be exposed to many of the basic skills that go into creating and managing bitcoin wallets. +- If you've read most of the ["Mastering Bitcoin"](https://github.com/bitcoinbook/bitcoinbook) book, this tutorial could serve as a stepping stone into your Bitcoin wallet development journey. + +- This short tutorial will expose you to the [`bdk library` ](https://docs.rs/bdk/latest/bdk/) and the practical knowledge needed for bitcoin wallet development. As a consequence you will deepen your technical understanding about bitcoin and the bdk library. + +- BDK also has [language-bindings](https://github.com/bitcoindevkit/bdk-ffi) for **Kotlin/Java, Swift, Python** which enable the use of BDK's **Rust** library as an API. You can later use these similar steps to create your own bitcoin mobile, desktop or even WebApp by using the bdk-ffi language bindings. + +*** + +## A few things before you begin: + +- Three things to look out for in each step of the tutorial: + - 1) :arrow_forward: / :large_orange_diamond: - Commands for the Terminal or Shell + - 2) :+1: - Preview of the command output. Note, not all commands will output code. + - 3) Preview Video of the tutorial for reference of what things should look like in action. + +*** +### Outline of Tutorial and Installation notes: + +### Brief Outline of Tutorial +- Step 1: Creating a mnemonic word list + XPRV (Extended Private Key) +- Step 2: Generate testnet Receive Address +- Step 3: Send funds to newly generated address +- Step 4: Sync Wallet +- Step 5: Check Balance of Wallet +- Step 6: Create a Transaction (PSBT) +- Step 7: Sign the Transaction (PSBT) +- Step 8: Broadcast Transaction + +*** +### Rust and Cargo installation: +- [Rust and Cargo Installation](https://rustup.rs/) + +*** +### `bdk-cli` installation: +- Download the [`bdk-cli` github repository locally](https://github.com/bitcoindevkit/bdk-cli.git) + - Enter the folder `cd bdk-cli` + - Install `cargo install --path . --features electrum,repl,compiler ` + - Once installation is done exit and reopen your terminal (command-line interface) + +### Emoji Legend: + +:arrow_forward: : Unix/Linux Commands to copied and pasted +:large_orange_diamond: : Windows Powershell Commands to copied and pasted +:+1: : Output/ preview of code + +*** +## Step 0: Check Version of bdk-cli +:arrow_forward: / :large_orange_diamond: `bdk-cli -V` +:+1: The output below confirms the command was successful. +``` +bdk-cli 0.6.0 +``` + +![](https://i.imgur.com/IcuyeMS.gif) + + +### Preview of bdk-cli help menu +:arrow_forward: / :large_orange_diamond: `bdk-cli --help` +:+1: The output below confirms the command was successful. + +```shell +The BDK Command Line Wallet App + +bdk-cli is a light weight command line bitcoin wallet, powered by BDK. This app can be used as a playground as well as +testing environment to simulate various wallet testing situations. If you are planning to use BDK in your wallet, bdk- +cli is also a great intro tool to get familiar with the BDK API. + +But this is not just any toy. bdk-cli is also a fully functioning bitcoin wallet with taproot support! + +For more information checkout + +USAGE: + bdk-cli [OPTIONS] + +FLAGS: + -h, --help Prints help information + -V, --version Prints version information + +OPTIONS: + -d, --datadir Sets the wallet data directory. Default value : "~/.bdk-bitcoin + -n, --network Sets the network [default: testnet] [possible values: bitcoin, testnet, signet, regtest] + +SUBCOMMANDS: + compile Compile a miniscript policy to an output descriptor + help Prints this message or the help of the given subcommand(s) + key Subcommands for Key operations + repl Options to configure a SOCKS5 proxy for a blockchain client connection + wallet Wallet subcommands that can be issued without a blockchain backend +``` + --- + +## Step 1: Seed Generate + +### 1a: Mnemonic word-list + XPRV (Extended Private Key) :key: + +Linux/Terminal: +:arrow_forward: `bdk-cli key generate | tee key.json` + +Windows Powershell: +:large_orange_diamond: `bdk-cli key generate | Out-File -FilePath "key.json"` + +```shell +{ + "fingerprint": "42b15d2f", + "mnemonic": "party salon worth satoshi envelope suggest garlic dry add pitch throw clap keen narrow antique oyster ketchup purchase gasp visual work venue fog crater", + "xprv": "tprv8ZgxMBicQKsPdwpamtjqMFpYRTafnE1bN2SphLEybCtRKakk6S1TgQCsZgiBwJuJNWe3jYdgVCTsKf9weMxj6tW4zNNKWptykszJpS2L8wE" +} +``` + +![](https://i.imgur.com/ii62Hul.gif) + +*** +### 1b: Save XPRV (Extended Private Key) into environment variable +Linux/Terminal: +:arrow_forward: `export XPRV_00=$(cat key.json | jq -r .xprv)` + +Windows Powershell: + +:large_orange_diamond: `$json = Get-Content -Path .\key.json | ConvertFrom-Json` + +:large_orange_diamond: `$mykeyValue = $json.xprv` + +:large_orange_diamond: `[System.Environment]::SetEnvironmentVariable('XPRV',$mykeyValue, 'Process')` + +![](https://i.imgur.com/KYW2Cdo.gif) + +*** +### 1c: Verify environment variable XPRV_00 is active +Linux/Terminal: +:arrow_forward: `env | grep XPRV` + +Windows Powershell: +:large_orange_diamond: `$env:XPRV` + +![](https://i.imgur.com/ZahbJwe.gif) + +*** +### 1d: Create Descriptor and Save into environment variable + +Linux/Terminal: +:arrow_forward: `export my_descriptor="wpkh($XPRV_00/84h/1h/0h/0/*)"` + +Windows Powershell: +:large_orange_diamond: `[System.Environment]::SetEnvironmentVariable('my_descriptor', "wpkh($env:XPRV/84h/1h/0h/0/*)", 'Process')` + + + +![](https://i.imgur.com/UV4Vgsq.gif) + +*** +### 1e: Verify environment variable my_descriptor is active + +Linux/Terminal: +:arrow_forward: `env | grep my_descriptor` + +Windows Powershell: +:large_orange_diamond: `$env:my_descriptor ` + +![](https://i.imgur.com/s7ZeRQN.gif) + + + +*** +## Step 2: Generate Receive-Address +Linux/Terminal: + +:arrow_forward: `bdk-cli wallet --wallet wallet_name --descriptor $my_descriptor get_new_address` + +Windows Powershell: +:large_orange_diamond:`bdk-cli wallet --descriptor $env:my_descriptor get_new_address` + + + +![](https://i.imgur.com/P8PjTAo.gif) + + + +:+1: The output below confirms the command was successful. + +```shell +{ + "address": "tb1qrh4sq5va0unqtxyfv8al9lz3sna3988cj59uya" +} +``` + +*** +## Step 3: Send testnet bitcoin to the newly created receive-address +Use a faucet to send funds to your newly created address. Here is a link to one: [Bitcoin Testnet Faucet](https://bitcoinfaucet.uo1.net) + +*** +## Step 4: Sync the wallet +Linux/Terminal: +:arrow_forward: ```bdk-cli wallet --wallet wallet_name --descriptor $my_descriptor sync``` + +Windows Powershell: +:large_orange_diamond: ``` bdk-cli wallet --descriptor $env:my_descriptor sync``` + + +:+1: The output below confirms the command was successful. + +```shell +{} +``` + +![](https://i.imgur.com/WFYBgVB.gif) + +*** +## Step 5: Check the balance + +Linux/Terminal: +:arrow_forward: `bdk-cli wallet --wallet wallet_name --descriptor $my_descriptor get_balance ` + +Windows Powershell: +:large_orange_diamond: +`bdk-cli wallet --descriptor $env:my_descriptor get_balance` + +:::tip +Note: The balance will only show after the transaction has been confirmed in a block at least once. +::: + +:+1: The output below confirms the command was successful: +```shell +{ + "satoshi": { + "confirmed": 100000, + "immature": 0, + "trusted_pending": 0, + "untrusted_pending": 0 + } +} +``` + +![](https://i.imgur.com/v8MAYB2.gif) + +*** +## Step 6: Create Transaction (PSBT) + +To create a PSBT (partially-signed-bitcoin-transaction) run the command: + +Linux/Terminal: +:arrow_forward: `bdk-cli wallet --wallet wallet_name --descriptor $my_descriptor create_tx --to tb1qw2c3lxufxqe2x9s4rdzh65tpf4d7fssjgh8nv6:50000` + + +Windows Powershell: +:large_orange_diamond: +` bdk-cli wallet --descriptor $env:my_descriptor create_tx --to tb1qjk6n943uwhqhdf7en600tnwxpslvwtr0udsehp:0 --send_all` +![](https://i.imgur.com/EUCovcJ.gif) + +:+1: The output below confirms the command was successful. + +```shell +{ + "details": { + "confirmation_time": null, + "fee": 113, + "received": 0, + "sent": 123000, + "transaction": null, + "txid": "029173d76253e3441f9dc26f91e6ef30dff486848e91a7941f0cacd0af25ee30" + }, + "psbt": "cHNidP8BAFUBAAAAAak8uMR3UGkAGUKWsq8Mv45qg2fdD93JQRIsa2P0wFloAQAAAAD/////AQfgAQAAAAAAGXapFDRKD0jKFQ7CuQOBdmC5tosTpnAmiKwAAAAAAAEA3gIAAAAAAQFY9sVfEEbyjrHXSlxXDxL+71WOMnsPpVElwk+3E/J9vAAAAAAA/v///wIYZRIAAAAAABYAFBKYf7yF+ss6EFdw2rDZTfdLhep8eOABAAAAAAAWABQd6wBRnX8mBZiJYfvy/FGE+xKc+AJHMEQCIFSIkvEUI9yUgEw4JocRs1aiVsBlKKXrOQaQb3XFqR21AiBqiEVzCVVSRGjckyPDgAQBnOdSzBYR6Rw6KFcCP+E27wEhAwIlXdfM2WYnYa36Hp4MS6YkplBAgBsb1tYG9NiWFWTKzPYhAAEBH3jgAQAAAAAAFgAUHesAUZ1/JgWYiWH78vxRhPsSnPgiBgP80FpaWYQzGzCnNI9blXbei61YpAmtoezMRxpVvBJ6SxgTizKsVAAAgAEAAIAAAACAAAAAAAAAAAAAAA==" +} +``` + +*** +### 6a: export PSBT to environment-variable +Linux/Terminal: + :arrow_forward: `export PSBT="PASTE_PSBT_HERE"` + + +Windows Powershell: +:large_orange_diamond:`[System.Environment]::SetEnvironmentVariable('PSBT',"PASTE_PSBT_HERE",'Process')` +![](https://i.imgur.com/CEDKcPZ.gif) + +*** +## Step 7: Sign Transaction (PSBT) + +Linux/Terminal: +:arrow_forward: ` bdk-cli wallet --wallet wallet_name --descriptor $my_descriptor sign --psbt $PSBT` + +Windows Powershell: +:large_orange_diamond:`bdk-cli wallet --descriptor $env:my_descriptor sign --psbt $env:PSBT` + + +- DON'T FORGET to COPY the PSBT for the next step + +![](https://i.imgur.com/f4o4Ce8.gif) + + +:+1: The output below confirms the command was successful. + + + +```shell +{ + "is_finalized": true, + "psbt": "cHNidP8BAFUBAAAAAak8uMR3UGkAGUKWsq8Mv45qg2fdD93JQRIsa2P0wFloAQAAAAD/////AQfgAQAAAAAAGXapFDRKD0jKFQ7CuQOBdmC5tosTpnAmiKwAAAAAAAEA3gIAAAAAAQFY9sVfEEbyjrHXSlxXDxL+71WOMnsPpVElwk+3E/J9vAAAAAAA/v///wIYZRIAAAAAABYAFBKYf7yF+ss6EFdw2rDZTfdLhep8eOABAAAAAAAWABQd6wBRnX8mBZiJYfvy/FGE+xKc+AJHMEQCIFSIkvEUI9yUgEw4JocRs1aiVsBlKKXrOQaQb3XFqR21AiBqiEVzCVVSRGjckyPDgAQBnOdSzBYR6Rw6KFcCP+E27wEhAwIlXdfM2WYnYa36Hp4MS6YkplBAgBsb1tYG9NiWFWTKzPYhAAEBH3jgAQAAAAAAFgAUHesAUZ1/JgWYiWH78vxRhPsSnPgiAgP80FpaWYQzGzCnNI9blXbei61YpAmtoezMRxpVvBJ6S0gwRQIhALWkBRSJzxuf0od4tPu3qFmEfJ2Y+/QBGtfjSFObWsPeAiA4QJx8Rk5pacrjHv5EOdw6RNHXcdtepFs+m0/Za/h0UQEiBgP80FpaWYQzGzCnNI9blXbei61YpAmtoezMRxpVvBJ6SxgTizKsVAAAgAEAAIAAAACAAAAAAAAAAAABBwABCGwCSDBFAiEAtaQFFInPG5/Sh3i0+7eoWYR8nZj79AEa1+NIU5taw94CIDhAnHxGTmlpyuMe/kQ53DpE0ddx216kWz6bT9lr+HRRASED/NBaWlmEMxswpzSPW5V23outWKQJraHszEcaVbwSeksAAA==" +} +``` + +*** +### 7a: export signed psbt to environment variable +Linux/Terminal: +:arrow_forward: `export SIGNED_PSBT="Paste_PSBT_HERE"` + +Windows Powershell: +:large_orange_diamond: +` $env:PSBTSIGNED = "STRINGHERE"` +![](https://i.imgur.com/VJsl8zR.gif) + +*** +## Step 8: Broadcast Transaction +Linux/Terminal: +:arrow_forward: `bdk-cli wallet --wallet wallet_name --descriptor $my_descriptor broadcast --psbt $SIGNED_PSBT` + +Windows Powershell: +:large_orange_diamond: +` bdk-cli wallet --descriptor $env:my_descriptor broadcast --psbt $env:PSBTSIGNED` + +![](https://i.imgur.com/yQZZk0d.gif) + + +:+1: The output below confirms the command was successful. + +```shell +{ + "txid": "a0877b7ce91ea6d141ba63277673f5bdf0edfdd45f91a39ba1a1ace15f839b52" +} +``` + +- Verify transaction in the memory pool on testnet [Mempool-testnet!](https://mempool.space/testnet) + +:::tip +Run sync one more time and see that the balance has decreased. +::: + +*** + +## Resources +- [BIP-32: Hierarchical Deterministic Wallets](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) +- [BIP: 39 - Mnemonic code for generating deterministic keys](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) +- [BIP: 44 - Multi-Account Hierarchy for Deterministic Wallets](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) +- [BIP: 84 - Derivation scheme for P2WPKH based accounts](https://github.com/bitcoin/bips/blob/master/bip-0084.mediawiki) +- [BIP: 174 - Partially Signed Bitcoin Transaction Format](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki) +- [What are Descriptors and miniscript?](https://blog.summerofbitcoin.org/miniscript-policy-descriptors-hidden-powers-of-bitcoin/) +- [Master Private Key and Extended Private Key](https://bitcoin.stackexchange.com/questions/97242/bip39-tool-bip32-extended-private-key-vs-bip32-root-key) +- [Minsc A Miniscript-based scripting language for Bitcoin contracts](https://min.sc) diff --git a/docs/_blog/bdk_with_tor.md b/docs/_blog/bdk_with_tor.md new file mode 100644 index 0000000000..9651c92b0d --- /dev/null +++ b/docs/_blog/bdk_with_tor.md @@ -0,0 +1,406 @@ +--- +title: "Using BDK with Tor" +description: "How to integrate Tor client to sync BDK wallet with tor enabled blockchain service" +authors: + - rorp +date: "2023-01-03" +tags: ["tutorial", "tor", "wallet", "blockchain"] +--- + +## Introduction + +It’s easy to underestimate the importance of privacy tech for Bitcoin, +especially when connecting to third party services. They can learn your +IP address and associate the transactions you sent over it. You can only +hope that this information will not be leaked anytime in the future with +unpredictable consequences. In order to use Bitcoin privately, you need +to encrypt and anonymize the data you send over the Internet. + +Tor is one of the must-have privacy preserving tools for the Internet in +general, and for Bitcoin in particular. Tor network consists of nodes that +use clever cryptographic methods to encrypt user data and transfer them as +anonymously as possible. + +In this article we show how to integrate Tor with your BDK application. + +## Prerequisite + +First, you would need to have a Tor daemon up and running. + +On Mac OS X you can install with Homebrew. + +```bash +brew install tor +brew services start tor +``` + +On Ubuntu or other Debian-based distributions. + +```bash +sudo apt install tor +``` + +In some cases you'll need to wait a minute or two for the bootstrapping to finish. +In general, Tor is not the fastest network, so if any of the examples below fail +due to timeout, simply restart it. + +At the very end of the article we’ll show how to integrate Tor directly to +your application. + +By default, Tor creates a [SOCKS5](https://en.wikipedia.org/wiki/SOCKS) proxy +endpoint and listens on port 9050. Your application should connect to the +proxy on `localhost:9050` and use it for its network activities. + +## Setting Up + +Create a new cargo project. + +```bash +mkdir ~/tutorial +cd tutorial +cargo new bdk-tor +cd bdk-tor +``` + +Open `src/main.rs` file remove all its contents and add these lines. + +```rust +use std::str::FromStr; +use bdk::bitcoin::util::bip32; +use bdk::bitcoin::util::bip32::ExtendedPrivKey; +use bdk::bitcoin::Network; +use bdk::database::MemoryDatabase; +use bdk::template::Bip84; +use bdk::{KeychainKind, SyncOptions, Wallet}; + +// add additional imports here + +fn main() { + let network = Network::Testnet; + + let xpriv = "tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy"; + + let xpriv = bip32::ExtendedPrivKey::from_str(xpriv).unwrap(); + + let blockchain = create_blockchain(); + + let wallet = create_wallet(&network, &xpriv); + + println!("Syncing the wallet..."); + + wallet.sync(&blockchain, SyncOptions::default()).unwrap(); + + println!( + "The wallet synced. Height: {}", + blockchain.get_height().unwrap() + ); +} + +fn create_wallet(network: &Network, xpriv: &ExtendedPrivKey) -> Wallet { + Wallet::new( + Bip84(*xpriv, KeychainKind::External), + Some(Bip84(*xpriv, KeychainKind::Internal)), + *network, + MemoryDatabase::default(), + ) + .unwrap() +} +``` + +In this code we create a testnet wallet with `create_wallet()` function and +try to sync it with a specific blockchain client implementation. We create a +blockchain client using `create_blockchain()` function. We’ll implement it +later for each type of blockchain client supported by BDK. + +## ElectrumBlockchain + +The Electrum client is enabled by default so the `Cargo.toml` dependencies +section will look like this. + +```toml +[dependencies] +bdk = { version = "^0.26"} +``` + +And the imports look like this. + +```rust +use bdk::blockchain::{ElectrumBlockchain, GetHeight}; +use bdk::electrum_client::{Client, ConfigBuilder, Socks5Config}; +``` + +Here is the implementation of `create_blockchain()` function for the +Electrum client. + +```rust +fn create_blockchain() -> ElectrumBlockchain { + let url = "ssl://electrum.blockstream.info:60002"; + let socks_addr = "127.0.0.1:9050"; + + println!("Connecting to {} via {}", &url, &socks_addr); + + let config = ConfigBuilder::new() + .socks5(Some(Socks5Config { + addr: socks_addr.to_string(), + credentials: None, + })) + .unwrap() + .build(); + + let client = Client::from_config(url, config).unwrap(); + + ElectrumBlockchain::from(client) +} +``` + +In this example we create an instance of `Socks5Config` which defines the +Tor proxy parameters for `ElectrumBlockchain`. + +## Blocking EsploraBlockchain + +The blocking version of `EsploraBlockchain` uses `ureq` crate to send HTTP +requests to Eslora backends. By default, its SOCKS5 feature is disabled, +so we need to enable it in `Cargo.toml`. + +```toml +[dependencies] +bdk = { version = "^0.26", default-features = false, features = ["use-esplora-blocking"]} +``` + +The imports are + +```rust +use bdk::blockchain::{EsploraBlockchain, GetHeight}; +use bdk::blockchain::esplora::EsploraBlockchainConfig; +use bdk::blockchain::ConfigurableBlockchain; +``` + +And `create_blockchain()` implementation is + +```rust +fn create_blockchain() -> EsploraBlockchain { + let url = "http://mempoolhqx4isw62xs7abwphsq7ldayuidyx2v2oethdhhj6mlo2r6ad.onion/testnet/api"; + let socks_url = "socks5://127.0.0.1:9050"; + + println!("Connecting to {} via {}", &url, &socks_url); + + let config = EsploraBlockchainConfig { + base_url: url.into(), + proxy: Some(socks_url.into()), + concurrency: None, + stop_gap: 20, + timeout: Some(120), + }; + + EsploraBlockchain::from_config(&config).unwrap() +} +``` + +Here we use `proxy()` method of the config builder to set the Tor proxy +address. Please note, that unlike the previous examples, the Esplora client +builder requires not just a proxy address, but a URL +“socks5://127.0.0.1:9050”. + +## Asynchronous EsploraBlockchain + +There’s no need in enabling SOCKS5 for the asynchronous Esplora client, +so we are good to go without additional dependencies. + +```toml +[dependencies] +bdk = { version = "^0.26", default-features = false, features = ["use-esplora-async"]} +``` + +The imports are the same as in previous example. + +```rust +use bdk::blockchain::{EsploraBlockchain, GetHeight}; +use bdk::blockchain::esplora::EsploraBlockchainConfig; +use bdk::blockchain::ConfigurableBlockchain; +``` + +`create_blockchain()` is almost identical. + +```rust +fn create_blockchain() -> EsploraBlockchain { + let url = "http://mempoolhqx4isw62xs7abwphsq7ldayuidyx2v2oethdhhj6mlo2r6ad.onion/testnet/api"; + let socks_url = "socks5h://127.0.0.1:9050"; + + println!("Connecting to {} via {}", &url, &socks_url); + + let config = EsploraBlockchainConfig { + base_url: url.into(), + proxy: Some(socks_url.into()), + concurrency: None, + stop_gap: 20, + timeout: Some(120), + }; + + EsploraBlockchain::from_config(&config).unwrap() +} +``` + +There are two notable differences though. First, we call `build_async()` to +create an asynchronous Esplora client. Second the SOCKS5 URL scheme is +“socks5h”. It’s not a typo. The async client supports two SOCKS5 schemes +“socks5” and “socks5h”. The difference between them is that the former +makes the client to resolve domain names, and the latter does not, so the +client passes them to the proxy as is. A regular DNS resolver cannot +resolve Tor onion addresses, so we should use “socks5h” here. + +## CompactFiltersBlockchain + +Add these lines to the dependencies section of `Cargo.toml` file to enable +BIP-157/BIP-158 compact filter support. + +It can take a while to sync a wallet using compact filters over Tor, so be +patient. + +```toml +[dependencies] +bdk = { version = "^0.26", default-features = false, features = ["compact_filters"]} +``` + +Now add the required imports into `src/main.rs`. + +```rust +use std::sync::Arc; +use bdk::blockchain::compact_filters::{Mempool, Peer}; +use bdk::blockchain::{CompactFiltersBlockchain, GetHeight}; +``` + +`create_blockchain()` function will look like this. + +```rust +fn create_blockchain() -> CompactFiltersBlockchain { + let peer_addr = "neutrino.testnet3.suredbits.com:18333"; + let socks_addr = "127.0.0.1:9050"; + + let mempool = Arc::new(Mempool::default()); + + println!("Connecting to {} via {}", peer_addr, socks_addr); + + let peer = + Peer::connect_proxy(peer_addr, socks_addr, None, mempool, Network::Testnet).unwrap(); + + CompactFiltersBlockchain::new(vec![peer], "./wallet-filters", Some(500_000)).unwrap() +} +``` + +Here we use `Peer::connect_proxy()` which accepts the address to the SOCKS5 +proxy and performs all the heavy lifting for us. + +## Integrated Tor daemon + +As an application developer you don’t have to rely on your users to install +and start Tor to use your application. Using `libtor` crate you can bundle +Tor daemon with your app. + +`libtor` builds a Tor binary from the source files. Since Tor is written in C +you'll need a C compiler and build tools. + +Install these packages on Mac OS X: + +```bash +xcode-select --install +brew install autoconf +brew install automake +brew install libtool +brew install openssl +brew install pkg-config +export LDFLAGS="-L/opt/homebrew/opt/openssl/lib" +export CPPFLAGS="-I/opt/homebrew/opt/openssl/include" +``` + +Or these packages on Ubuntu or another Debian-based Linux distribution: + +```bash +sudo apt install autoconf automake clang file libtool openssl pkg-config +``` + +Then add these dependencies to the `Cargo.toml` file. + +```toml +[dependencies] +bdk = { version = "^0.26" } +libtor = "47.8.0+0.4.7.x" +``` + +This is an example of how we can use `libtor` to start a Tor daemon. + +```rust +use std::fs::File; +use std::io::prelude::*; +use std::thread; +use std::time::Duration; + +use libtor::LogDestination; +use libtor::LogLevel; +use libtor::{HiddenServiceVersion, Tor, TorAddress, TorFlag}; + +use std::env; + +pub fn start_tor() -> String { + let socks_port = 19050; + + let data_dir = format!("{}/{}", env::temp_dir().display(), "bdk-tor"); + let log_file_name = format!("{}/{}", &data_dir, "log"); + + println!("Staring Tor in {}", &data_dir); + + truncate_log(&log_file_name); + + Tor::new() + .flag(TorFlag::DataDirectory(data_dir.into())) + .flag(TorFlag::LogTo( + LogLevel::Notice, + LogDestination::File(log_file_name.as_str().into()), + )) + .flag(TorFlag::SocksPort(socks_port)) + .flag(TorFlag::Custom("ExitPolicy reject *:*".into())) + .flag(TorFlag::Custom("BridgeRelay 0".into())) + .start_background(); + + let mut started = false; + let mut tries = 0; + while !started { + tries += 1; + if tries > 120 { + panic!( + "It took too long to start Tor. See {} for details", + &log_file_name + ); + } + + thread::sleep(Duration::from_millis(1000)); + started = find_string_in_log(&log_file_name, &"Bootstrapped 100%".into()); + } + + println!("Tor started"); + + format!("127.0.0.1:{}", socks_port) +} +``` + +First, we create a Tor object, and then we call `start_background()` method +to start it in the background. After that, we continuously try to find +“Bootstrapped 100%” string in the log file. Once we find it, Tor is +ready to proxy our connections. We use port 19050 because, the user can +have their own instance of Tor running already. + +Next you can modify `create_blockchain()` like this + +```rust +fn create_blockchain() -> ElectrumBlockchain { + let url = "ssl://electrum.blockstream.info:60002"; + let socks_addr = start_tor(); + + ... +} +``` + +In this example we start Tor first, then use the address returned by +`start_tor()` function as proxy address. + +We omitted `find_string_in_log()` and `truncate_log()` for brevity. You +can find their implementations in [esplora_backend_with_tor.rs](https://github.com/bitcoindevkit/bdk/blob/master/examples/esplora_backend_with_tor.rs) diff --git a/docs/_blog/compact_filters_demo.md b/docs/_blog/compact_filters_demo.md new file mode 100644 index 0000000000..0bd7aab50d --- /dev/null +++ b/docs/_blog/compact_filters_demo.md @@ -0,0 +1,384 @@ +--- +title: "BDK wallet as a BIP157 SPV light client" +description: "Tutorial showing usage of compact filters (BIP157) using bdk-cli command line tools" +authors: + - Rajarshi Maitra +date: "2021-06-20" +tags: ["tutorial", "BDK", "bdk-cli", "compact_filters", "BIP157", "Neutrino"] +--- + +## Introduction + +### Compact Filters: +Compact filters are the latest specification of Bitcoin SPV node implementation as per [BIP157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) and [BIP158](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki). Such light clients were envisioned by Satoshi himself in his original white paper, but due to lack of robust privacy and trust guarantees using conventional [bloomfilters](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki), these type of nodes never got popular. + +Enters [BIP157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki), which described a new type of filters for Bitcoin Blockchain data, known as `compact_filters`. The [Neutrino](https://github.com/lightninglabs/neutrino) project pioneered the use of compact filter based light client nodes for using with Lightning Network wallets. Using compact filters, a light-node can talk to one or more full nodes, and fetch relevant information from the blockchain, with much more robust privacy and security guarantees than previously possible. Compact filter based nodes are best suitable to be used with mobile wallets, to create more trustless mobile applications on Bitcoin. Any wallet application that needs to have an "eye on the blockchain" has an use for such light clients. + +`BIP157` type filters allows to create tiny sized SPV nodes, that can fetch blockchain data and can identify inconsistency, so it can actively defend itself, while also preserving its privacy. Such nodes are most useful for Lightning Network mobile applications. + +Example of such `compact_filters` wallets in wild is [Breeze](https://github.com/breez/breezmobile) Lightning mobile wallet. + +Bitcoin core supports serving `BIP157` type filters from `v0.21.0`. + +### BDK and Compact filters +BDK is a bitcoin wallet development library that can be used to create bitcoin wallets with custom `Database` and `Blockchain` backends. BDK is a [descriptor](https://bitcoindevkit.org/descriptors/) based wallet, i.e. the wallet keychain is described by a set of descriptors. + +Using BDK one can instantiate wallets of various kinds as per requirement. BDK abstracts away all the heavy lifting works, and allow wallet devs to concentrate on logic that they care about, i.e. writing wallet codes. For more detailed documentation on BDK capabilities check these [blog](https://bitcoindevkit.org/blog/2020/12/hello-world/), [bog](https://bitcoindevkit.org/blog/2020/11/descriptors-in-the-wild/) and [docs](https://docs.rs/bdk/). + +The main three components of abstraction in BDK are + - `Database` + - `Descriptors` + - `Blockchain` + +BDK comes with default implementations of all them that developers can start with out of the box. Developers can also create their own custom implementations and plug it into BDK (thanks to rust magic of `Traits`). + +BDK also supports [BIP158](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki) communication protocol, which allows creation of `BIP157` type compact filter SPV nodes. This capability is extended to wallet with BDK's `Blockchain` data structure. The [API](https://docs.rs/bdk/latest/bdk/blockchain/trait.Blockchain.html) for `compact_filters` backend is similar to any other kind of backends, so wallet devs don't need to worry about all the details. Its ok if the dev haven't even heard of `BIP157`, BDK takes care of that in background. + +This capability can be unlocked by compiling BDK with the `compact_filters` feature. Once enabled, BDK will be able to create wallets with the `compact_filters` type `Blockchain` backend. (The default backend is electrum server) + +### bdk-cli +`bdk-cli` is a lightweight [REPL](https://codewith.mu/en/tutorials/1.0/repl) wrapper over the BDK library to facilitate quick and easy demonstration of BDK capabilities in command-line. Wallet devs can use this tool to quickly try out different possibilities with BDK. + +In this tutorial, We will use `bdk-cli` to demonstrate some basic wallet functionalities using `compact_filters` backend. + +## Tutorial Scope +Basic wallet workflow we will cover: + + - create and sync a wallet, + - receive a transaction, + - create a transaction, + - sign and broadcast the transaction, + - fetch updated balance, + +The BDK wallet will have a `BIP157` SPV backend (aka `compact_filters` backend) that will connect with a Bitcoin core node serving filter data. + +It will publish and extract transaction data through that node. + +We will have a Bitcoin Core wallet and a BDK wallet, sending and receiving transactions between each other, in regtest. + +## Prerequisites +Following things are required to start with the tutorial. + +1. A Bitcoin Core regtest node listening at `localhost:18444` signalling for compact filter support. +2. `bdk-cli` compiled with `compact_filter` features. + +If you already have these two setup and working, you can skip this and jump to the [Tutorial](#tutorial) section. + +### Install and run `bitcoind` +You can definitely do it with your own `bitcoind` installation. `BIP157` support has been included in Bitcoin Core `v0.21.0`. So anything above that will work. + +You also need to ensure proper configuration settings for signalling `compact_filters` support. + +For ease of testing, the BDK project hosts docker images that can be used to spawn Bitcoin Core with all the relevant configurations. + +- spawn a regtest node using [bitcoin-regtest-box](https://github.com/bitcoindevkit/bitcoin-regtest-box) docker file. + + Start the regtest box docker container. + + ```shell + $ docker run --detach --rm -p 127.0.0.1:18443-18444:18443-18444/tcp --name bdk-box bitcoindevkit/bitcoind + ``` + This will spin up a docker container running `bicoind` and listening to port `18444` and `18333`. You can keep this terminal alive to see communication events with BDK and the node. + +- Check node is reachable + + In another terminal try connecting to the node with `bitcoin-cli` + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest getnetworkinfo + { + "version": 210000, + "subversion": "/Satoshi:0.21.1/", + "protocolversion": 70016, + "localservices": "0000000000000449", + "localservicesnames": [ + "NETWORK", + "WITNESS", + "COMPACT_FILTERS", + "NETWORK_LIMITED" + ... + ], + } + + ``` + In the output, the `version` should show `210000`. `localservicesnames` should contain `"COMPACT_FILTERS"`. If you see this, then Bitcoin Core is correctly configured. + +### Install and run bdk-cli +- Install `bdk-cli` with `compact_filters` feature + + ```shell + $ cargo install --git https://github.com/bitcoindevkit/bdk-cli.git bdk-cli --features compact_filters + ``` +- Check installation + ```shell + $ bdk-cli --help + ... + USAGE: + bdk-cli [OPTIONS] + FLAGS: + -h, --help Prints help information + -V, --version Prints version information + OPTIONS: + -n, --network Sets the network [default: testnet] + + SUBCOMMANDS: + help Prints this message or the help of the given subcommand(s) + key Key management sub-commands + repl Enter REPL command loop mode + wallet Wallet options and sub-commands + ``` +Once these are setup correctly, you can start with the tutorial next. + + + +## Tutorial +[Note: For brevity `bdk-cli` results are stored in command line variables using `jq` tool. It is recommended to check the full results to see different information returned by `bdk-cli` commands.] + +### Bitcoin Core Wallet Generation + +This is standard procedure with `bitcoin-cli`. + +- Create a wallet and generate 101 blocks. + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest createwallet test + { + "name": "test", + "warning": "" + } + ``` + + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest getnewaddress + bcrt1qatd7yq0jukwusuaufltlejmeydpvnpv43r5gc2 + ``` + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest generatetoaddress 101 bcrt1qatd7yq0jukwusuaufltlejmeydpvnpv43r5gc2 + [ + "3813ed6eb716f4743b9657d918799acf743add985a8ded28d8aa3629dd4496b6", + "70da855913bdf791b6e458c611cebdef79b7a9840eb103ce58c71c1c7e3c49bc", + "682ca732ef72719cd6f82c5047c7690fb1cd2df2543d035ac4ea99e974b8d172", + "78799e4771017d4f46aa3c240054e2d61f54cea07ec44cb18ae712761e0aaa1e", + ... + ] + ``` + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest getbalance + 50.00000000 + ``` + Now the core wallet has generated new blocks and is funded with test bitcoin. + + +### BDK Wallet Generation +BDK is a descriptor based wallet library. So in order to use it we will need some descriptors to work with. + +BDK wallet will ask for two descriptors as input, corresponding to `receive` and `change` addresses. Its recommended to have these two descriptors separate as BDK will handle them separately and ensure `change` addresses are never used for receiving funds. + +Or developers can decide to use a single descriptor too, in that case BDK will use that descriptor for deriving both `receive` and `change` addresses. + +We will use `bdk-cli` itself to generate such descriptors. + +- #### Generate a privatekey + ```shell + $ BDK_xprv=$(bdk-cli key generate | jq -r '.xprv') + $ echo $BDK_xprv + tprv8ZgxMBicQKsPefY7tdq7EKny81n9tfSvUYfSHAZByXdjPAZVysvaB6sFd2YavqfqMBgbHaXUG5oWM6sYvdJn6vnUizzQKTYAJ36bQsfPv4N + ``` + `bdk-cli key generate` will generate a fresh master key with `mnemonic` and `xprv`. We have extracted the value of extended private key and stored it in `BDK_xprv` variable. + + The returned `mnemonic` can be used to restore back the wallet if wallet data directory is lost. + +- #### Generate Descriptors + `bdk-cli key derive` can derive an `xpub`s given a `master key` and `derivation_path`. + + We will use the following paths for our `receive` and `change` descriptors + + - `receive` path: `m/84h/1h/0h/0` + - `change` path: `m/84h/1h/0h/1`, + + We can then simply wrap them in a `"wpkh()"` to create our descriptors string and store them. + + When asked for a new address, BDK will derive one from the `receive` descriptor. + + And while constructing transaction, BDK will use the `change` descriptor to derive change address. + + ```shell + $ BDK_recv_desc="wpkh($(bdk-cli key derive --path m/84h/1h/0h/0 --xprv $BDK_xprv | jq -r '.xprv'))" + $ echo $BDK_recv_desc + wpkh([ff09c7c9/84'/1'/0'/0]tprv8hkdEGgwLLnqsdfkJFidpTj5d6z5qFdP6Qwzsviea3HrS9C2mXXaDivPKCCgcaWvnGNX9eciLUQs91PWYXJqrChfnAagViCgG6L5phaNyWr/*) + ``` + ```shell + $ BDK_chng_desc="wpkh($(bdk-cli key derive --path m/84h/1h/0h/1 --xprv $BDK_xprv | jq -r '.xprv'))" + $ echo $BDK_chng_desc + wpkh([ff09c7c9/84'/1'/0'/1]tprv8hkdEGgwLLnqtbYkGG7fSy7v43RF2SQGGjNuZtmBzEHh7H8xgpXBETQAbVPqi8rkvLNFKLYY4rDzXA4fn5Ha1yuazZqhQPe3uNKmFS7648s/*) + ``` + Note: `BDK_xprv` has been used as the `master key`, this will allow BDK to have signing capabilities. + We could have used an `xpub` master key here instead, that would create an `watch-only` wallet. + +- #### Create and Sync a wallet + We will now instruct BDK to create a new wallet with following instructions + + ```shell + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc sync + {} + ``` + - name (`--wallet`) `bdk-test`, + - `receive` descriptor (`-d`) as `$BDK_recv_desc` and change descriptor (`-c`) as `$BDK_chng_desc`, + - connected to a full node (`--node`) listening at `127.0.0.1:18444`, + - and finally create and sync the wallet with the `sync` command. + + If you are using a `regtest` node, also add `--network regtest`, the default is `testnet`. + + `bdk-cli` makes multiple parallel connections that can be configured with the `--conn-count` parameter (default is 4). This makes syncing parallel and fast. Use `bdk-cli --help` to see all other options. + + Getting an empty return means wallet creation succeeded. + + BDK has created a wallet named `bdk-test` in its data directory. Which is by default stored at `~/.bdk-bitcoin/compact_filters` folder. + + Looking into that folder different files and directories maintained by BDK can be seen. + ```shell + $ ls .bdk-bitcoin/compact_filters/ + 000004.log CURRENT LOCK MANIFEST-000003 OPTIONS-000010 + bdk-test IDENTITY LOG OPTIONS-000008 + ``` +### Recieve Coins + +We will use the `core` wallet to send 5 BTC to our`bdk-test` wallet. + +- Fetch a new address using `bdk-cli` + ```shell + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc get_new_address + { + "address": "bcrt1qx2479wywulf50pqx5uy64zhxq9f3tuvlh8u0s9" + } + ``` + +- Transfer funds to the previous address and generate a block, using `bitcoin-cli` + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest sendtoaddress bcrt1qx2479wywulf50pqx5uy64zhxq9f3tuvlh8u0s9 5 + + + $ docker exec -it bdk-box /root/bitcoin-cli -regtest generatetoaddress 1 bcrt1qw3ht9xtc9pgyvmqay0ap9fw8mxd27az8el0uz3 + ``` + + `core` has sent 5 BTC to our `bdk-test` wallet. Which is confirmed in a new block. + + `bdk-test` can see that now by syncing again. + + (Note: BDK required explicit `sync()` calls to give wallet developers flexibility on when to sync). + ```shell + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc sync + {} + + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc get_balance + { + "satoshi": 500000000 + } + ``` + + We can see `500000000` sats balance in our `bdk-test` wallet. + + BDK has fetched blockchain details concerning its wallet descriptors, from the core node, using compact filters. + +### Creating a transaction. + Now we want to create a transaction sending coins from `bdk-test` wallet to the `core` wallet. + +- fetch a new `core` address + ```shell + $ core_addrs=$(docker exec -it bdk-box /root/bitcoin-cli -regtest getnewaddress | tr -d '\r') + ``` + +- Create a raw transaction using `bdk-cli` to the above address. This will generate a `psbt` which we will sign. + ```shell + $ psbt=$(bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc create_tx --to $core_addrs:200000000 | jq -r '.psbt') + ``` + (Recommended to check all the other information returned by `bdk-cli create_tx`) + +### Sign and Broadcast the transaction +Asking BDK to sign a transaction is as straight forward as it can get. BDK already holds the `xprv` deatils to sign a transaction. It returns a finalised `signed_psbt` which we will next broadcast to the network. + +- Sign the transaction + ```shell + $ signed_psbt=$(bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc sign --psbt $psbt | jq -r '.psbt') + ``` + +- Broadcast the transaction + ```shell + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc broadcast --psbt $signed_psbt + { + "txid": "c343f5b25372e285308eba912d1fe8fade9f64afde6d95306e248e52e0852252" + } + ``` + This makes BDK broadcast the transaction via the connected core node, and it returns the corresponding Txid. + +### Confirming the Transaction + The transaction has been received by the `core` node and waiting in its mempool for inclusion in block. + We can see the transaction via its `txid` received in previous step. + +- Check transaction in mempool + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest gettransaction c343f5b25372e285308eba912d1fe8fade9f64afde6d95306e2248e52e0852252 + { + "amount": 2.00000000, + "confirmations": 0, + "trusted": false, + "txid": "c343f5b25372e285308eba912d1fe8fade9f64afde6d95306e248e52e0852252", + "walletconflicts": [ + ], + "time": 1621697202, + "timereceived": 1621697202, + "bip125-replaceable": "no", + "details": [ + { + "address": "bcrt1q3h4hs6mve5dcl7da3d4acmlp20hh8c3t4mldwe", + "category": "receive", + "amount": 2.00000000, + "label": "", + "vout": 1 + } + ], + "hex": "01000000000101d84e8cb7477f9fe6f265b56d5416ff47da9a70be18f65ec50731b8257c67f2bd0100000000ffffffff0273a2e11100000000160014874270187001febc4cebd8cb083cf2c783e8f1ac00c2eb0b000000001600148deb786b6ccd1b8ff9bd8b6bdc6fe153ef73e22b0247304402201037d9ef5b80392296311c8899b1f12a0987778d694a442a88bafa6fbd7a7c9a022011293176255897444d9c71b0b9cd13b2aedb749b142577566c90a63d61025e2c01210202427d16b29c1c8546255363a74326ee9ab3196770bb3fccc7b679d52f9c1ccf00000000" + } + ``` + This means, core has recieved the transaction in its mempool and waiting for confirmation. + +- Generate 1 block to confirm the transaction + ```shell + $ docker exec -it bdk-box /root/bitcoin-cli -regtest generatetoaddress 1 bcrt1qatd7yq0jukwusuaufltlejmeydpvnpv43r5gc2 + [ + "55436ff0169bbb3e70ab10cb7cdd45ab86204d5d7864a109142d91120d023197" + ] + ``` + +- Sync the `bdk-test` wallet and ask for available balance. + ```shell + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc sync + {} + + $ bdk-cli --network regtest wallet --node "127.0.0.1:18444" --wallet bdk-test -d $BDK_recv_desc -c $BDK_chng_desc get_balance + { + "satoshi": 299999859 + } + ``` + + If you see the balance updated, voila! + + What happened here is: + - core created a new block containing the transaction. + - `bdk-cli` fetched the corresponding filter data. + - It noticed it got a concerning transaction. + - It asked for the details of that transaction from the core node. + - It updated its wallet details with this new information. + - The update is reflected in the wallet balance. + +### Shutdown Docker ### + +You may now shutdown the regtest docker container. + +Note: This will also clean up any data in the bitcoin core, including the wallet. + +```shell +$ docker kill bdk-box +``` + +## End Words + +In this tutorial we went through the process of receiving, creating, signing and broadcasting transaction using the BDK wallet with `compact_filters` feature. This demonstrates how BDK capabilities can be used to create SPV light wallets with integrated `BIP157` type `compact_filters` node. diff --git a/docs/_blog/descriptor_based_paper_wallet.md b/docs/_blog/descriptor_based_paper_wallet.md new file mode 100644 index 0000000000..4ac3245067 --- /dev/null +++ b/docs/_blog/descriptor_based_paper_wallet.md @@ -0,0 +1,191 @@ +--- +title: "Making Descriptor-based paper wallets" +description: "Demonstrate how to create descriptor-based paper wallet and how to spend them with bdk" +authors: + - Riccardo Casatta + - Steve Myers +date: "2021-03-30" +tags: ["guide", "descriptor", "paper wallets"] +--- + +## Introduction + +In this post, we will use the [Rusty Paper Wallet] tool to create a multi-owned descriptor-based paper wallet. We will use [bdk] via the [bdk-cli] tool to test our descriptor and to be able to sweep the funds from our paper wallet to a new address. + +## About paper wallets + +Paper wallets have a lot of drawbacks, as explained in the [paper wallet Wiki article], as always, do your own research before deciding to use it with mainnet bitcoins. In this post we will +only be using testnet coins. + +## Descriptors + +The [previous version] of the [Rusty Paper Wallet] followed the original paper wallet design: WIF[^WIF] as secret part with the option to generate a different kind of addresses (legacy, nested segwit, and segwit). + +There were plans to [support mnemonic](https://github.com/RCasatta/rusty-paper-wallet/issues/5) instead of WIF keys because it may[^WIFcore] save the sweep transaction[^sweep] and there are more wallets capable of importing a mnemonic instead of a WIF. + +However, choosing a single address type or having wallet support for a specific format is the kind of problem [descriptors] solve perfectly, so the latest [Rusty Paper Wallet] version now accepts a descriptor and the network as parameters. + +## Example use case + +So let's say your grandma wants to buy bitcoin and asked for your help. + +You are a little afraid she may lose the private key. At the same time, you don't want to duplicate the keys and give those to her daughters Alice and Barbara, because both of them could spend and accuse the other of having done so. + +Even though we trust everyone in the family it is better to play it safe and divide the responsibility of protecting Grandma's bitcoin. + +This is a perfect case for a 2 of 3 multi-signature paper wallet. This way also protects the participants from having their copy of the wallet stolen. To compromise Grandma's wallet a thief would need to find and steal at least two of them. + +Note that you as the wallet creator are still the single point of trust because you are going to generate the keys for everyone. Setups combining self generated keys from the participants is possible future work. + +## Creating the paper wallet + +For this example the spending descriptor would be: + +`wsh(multi(2,Grandma,Alice,Barbara))` + +You need [rust] installed to use [Rusty Paper Wallet]. The -n option below explicitly selects +generating `testnet` keys. Use `rusty-paper-wallet --help` to see usage instructions and other +options. + +```shell +$ cargo install rusty-paper-wallet +$ rusty-paper-wallet "wsh(multi(2,Grandma,Alice,Barbara))" -n testnet +data:text/html;base64,PCFET0N... +``` + +The [output] of the command is very long and has been shortened. The string is a [data URI scheme] paste-able in the address bar of a browser. By using a data URI no files are written on the hard disk, leaving less trace of secret material on the computer. +It's also a good idea to use incognito mode in the browser to prevent it from saving the page in the history. + +The following is the result: + + + +Under the hood, the command created a key pair randomly for every alias present in the descriptor, then replaced the aliases with the created keys and generated the corresponding address. This address is the same for every paper wallet and it is shown in the upper part of the paper wallet (the public part) along with the alias, linking the paper wallet to the owner. + +The lower part is the secret part, the written part is the descriptor with the aliases, followed by a legend linking the aliases with the keys. In the legend, all the keys are public but the one of the owner which is a private WIF. The secret QR code instead contains the descriptor already with the keys. + +The paper wallet must then be printed, and it is better to use a printer without wifi and also to be aware that some sensitive data may remain in the printer's cache. + +Then the paper wallet must be cut along the dotted lines, the secret part should be folded twice over the black zone[^blackzone]. The black zone helps to avoid showing the secret parts in the presence of back-light. Once the folding is done the paper wallet should be plasticized to prevent being damaged by water. + +## BDK + +Any descriptor based wallet can be used to check the balance of and sweep the funds from +Grandma's paper wallet. For this post we'll demonstrate using the [bdk-cli] tool to do these steps. +Another area where [bdk] could be used with [Rusty Paper Wallet] is to compile a more +complicated miniscript spending policy into a descriptor, as we have done in the [spending policy demo] post. + +## Funding tx + +Since Grandma's wallet was created as a `wsh` descriptor, bitcoin can be sent to it from any +segwit capable wallet, we'll use a public [bitcoin testnet faucet]. Once the funds are sent the +deposit address `tb1qu6lcua9w2zkarjj5xwxh3l3qtcxh84hsra3jrvpszh69j2e54x7q3thycw` we can also use this +address and a testnet explorer to [confirm the funds were received]. + +## Sweep tx + +Now that Grandma's paper wallet is funded it's time to demonstrate how to use [bdk-cli] to sweep these +funds to a new address. Let's assume Grandma lost her original paper wallet and has asked +her daughters to sweep them to a new single signature wallet so she can spend them. + +### Step 1: Alice creates and signs a PSBT + +Alice uses the private text or QR code from her paper wallet to find her private key and the +public keys for Grandma and Barbara. With this info she creates a PSBT to sweep Grandma's funds +to a new address (in this example we'll send them back to our [bitcoin testnet faucet]). Notice how Alice +includes her wallet's descriptor checksum '#em3q73l5', this [guarantees] she has entered her descriptor correctly. + +```shell +$ SWEEP_TO_ADDR=tb1qm5tfegjevj27yvvna9elym9lnzcf0zraxgl8z2 + +$ ALICE_WIF=cSSKRHDmQEEutp5LD14tAcixu2ehSNPDTqNek1zMa9Pet98qxHq3 +$ BARBARA_PUBKEY=02a3f3f2658b9812ddeabfbde2fde03f8a65369e4ed621f29fa8ba0cc519b789fb +$ GRANDMA_PUBKEY=03f1bd2bff8e9c61f58a8d46d18fd8f3149b1f2d76b3c423a7874a5d5811d67cee +$ ALICE_DESCRIPTOR="wsh(multi(2,$GRANDMA_PUBKEY,$ALICE_WIF,$BARBARA_PUBKEY))#em3q73l5" + +# confirm descriptor creates the expected deposit address +$ bdk-cli wallet -w alice -d $ALICE_DESCRIPTOR get_new_address +{ + "address": "tb1qu6lcua9w2zkarjj5xwxh3l3qtcxh84hsra3jrvpszh69j2e54x7q3thycw" +} + +# sync the wallet and show the balance +$ bdk-cli wallet -w alice -d $ALICE_DESCRIPTOR sync +{} + +$ bdk-cli wallet -w alice -d $ALICE_DESCRIPTOR get_balance +{ + "satoshi": 10000 +} + +# create and sign PSBT +$ UNSIGNED_PSBT=$(bdk-cli wallet -w alice -d $ALICE_DESCRIPTOR create_tx --send_all --to $SWEEP_TO_ADDR:0 | jq -r ".psbt") + +$ ALICE_SIGNED_PSBT=$(bdk-cli wallet -w alice -d $ALICE_DESCRIPTOR sign --psbt $UNSIGNED_PSBT | jq -r ".psbt") +``` + +### Step 2: Barbara signs Alice's signed PSBT and broadcasts the tx + +Now it's Barbara's turn to use the private text or QR code from her paper wallet to get her private +key and the public keys for Grandma and Alice. With this info plus Alice's signed PSBT she can +create a fully signed PSBT to broadcast and complete the sweep of Grandma's funds. + +```shell +$ ALICE_PUBKEY=02e486e32f0f87136fa042cb53219ace8537ea1d036deb2f4293570b94325d11cb +$ BARBARA_WIF=cSfMLzSZ9NjWUTqL3sFpgWJssnu2qgmE2cm5N1jPDRRJuDcrsPEB +$ GRANDMA_PUBKEY=03f1bd2bff8e9c61f58a8d46d18fd8f3149b1f2d76b3c423a7874a5d5811d67cee +$ BARBARA_DESCRIPTOR="wsh(multi(2,$GRANDMA_PUBKEY,$ALICE_PUBKEY,$BARBARA_WIF))#nxfa5n0z" + +# confirm descriptor creates the expected deposit address +$ bdk-cli wallet -w barbara -d $BARBARA_DESCRIPTOR get_new_address +{ + "address": "tb1qu6lcua9w2zkarjj5xwxh3l3qtcxh84hsra3jrvpszh69j2e54x7q3thycw" +} + +# sync the wallet and show the balance +$ bdk-cli wallet -w barbara -d $BARBARA_DESCRIPTOR sync +{} + +$ bdk-cli wallet -w barbara -d $BARBARA_DESCRIPTOR get_balance +{ + "satoshi": 10000 +} + +$ FINAL_PSBT=$(bdk-cli wallet -w barbara -d $BARBARA_DESCRIPTOR sign --psbt $ALICE_SIGNED_PSBT | jq -r ".psbt") + +$ bdk-cli wallet -w barbara -d $BARBARA_DESCRIPTOR broadcast --psbt $FINAL_PSBT +{ + "txid": "9ecd8e6be92b7edd8bf1799f8f7090e58f813825f826bdb771b4cdb444cdeb59" +} +``` + +And finally we verify that Alice and Barbara successfully created and broadcast Grandma's [sweep tx]. + +## Conclusion + +In this post we showed how to create a multi-sig descriptor based paper wallet using +[Rusty Paper Wallet] and then sweep the funds from our example paper wallet to a new address. If you +found this post interesting please comment below. Or give it a try yourself and if you run into any +problems or would like to suggest improvements leave an issue in the [Rusty Paper Wallet] or +[bdk-cli] github repos. Thanks! + +[paper wallet wiki article]: https://en.bitcoin.it/wiki/Paper_wallet +[previous version]: https://github.com/RCasatta/rusty-paper-wallet/tree/339fa4418d94f6fdd96f3d0301cab8a0bc09e8bd +[Rusty Paper Wallet]: https://github.com/RCasatta/rusty-paper-wallet +[support mnemonic]: https://github.com/RCasatta/rusty-paper-wallet/issues/5 +[descriptors]: /descriptors +[bdk]: https://github.com/bitcoindevkit/bdk +[rust]: https://www.rust-lang.org/tools/install +[output]: /descriptor-based-paper-wallets/data-url.txt +[data URI scheme]: https://en.wikipedia.org/wiki/Data_URI_scheme +[bdk-cli]: https://github.com/bitcoindevkit/bdk-cli +[bitcoin testnet faucet]: https://bitcoinfaucet.uo1.net/ +[confirm the funds were received]: https://mempool.space/testnet/address/tb1qu6lcua9w2zkarjj5xwxh3l3qtcxh84hsra3jrvpszh69j2e54x7q3thycw +[sweep tx]: https://mempool.space/testnet/tx/9ecd8e6be92b7edd8bf1799f8f7090e58f813825f826bdb771b4cdb444cdeb59 +[spending policy demo]: /blog/2021/02/spending-policy-demo/#step-4-create-wallet-descriptors-for-each-participant +[guarantees]: https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md#checksums + +[^WIF]: Wallet Input Format, a string encoding a ECDSA private key https://en.bitcoin.it/wiki/Wallet_import_format +[^WIFcore]: Unless the user import the WIF directly into bitcoin core +[^sweep]: Some wallets refers to sweep as the action to create a transaction taking all the funds from the paper wallet and sending those to the wallet itself. +[^blackzone]: Ideally, the black zone should be twice as long as the secret part to cover it back and front, long descriptor may leave a shorter black zone, ensure to have you printer set with vertical layout for best results. diff --git a/docs/_blog/descriptors_in_the_wild.md b/docs/_blog/descriptors_in_the_wild.md new file mode 100644 index 0000000000..3bc4cc38ec --- /dev/null +++ b/docs/_blog/descriptors_in_the_wild.md @@ -0,0 +1,345 @@ +--- +title: "A Multisig between BDK and Core" +description: "Guide to setup a 2-of-2 multisig using Bitcoin Core and BDK" +authors: + - Gabriele Domenichini +date: "2020-11-18" +tags: ["guide", "descriptor"] +--- + +## Introduction + +I have tried to setup a 2 of 2 multi signature infrastructure with two +different wallets, which know nothing about each other, but are compliant with +two very important protocols: [Output Descriptors] and [Partially Signed +Bitcoin Transactions][PSBT] described in BIP 174. + +Before these two protocols came into existence, making a multi signature setup +and spending from it was possible only if the involved parties were using the +same wallet (eg. Electrum Desktop Wallet). This limitation was due to the fact +that the two parties had to agree: + +* on the particular type of script and address to use +* on the way the transaction would be shared composed and signed with all the +involved parties. + +[Output Descriptors] are a way to express which kind scriptPubKey and +addresses to produce with a key or a series of keys. + +[PSBT] is instead the standard protocol used to create a transaction and to enrich +it with the necessary signatures and other components, to make it valid and complete. + +Together they provide a common ground to create and use a multi signature +infrastructure in a heterogeneous environment, and this is what I have put +to test. + +## The use case + +Imagine Alice and Bob owning a company and being willing to put the corporate cash +in a 2of2 multi signature setup, so that each one of them have to agree and sign each +transaction. + +## The role of Descriptors + +If Alice and Bob cannot agree on the software to use, to monitor the same financial +situation, the two software must control and produce exactly the same series +of multisignature addresses. + +To make two different software produce the same addresses in a deterministic way +we must ensure that they: +* produce the same pair of public keys +* combine them in the same order +* put them inside the same scriptPubKey to produce the same address + +Here is where the [Output Descriptors] come into play. They describe: + +* the sequence of public keys each extended key (xpub) will produce +* the sequence in which the new public keys of various parties will enter into +the script +* the type of script the wallet will prepare with that group keys and so the type +of address the group of keys will produce. + +**By sharing the same Descriptor, every compliant wallet will derive +deterministically the same series of multisig addresses**. + +Imagine Alice using Bitcoin Core (from now on ["Core"][Bitcoin Core]) as a +Wallet and Bob using a "Last generation" wallet, Bitcoin Development Kit +(from now on ["BDK"][BDK]), which uses descriptors and miniscript natively. + +Each of these two software wallets should be able to: + +* Create a new address which is seen as belonging to the multi signature +wallet in both software +* Express the consent of each party by partially signing the transaction in a way +the other wallet can understand and complete it with its own signature. + +The infrastructure of multiple Extended keys combined toghether to produce +multiple multisignature addresses is often referred as +*[Hierarchical Deterministic][HDWallet] multi signature wallet or HDM*. + +What follows are the steps to create the HDM usable both in Core and +in BDK. + +*Note: In Core, [Descriptor wallets] are still experimental and in general, +both wallets should be tested for descriptor capabilities only in testnet.* + +## Our playground + +We will build a 2of2 key set up that will be used cooperatively by Bitcoin Core +and Bitcoin Development Kit. +The steps Alice and Bob will do are: + +1. creation of the seed and the derived Extended Master Public and send it to +the other party +2. Create the multi signature descriptor for each wallet +3. Use each other's software to receive testnet coins from a faucet +4. return part of the coins to the faucet signing the transaction with both +wallets. + +We need: +* [Bitcoin Dev Kit][BDK] +* [Bitcoin Core] (v0.21.0 or later) + +### 1. Creating the seeds and the derived Extended Public keys + +#### Seeds and Extended Master Public + +We build an Extended Private Master Key for both wallet and derive a BIP84 +Extended Master Public for Bitcoin Core and then for BDK. + +For Bitcoin Core (Alice): + +``` +# new Extended wallet data +export core_key=$(bdk-cli key generate) + +# New Extended Master Private + +export core_xprv=$(echo $core_key | jq -r '.xprv') + +# Now I derive the xpubs (one for receiving and one for the change) +# together with informations about the derivation path to be communicated +# to BDK wallet's owner (Bob). + +export core_xpub_84_for_rec_desc=$(bdk-cli key derive --path m/84h/0h/0h/0 --xprv $core_xprv | jq -r '.xpub') +export core_xpub_84_for_chg_desc=$(bdk-cli key derive --path m/84h/0h/0h/1 --xprv $core_xprv | jq -r '.xpub') +``` + +For BDK (Bob) we do the same: + +``` +# new Extended wallet data + +export BDK_key=$(bdk-cli key generate) + +# New Extended Master Private + +export BDK_xprv=$(echo $BDK_key | jq -r '.xprv') + +# Now I build the derived xpubs to be communicated (to Alice). + +export BDK_xpub_84_for_rec_desc=$(bdk-cli key derive --path m/84h/0h/0h/0 --xprv $BDK_xprv | jq -r '.xpub') +export BDK_xpub_84_for_chg_desc=$(bdk-cli key derive --path m/84h/0h/0h/1 --xprv $BDK_xprv | jq -r '.xpub') +``` + +### 2. Creation of the multi signature descriptor for each wallet + +To build a multisig wallet, each wallet owner must compose the descriptor +adding: +* his derived extended **private** key AND +* all the extended **public** keys of the other wallets involved in the +multi signature setup + +*The different nature of the two keys (one is private and one is public) is +due to the fact that each wallet, to be able to partially sign the transaction, +**must manage the private key of the wallet's owner*** AND have the other +party's public key. Otherwise, if we put both public keys, we would obtain +a watch-only wallet unable to sign the transactions. If we +had both extended private keys inside the descriptor, we would allow each party +to finalize the transactions autonomously. + +#### In Bitcoin Core: + +In our case, the multi signature descriptor for Bitcoin Core will be composed +with: + +* The BIP84 derived Extended **Public** Key from BDK +* The BIP84 derived Extended **Private** Key from Core. + +BDK wallet's owner will send to Core's owner the derived xpub for this purpose. +This is how the Core's multisig descriptor will be created and put into an +environment variable: + +``` +export core_rec_desc="wsh(multi(2,$BDK_xpub_84_for_rec_desc,$core_xprv/84'/0'/0'/0/*))" +``` + +Where of course `$BDK_xpub_84_for_rec_desc`is the derived master public created +in BDK and received by Core's owner. + +The meaning of what is before and after is illustrated in the doc that explain +the use of [Output Descriptors in Bitcoin Core][Output Descriptors]. + +We add the necessary checksum using the specific `bitcoin-cli` call. + +``` +export core_rec_desc_chksum=$core_rec_desc#$(bitcoin-cli -testnet getdescriptorinfo $core_rec_desc | jq -r '.checksum') +``` + +We repeat the same to build the descriptor to receive the change. + +``` +export core_chg_desc="wsh(multi(2,$BDK_xpub_84_for_chg_desc,$core_xprv/84'/0'/0'/1/*))" +export core_chg_desc_chksum=$core_chg_desc#$(bitcoin-cli -testnet getdescriptorinfo $core_chg_desc|jq -r '.checksum') +``` + +#### In BDK: + +For BDK we set the derivation for receiving addresses and change addresses +in the command line (maybe setting an alias) + +Building the descriptor: + +``` +export BDK_rec_desc="wsh(multi(2,$BDK_xprv/84'/0'/0'/0/*,$core_xpub_84_for_rec_desc))"` +``` + +Please note that the order of the extended key in the descriptor MUST be the +same in the 2 wallets. + +*We have chosen to put BDK first and in each software wallet, the public key +derived from BDK will always come first. In alternative, we could have chosen to +produce the descriptor, [chosing a `soretedmulti` multisignature setup][sortedmulti]*. + +``` +export BDK_rec_desc_chksum=$BDK_rec_desc#$(bitcoin-cli -testnet getdescriptorinfo $BDK_rec_desc | jq -r '.checksum') +export BDK_chg_desc="wsh(multi(2,$BDK_xprv/84'/0'/0'/1/*,$core_xpub_84_for_chg_desc))" +export BDK_chg_desc_chksum=$BDK_chg_desc#$(bitcoin-cli -testnet getdescriptorinfo $BDK_chg_desc | jq -r '.checksum') +``` + +To take a look at the variables we have produced so far: +``` +env | grep 'core_' +env | grep 'BDK_' +``` + +Now we will use the multisig descriptor wallet to receive testnet coins with +Alice and Bob's software + +### 3. Use each other's software to receive testnet coins from a faucet + +#### In Bitcoin Core + +Alice must create an empty, experimental new "descriptors wallet" in Core and +to import the multisig Output Descriptor. + +``` +bitcoin-cli -testnet createwallet "multisig2of2withBDK" false true "" false true false +```` +The flag are to: +* use the private keys +* make it empty +* no password provided to the wallet +* reusing of addresses not allowed +* "new experimental descriptors wallet" +* don't load it on start up + +``` +bitcoin-cli -testnet -rpcwallet=multisig2of2withBDK importdescriptors "[{\"desc\":\"$core_rec_desc_chksum\",\"timestamp\":\"now\",\"active\":true,\"internal\":false},{\"desc\":\"$core_chg_desc_chksum\",\"timestamp\":\"now\",\"active\":true,\"internal\":true}]" +``` +Now Alice asks for her first receiving multisignature address. + +``` +export first_address=$(bitcoin-cli -testnet -rpcwallet=multisig2of2withBDK getnewaddress) +echo $first_address +``` + +#### BDK +In BDK Bob can specify directly the descriptors on the command line to produce +the multisig address, because BDK is descriptors aware natively. + +``` +repl -d "$BDK_rec_desc_chksum" -c "$BDK_chg_desc_chksum" -n testnet -w $BDK_fingerprint get_new_address` +``` + +Et voilà: if we have done everything correctly, the newly created address in +Core is the same of the newly created address in BDK. this is part of the +"miracle" of descriptors' interoperability. + +#### We ask for testnet coins giving the first created address. + +To find testnet coins for free, you can just google "testnet faucet" and you +should find some satoshis to play with. Just give to the site your first +generated address and, in twenty minutes, you will find the satoshis in +your balance both in Core and in BDK. + +``` +# to check it in Core: + +bitcoin-cli -testnet -rpcwallet=multisig2of2withBDK getbalance + +# In BDK: + +# Sync with the blockchain +repl -d "$BDK_rec_desc_chksum" -c "$BDK_chg_desc_chksum" -n testnet -w $BDK_fingerprint sync +# Get the balance +repl -d "$BDK_rec_desc_chksum" -c "$BDK_chg_desc_chksum" -n testnet -w $BDK_fingerprint get_balance +``` +Some testnet faucets have an address to send back the unused satoshi after +the use. Take note of that because we will use it in the next step. + +### 4. we return part of the satoshis received back to the faucet + +``` +export psbt=$(bitcoin-cli -testnet -rpcwallet=multisig2of2withBDK walletcreatefundedpsbt "[]" "[{\"tb1qrcesfj9f2d7x40xs6ztnlrcgxhh6vsw8658hjdhdy6qgkf6nfrds9rp79a\":0.000012}]" | jq -r '.psbt') + +export psbt=$(bitcoin-cli -testnet -rpcwallet=multisig2of2withBDK walletprocesspsbt $psbt | jq -r '.psbt') +{ + "psbt": "cHNidP8BAIkCAAAAATj90EC+NAuXj7y6SseZJucoJM6sGnUcVm9koTveZECTAAAAAAD+////AmACAAAAAAAAIgAg98ol9j4AalD71E0mV5QV0uM6/vCT+pi2twxr/zrvLROwBAAAAAAAACIAIB4zBMipU3xqvNDQlz+PCDXvpkHH1Q95Nu0mgIsnU0jbAAAAAAABAIkCAAAAAQS+ObgGG6UwtvaO3KYph2E3/ws7Q83RbmR3rxC0fKYSAQAAAAD+////AtAHAAAAAAAAIgAg6GXadcNj7k4yKUbnVlTLiedXQFXYdCBoNygop/PISNDAHQAAAAAAACIAIBQpiDTgPIMt0ld8cmuYqlY+EIPjvrmMqZruDhs61hQNAAAAAAEBK9AHAAAAAAAAIgAg6GXadcNj7k4yKUbnVlTLiedXQFXYdCBoNygop/PISNAiAgNt0j7Ae0iA7qlLolruNqLWkPA96J0qgMLK1M7WOGMAfUcwRAIgS6x0i1J1HRzllIPf4WlFY+Dl8kCCLK81TL2djZxTFXMCICJVBKkKNxu1w1mRVor6iFTSVXiJjmWwBXVeJLISvBwAAQEFR1IhArn3tec7n7318rnWqf0dIIwtLtfxo6Zt0HV70UvZYaWvIQNt0j7Ae0iA7qlLolruNqLWkPA96J0qgMLK1M7WOGMAfVKuIgYCufe15zufvfXyudap/R0gjC0u1/Gjpm3QdXvRS9lhpa8YNEw2cFQAAIAAAACAAAAAgAAAAAAAAAAAIgYDbdI+wHtIgO6pS6Ja7jai1pDwPeidKoDCytTO1jhjAH0YO/laXFQAAIAAAACAAAAAgAAAAAAAAAAAAAEBR1IhAqccvA3rL13D1K4GeWjcahDsO3P8oaVNBttk4MlCKXIcIQLHKhjmPuCQjyS77ZfaMN2tdgNKcf/+57VXGZhz/UWTl1KuIgICpxy8DesvXcPUrgZ5aNxqEOw7c/yhpU0G22TgyUIpchwYNEw2cFQAAIAAAACAAAAAgAEAAAADAAAAIgICxyoY5j7gkI8ku+2X2jDdrXYDSnH//ue1VxmYc/1Fk5cYO/laXFQAAIAAAACAAAAAgAEAAAADAAAAAAA=", + "complete": false +} +``` + +Exactly! Note the `"complete": false`. We have processed the transaction with +Core but we miss one of the necessary key of the multisig 2of2 setup (The one +contained inside BDK). + +`tb1qrcesfj9f2d7x40xs6ztnlrcgxhh6vsw8658hjdhdy6qgkf6nfrds9rp79a` is the address +we got from the faucet site to return the satoshis. + +The [PSBT] is sent over to the BDK wallet owner who tries to sign the +transaction: + +``` +repl -d "$BDK_rec_desc_chksum" -c "$BDK_chg_desc_chksum" -n testnet -w $BDK_fingerprint sign --psbt $psbt +{ + "is_finalized": true, + "psbt": "cHNidP8BAIkCAAAAATj90EC+NAuXj7y6SseZJucoJM6sGnUcVm9koTveZECTAAAAAAD+////AmACAAAAAAAAIgAg98ol9j4AalD71E0mV5QV0uM6/vCT+pi2twxr/zrvLROwBAAAAAAAACIAIB4zBMipU3xqvNDQlz+PCDXvpkHH1Q95Nu0mgIsnU0jbAAAAAAABAIkCAAAAAQS+ObgGG6UwtvaO3KYph2E3/ws7Q83RbmR3rxC0fKYSAQAAAAD+////AtAHAAAAAAAAIgAg6GXadcNj7k4yKUbnVlTLiedXQFXYdCBoNygop/PISNDAHQAAAAAAACIAIBQpiDTgPIMt0ld8cmuYqlY+EIPjvrmMqZruDhs61hQNAAAAAAEBK9AHAAAAAAAAIgAg6GXadcNj7k4yKUbnVlTLiedXQFXYdCBoNygop/PISNAiAgNt0j7Ae0iA7qlLolruNqLWkPA96J0qgMLK1M7WOGMAfUcwRAIgS6x0i1J1HRzllIPf4WlFY+Dl8kCCLK81TL2djZxTFXMCICJVBKkKNxu1w1mRVor6iFTSVXiJjmWwBXVeJLISvBwAASICArn3tec7n7318rnWqf0dIIwtLtfxo6Zt0HV70UvZYaWvRzBEAiBkVDLgVEwvENnLx+04o7gGpGjFDBwAXTJmf8Yvo35oygIgbuBkHsvPC9jmZcMZ9P+Pwp01yxSaWo+5feyPmd3ai1kBAQVHUiECufe15zufvfXyudap/R0gjC0u1/Gjpm3QdXvRS9lhpa8hA23SPsB7SIDuqUuiWu42otaQ8D3onSqAwsrUztY4YwB9Uq4iBgNt0j7Ae0iA7qlLolruNqLWkPA96J0qgMLK1M7WOGMAfRg7+VpcVAAAgAAAAIAAAACAAAAAAAAAAAAiBgK597XnO5+99fK51qn9HSCMLS7X8aOmbdB1e9FL2WGlrxg0TDZwVAAAgAAAAIAAAACAAAAAAAAAAAABBwABCNoEAEcwRAIgZFQy4FRMLxDZy8ftOKO4BqRoxQwcAF0yZn/GL6N+aMoCIG7gZB7LzwvY5mXDGfT/j8KdNcsUmlqPuX3sj5nd2otZAUcwRAIgS6x0i1J1HRzllIPf4WlFY+Dl8kCCLK81TL2djZxTFXMCICJVBKkKNxu1w1mRVor6iFTSVXiJjmWwBXVeJLISvBwAAUdSIQK597XnO5+99fK51qn9HSCMLS7X8aOmbdB1e9FL2WGlryEDbdI+wHtIgO6pS6Ja7jai1pDwPeidKoDCytTO1jhjAH1SrgABAUdSIQKnHLwN6y9dw9SuBnlo3GoQ7Dtz/KGlTQbbZODJQilyHCECxyoY5j7gkI8ku+2X2jDdrXYDSnH//ue1VxmYc/1Fk5dSriICAqccvA3rL13D1K4GeWjcahDsO3P8oaVNBttk4MlCKXIcGDRMNnBUAACAAAAAgAAAAIABAAAAAwAAACICAscqGOY+4JCPJLvtl9ow3a12A0px//7ntVcZmHP9RZOXGDv5WlxUAACAAAAAgAAAAIABAAAAAwAAAAAA" +} +``` +The signature has succeded (note the "is_finalized": true,) and now we can +broadcast the transction. +``` +repl -d "$BDK_rec_desc_chksum" -c "$BDK_chg_desc_chksum" -n testnet -w $BDK_fingerprint broadcast --psbt "cHNidP8BAIkCAAAAATj90EC+NAuXj7y6SseZJucoJM6sGnUcVm9koTveZECTAAAAAAD+////AmACAAAAAAAAIgAg98ol9j4AalD71E0mV5QV0uM6/vCT+pi2twxr/zrvLROwBAAAAAAAACIAIB4zBMipU3xqvNDQlz+PCDXvpkHH1Q95Nu0mgIsnU0jbAAAAAAABAIkCAAAAAQS+ObgGG6UwtvaO3KYph2E3/ws7Q83RbmR3rxC0fKYSAQAAAAD+////AtAHAAAAAAAAIgAg6GXadcNj7k4yKUbnVlTLiedXQFXYdCBoNygop/PISNDAHQAAAAAAACIAIBQpiDTgPIMt0ld8cmuYqlY+EIPjvrmMqZruDhs61hQNAAAAAAEBK9AHAAAAAAAAIgAg6GXadcNj7k4yKUbnVlTLiedXQFXYdCBoNygop/PISNAiAgNt0j7Ae0iA7qlLolruNqLWkPA96J0qgMLK1M7WOGMAfUcwRAIgS6x0i1J1HRzllIPf4WlFY+Dl8kCCLK81TL2djZxTFXMCICJVBKkKNxu1w1mRVor6iFTSVXiJjmWwBXVeJLISvBwAASICArn3tec7n7318rnWqf0dIIwtLtfxo6Zt0HV70UvZYaWvRzBEAiBkVDLgVEwvENnLx+04o7gGpGjFDBwAXTJmf8Yvo35oygIgbuBkHsvPC9jmZcMZ9P+Pwp01yxSaWo+5feyPmd3ai1kBAQVHUiECufe15zufvfXyudap/R0gjC0u1/Gjpm3QdXvRS9lhpa8hA23SPsB7SIDuqUuiWu42otaQ8D3onSqAwsrUztY4YwB9Uq4iBgNt0j7Ae0iA7qlLolruNqLWkPA96J0qgMLK1M7WOGMAfRg7+VpcVAAAgAAAAIAAAACAAAAAAAAAAAAiBgK597XnO5+99fK51qn9HSCMLS7X8aOmbdB1e9FL2WGlrxg0TDZwVAAAgAAAAIAAAACAAAAAAAAAAAABBwABCNoEAEcwRAIgZFQy4FRMLxDZy8ftOKO4BqRoxQwcAF0yZn/GL6N+aMoCIG7gZB7LzwvY5mXDGfT/j8KdNcsUmlqPuX3sj5nd2otZAUcwRAIgS6x0i1J1HRzllIPf4WlFY+Dl8kCCLK81TL2djZxTFXMCICJVBKkKNxu1w1mRVor6iFTSVXiJjmWwBXVeJLISvBwAAUdSIQK597XnO5+99fK51qn9HSCMLS7X8aOmbdB1e9FL2WGlryEDbdI+wHtIgO6pS6Ja7jai1pDwPeidKoDCytTO1jhjAH1SrgABAUdSIQKnHLwN6y9dw9SuBnlo3GoQ7Dtz/KGlTQbbZODJQilyHCECxyoY5j7gkI8ku+2X2jDdrXYDSnH//ue1VxmYc/1Fk5dSriICAqccvA3rL13D1K4GeWjcahDsO3P8oaVNBttk4MlCKXIcGDRMNnBUAACAAAAAgAAAAIABAAAAAwAAACICAscqGOY+4JCPJLvtl9ow3a12A0px//7ntVcZmHP9RZOXGDv5WlxUAACAAAAAgAAAAIABAAAAAwAAAAAA" +{ + "txid": "a0b082e3b0579822d4a0b0fa95a4c4662f6b128ffd43fdcfe53c37473ce85dee" +} +``` + +## Conclusion + +We have built an HDM and we have used it with two indipendent wallets, which +are compatible with [BIP 174][PSBT] and [Output Descriptors]. Hopefully we +will see many other compatible wallets beyound [Bitcoin Core] and [BDK], +with which we will be able to easily set up multi signature schemes. + + +[Descriptor wallets]: https://github.com/bitcoin/bitcoin/pull/16528 +[Electrum]: https://electrum.org +[Output Descriptors]: https://bitcoinops.org/en/topics/output-script-descriptors/ +[PSBT]: https://en.bitcoin.it/wiki/BIP_0174 +[HDWallet]: https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki +[sortedmulti]: https://github.com/bitcoin/bitcoin/pull/17056?ref=tokendaily +[BDK]: https://bitcoindevkit.org/ +[Bitcoin Core]: https://bitcoincore.org/ +[pycoin]: https://github.com/richardkiss/pycoin diff --git a/docs/_blog/exploring_bdk_flutter.md b/docs/_blog/exploring_bdk_flutter.md new file mode 100644 index 0000000000..df9e7f08f3 --- /dev/null +++ b/docs/_blog/exploring_bdk_flutter.md @@ -0,0 +1,774 @@ +--- +title: "BDK-FLUTTER: Building Flutter Apps with BDK" +description: "A tutorial and guide to using bdk-flutter for building bitcoin apps" +authors: + - Bitcoin Zavior +date: "2022-10-05" +tags: + [ + "bitcoin", + "React Native", + "Flutter", + "iOS", + "Android", + "mobile", + "bdk-rn", + "bdk", + "tutorial", + "guide", + "wallet", + ] +--- + +## Introduction + +`bdk-flutter` is the **Bitcoin Dev kit**'s **Flutter** library which enables building bitcoin applications for Android and iOS mobile platforms. Using `bdk-flutter` is similar to using any other Flutter module. Just do `flutter pub add bdk_flutter` and you are ready to code! This is the first tutorial on how to use `bdk-flutter`, more coming soon, make sure to [follow](https://twitter.com/BitcoinZavior) to be notified of new ones. There will also be a **`bdk-flutter`** focused Livestream on [Twitch](https://www.twitch.tv/bitcoindevelopers) on the Bitcoin Developers [YouTube Channel](https://www.youtube.com/channel/UCUq_ZdezVWKPvkWRicAYxLA/videos) so make sure to subscribe. + +This tutorial will explore `bdk-flutter` usage and the API it provides. This guide will walk through the development process and code for making a bitcoin application. The bitcoin application we create will be a non-custodial HD Wallet. The application will have the functionality to create a new wallet or restore from a known mnemonic seed phrase. This application will also be able to interact with the bitcoin network to sync UTXOs from new blocks and broadcast transactions. + +The tutorial will focus on bitcoin concepts and `bdk-flutter` API. So it will gloss over Flutter and Dart. If you are interested in learning more about Flutter and Dart please refer to the Flutter [learning portal](https://flutter.dev/learn). The code for this tutorial is available on the [LtbLightning GitHub](https://github.com/LtbLightning/bdk-flutter-quickstart) + + + +### Prerequisites + +To use `bdk-flutter` in a Flutter App, a Flutter development environment is required. Please refer to resources out there on the internet if you need to set this up, here is one of many good resources to guide you on [environment setup](https://docs.flutter.dev/get-started/install) + +### Bitcoin Basics + +The bitcoin concepts used in this blog post are detailed and explained very well in external bitcoin resources. Here are some links for reference: + +[Mastering Bitcoin(HD Wallet chapter)](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch04.asciidoc) + +[Bitcoin Output Descriptors from bitcoin GitHub](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md) + +Now let's jump into Bitcoin Dev Kit + +## Bitcoin Dev Kit and bdk-flutter + +`bdk-flutter` is **Bitcoin Dev kit**'s **Flutter** library for building Bitcoin apps in **Flutter**. +It encapsulates all of the low-level APIs and methods for BDK and exposes them in a Flutter context. To use BDK in Flutter apps only the `bdk-flutter` module is required. `bdk-flutter` can be used like any other Flutter library and is available on [pub.dev](https://pub.dev/packages/bdk_flutter) + +## Getting Started + +Although we won't delve deep into Flutter we will focus more on bitcoin and `bdk-flutter`, however, some rudimentary Flutter setup is required, especially a basic Flutter app to add our code. + +start by creating a new Flutter project. + +`flutter create bdk-flutter-quickstart` + +Once done let's `cd` into the new project directory and run the basic Flutter app that's created + +```shell +cd bdk-flutter-quickstart +flutter run +``` + +This should start building the app and then launch the app in a simulator. So far we have created a basic Flutter project if this doesn't work then refer to the Flutter development setup guide to troubleshoot. + + BDK Flutter Quick Start + +## Setting up Flutter app structure + +Let's set up a very basic app structure. Let's create an `assets` folder in the project root and then add new folders `widgets`, `screens`, and `styles` inside the existing `lib` folder. + +Paste the following code in your `pubspec.yaml` file, assets section. + + - assets/ + +Please make sure your assets section looks like the screenshot below. +BDK Flutter Quick Start + +Once done let's run a `get` command from the pub tool commands, this will get all the required dependencies for our project. + +```shell +flutter pub get +``` + +To make this quick you can download the theme, styled widgets and images used in the tutorial from the repository. The `theme.dart` file has the theme we will use and this can be taken from [here](https://github.com/LtbLightning/bdk-flutter-quickstart/blob/master/lib/styles/theme.dart) and moved to the styles folder. The `widgets.dart` file has the styled widgets we will use and these can be taken from [here](https://github.com/LtbLightning/bdk-flutter-quickstart/blob/master/lib/widgets/widgets.dart) and moved to the widgets folder. The image assets can be taken from [here](https://github.com/LtbLightning/bdk-flutter-quickstart/tree/master/assets) Alternatively, you can write your theme, widgets and use your images if you intend to style the app differently. + +In addition to the the theme, widgets and assets. We also need to create a `screens` folder and create a `home.dart` file inside it, this will be where most of the code will be added. + +Once done the file structure should look like this: + + + +
Locate `main.dart` in the project root, this will have the default code added by `flutter create`, let's delete all contents of `main.dart` and replace it with the following code to use `home.dart` as our main screen. This will probably crash the app but that's fine, it will be up and running once we add code to `home.dart` in the next few steps + +```dart +// main.dart + +import 'package:bdk_flutter_quickstart/screens/home.dart'; +import 'package:bdk_flutter_quickstart/styles/theme.dart'; +import 'package:flutter/material.dart'; + +void main() { + runApp(const MyApp()); +} + +class MyApp extends StatelessWidget { + const MyApp({Key? key}) : super(key: key); + + // This widget is the root of your application. + @override + Widget build(BuildContext context) { + return MaterialApp( + debugShowCheckedModeBanner: false, + title: 'BDK-FLUTTER TUTORIAL', + theme: theme(), + home: const Home(), + ); + } +} +``` + +## Installing `bdk-flutter` + +With the Flutter project in place, we can now add `bdk-flutter` using `flutter pub add`. + +```shell +flutter pub add bdk_flutter +``` + +This will add a line like this to your package's `pubspec.yaml` and this will also run an implicit flutter pub get to download `bdk-flutter` from `pub.dev`: + +```shell +dependencies: + bdk_flutter: ^0.28.2 +``` + +## Configuring + +Make sure your app meets the following requirements for using `bdk-flutter` + +**Android** + +MinSdkVersion : API 23 or higher. + +**IOS** + +Deployment target: iOS 12.0 or greater. + +Locate your Podfile in the ios folder of your project and paste the following code at the beginning + +``` +platform :ios, '12.0' +``` + +After changing the deployment target in your project's `PodFile`, let's use the following `command` to install pod dependencies for iOS. + +```shell +cd ios && pod install && cd .. +``` + +Once done, bdk-flutter is installed and configured and ready to be used in our **bdk-flutter-quickstart** App. + +## Importing `bdk-flutter` + +Locate `home.dart` which we added in the setup section and import `bdk-flutter` at the top of the file. Create a stateful widget called `Home` + +```dart +// screens/home.dart + +import 'package:bdk_flutter/bdk_flutter.dart'; + + class Home extends StatefulWidget { + const Home({Key? key}) : super(key: key); + + @override + State createState() => _HomeState(); +} + +class _HomeState extends State { + TextEditingController mnemonic = TextEditingController(); + @override + Widget build(BuildContext context) { + return Container(); + } +} +``` + +Before we start using `bdk-flutter` let's add some additional imports and also import styles, to create a basic layout to build our home screen + +```dart +// screens/home.dart + +import 'package:bdk_flutter/bdk_flutter.dart'; +import 'package:bdk_flutter_quickstart/widgets/widgets.dart'; +import 'package:flutter/cupertino.dart'; +import 'package:flutter/material.dart'; + +class Home extends StatefulWidget { + const Home({Key? key}) : super(key: key); + + @override + State createState() => _HomeState(); +} + +class _HomeState extends State { + + @override + Widget build(BuildContext context) { + return Scaffold( + resizeToAvoidBottomInset: true, + backgroundColor: Colors.white, + /* AppBar */ + appBar: buildAppBar(context), + body: SingleChildScrollView( + child: Container( + padding: const EdgeInsets.symmetric(horizontal: 30), + child: Column( + children: const [ + /* Balance */ + + /* Create Wallet */ + + /* Send Transaction */ + ], + ), + ), + )); + } +} +``` + +We now have an app title section and a structure to hold the rest of our app components. + + + +## Calling bdk-flutter methods + +To call all methods properly from the `bdk-flutter` package, first, we need to create state variables to store `Wallet` and `Blockchain` objects. + +Here we use the late keyword to declare both `Wallet` and `Blockchain`. These are non-nullable variables that are initialized after the declaration. + +```dart +import 'package:bdk_flutter/bdk_flutter.dart'; + +late Wallet wallet; +late Blockchain blockchain; +``` + +The first step in creating a non-custodial bitcoin app is creating a mnemonic seed phrase for the wallet. + +`bdk-flutter` provides a `Mnemonic` class to create a `Mnemonic`. The `create` method is a named constructor and can be used to create a mnemonic, it takes `WordCount` as its required parameter. + +```dart +var res = await Mnemonic.create(WordCount.Words12); +``` + +We can generate a mnemonic of longer length by passing in a wordCount argument of required length. + +To create a mnemonic with a `WordCount` of 18 words, we can use `(WordCount.Words18)` +Refer to the API docs on [pub.dev](https://pub.dev/documentation/bdk_flutter/latest/bdk_flutter/bdk_flutter-library.html) for more details. + +```dart + var res = await Mnemonic.create(WordCount.Words18); +// here response is saved as a 'Mnemonic' object +``` + +In order to use this in our Flutter app, we want a button that will generate a mnemonic when clicked, and a text input box to show the generated mnemonic. Let's first create a `TextEditingController` for the `mnemonic` textfield to store the mnemonic, and an internal `generateMnemonicHandler` method which can be called on button click. We will also need a button that will call the internal `generateMnemonicHandler` method when clicked. Adding the following code achieves all of this. + +```dart +class Home extends StatefulWidget { + const Home({Key? key}) : super(key: key); + + @override + State createState() => _HomeState(); + } + +class _HomeState extends State { + late Wallet wallet; + late Blockchain blockchain; + TextEditingController mnemonic = TextEditingController(); + + Future generateMnemonicHandler() async { + var res = await Mnemonic.create(WordCount.Words12); + setState(() { + mnemonic.text = res.asString(); + }); + } + @override + Widget build(BuildContext context) { + return Scaffold( + resizeToAvoidBottomInset: true, + backgroundColor: Colors.white, + /* Header */ + appBar: buildAppBar(context), + body: SingleChildScrollView( + child: Container( + padding: const EdgeInsets.symmetric(horizontal: 30), + child: Column( + children: [ + /* Balance */ + + /* Result */ + + + /* Create Wallet */ + StyledContainer( + child: Column( + mainAxisAlignment: MainAxisAlignment.start, + crossAxisAlignment: CrossAxisAlignment.center, + children: [ + SubmitButton( + text: "Generate Mnemonic", + callback: () async { + await generateMnemonicHandler(); + } + ), + TextFieldContainer( + child: TextFormField( + controller: mnemonic, + style: Theme.of(context).textTheme.bodyText1, + keyboardType: TextInputType.multiline, + maxLines: 5, + decoration: const InputDecoration( + hintText: "Enter your mnemonic")), + ), + ] + ) + ), + /* Send Transaction Buttons */ + + ], + ), + ), + )); + } +} +``` + +Now we need to add a component to display the output of our method calls and this will also need a `displayText` variable to track our method call response. To achieve this add the following code. + +```dart +// screens/home.dart + +// add this as another state variable under mnemonic +String? displayText; + +// modify the generateMnemonicHandler method to also set mnemonic as displayText + + Future generateMnemonicHandler() async { + var res = await Mnemonic.create(WordCount.Words12); + setState(() { + mnemonic.text = res.asString(); + displayText = res.asString(); + }); + } +``` + +and finally, let's add the component to display the output under ` /* Result */` + +```dart +// screens/home.dart + + /* Result */ + // display the component only if displayText has a value + ResponseContainer(text:displayText ?? "No Response"), +``` + +We should now have a working "Generate Mnemonic" button that displays the new mnemonic + + + +A quick recap, we added a button to call a click handler (`generateMnemonicHandler`) which calls `generateMnemonic` API of `bdk-flutter`. The click handler also sets the state for the app and also updates the `displayText` variable to display the output of the call in the display section. We will follow this pattern for the remaining calls to `bdk-flutter`. + +## Creating a wallet + +Before moving on to creating a wallet, let's add a section at the top to display the balance of the wallet. + +To display the balance we will need a state variable to store the balance and a display component to display it. We will also be creating a receive address for the wallet so a state variable will be required for the address as well. + +Under the `mnemonic` and `displayText` state variables, let's add one for `balance` and one for `address` + +```dart +class _HomeState extends State { + TextEditingController mnemonic = TextEditingController(); + String? displayText; + String? balance; + String? address; +``` + +Just below `/* Balance */` and above `/* Result */` add the following UI components to display the balance. We only want to show the balance when it has a value so we will use a null-aware operator `??` for a quick `null` check and use `0` in case of a `null` value. + +```dart + /* Balance */ + BalanceContainer( + text: "${balance ?? "0"} Sats", + ), + /* Result */ +``` + +`bdk_flutter` creates a wallet using output descriptors which define the derivation path to derive addresses and sign transactions. More about output descriptors [here](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md). Before creating the `Wallet` we need to create a `descriptor` object which will be used to generate receive addresses and a `changeDescriptor` object to to create change addresses to collect from outgoing transactions. + +`bdk_flutter`'s `Descriptor` class has a number of descriptor templates that will help you create a simple wallet. + +Let's add some code to create a simple `wpkh` descriptor object by using the `BIP84` template. This template will create a descriptor in the format `wpkh(key/84'/{0,1}'/0'/{0,1}/*)` + +This descriptor will create receive (`KeyChainKind.External`) and change descriptor (` KeyChainKind.Internal`) for a specified mnemonic. + +```dart +Future> getDescriptors(String mnemonic) async { + final descriptors = []; + try { + for (var e in [KeychainKind.External, KeychainKind.Internal]) { + final mnemonicObj = await Mnemonic.fromString(mnemonic); + final descriptorSecretKey = await DescriptorSecretKey.create( + network: Network.Testnet, + mnemonic: mnemonicObj, + ); + final descriptor = await Descriptor.newBip84( + secretKey: descriptorSecretKey, + network: Network.Testnet, + keychain: e, + ); + descriptors.add(descriptor); + } + return descriptors; + } on Exception catch (e) { + setState(() { + displayText = "Error : ${e.toString()}"; + }); + rethrow; + } + } + +``` + +Under the `address` state variable, let's add a state variable called `wallet` of the type `Wallet` for saving the bitcoin wallet. + +To create a wallet with `bdk-flutter` call the `create` constructor with `descriptor`, `changeDescriptor` `network`, and the `databaseConfig`. For database, we can use memory as the database by specifying `DatabaseConfig.memory()` +Following our pattern of a button, click handler and bdk-flutter API call, Let's add an internal method which will serve as the click handler for the "Create Wallet" button. We want to see the output of this call so let's use `setState()` to set the `wallet` object created and the `displayText` variable with the wallet's first receive address. + +```dart + Future createOrRestoreWallet( + String mnemonic, Network network, String? password) async { + try { + final descriptors = await getDescriptors(mnemonic); + final res = await Wallet.create( + descriptor: descriptors[0], + changeDescriptor: descriptors[1], + network: network, + databaseConfig: const DatabaseConfig.memory()); + var addressInfo = await res.getAddress(addressIndex: const AddressIndex()); + setState(() { + address = addressInfo.address; + wallet = res; + displayText = "Wallet Created: $address"; + }); + } on Exception catch (e) { + setState(() { + displayText = "Error: ${e.toString()}"; + }); + } + } +``` + +A new button will be required to call `createOrRestoreWallet()` + +Let's add a new button just below the mnemonic `TextFieldContainer` + +```dart +SubmitButton( + text: "Create Wallet", + callback: () async { + await createOrRestoreWallet( + mnemonic.text, + Network.Testnet, + "password", + ); + }, + ), +``` + +The response returned by `create()` is a `Wallet` object. + +The App should now be creating a wallet when we click **Create Mnemonic** followed by **Create Wallet**. + + + +Before going forward, we need to create a `Blockchain` object as well. The Blockchain object will encapsulate the bitcoin node configuration which the wallet will use for syncing blocks and broadcasting transactions. + +Let's add an internal method to create and initialize the `Blockchain` object. + +```dart + Future blockchainInit() async { + blockchain = await Blockchain.create( + config: BlockchainConfig.electrum( + config: ElectrumConfig( + stopGap: 10, + timeout: 5, + retry: 5, + url: "ssl://electrum.blockstream.info:60002", + validateDomain: false))); + } +``` + +Here we are initializing the `late` non-nullable `blockchain` variable, by calling the named constructor `create`, which takes a `BlockchainConfig` object. +The bitcoin node specified is an Electrum node and we are specifying the url for Blockstream's public Electrum Testnet servers over SSL. + +After creating the `blockchainInit()` method, call it from `createOrRestoreWallet()`, so the `blockchain` variable gets initialized before the `wallet` is created. + +Include the following line of code inside `createOrRestoreWallet() ` just before calling Wallet.create(). + +```dart +.... + await blockchainInit(); + final res = await Wallet.create( + ..... +``` + +**blockChainConfig**: BlockchainConfig is an enum that has 3 values, `BlockchainConfig.electrum` for [`electrum`](https://github.com/romanz/electrs) ,`BlockchainConfig.esplora` for [`esplora`](https://github.com/Blockstream/esplora) and `BlockchainConfig.rpc` . + +`BlockchainConfig.electrum`, `BlockchainConfig.rpc` & `BlockchainConfig.esplora` has `ElectrumConfig` object, `RpcConfig` object and `EsploraConfig` object, respectively as its parameter. + +**ElectrumConfig**: This is the object type of `BlockchainConfig.electrum`'s config that takes a timeout, retry & url as its required parameter. + +**EsploraConfig**: This is the object type of `BlockchainConfig.esplora`'s config that takes baseUrl & stopGap as its required parameter. + +**RpcConfig**: This is the object type of `BlockchainConfig.rpc`'s config that takes url, network, & walletName as its required parameter. If `Rpc Blockchain` has its authentication values inside a cookie file, please pass in cookie path as authCookie parameter, or you can pass in rpc username and password using `UserPass` class. + +Refer to the readme for a complete list of options for [createWallet()](https://github.com/LtbLightning/bdk-flutter#createwallet) + +## UTXOs and balance + +With the `Wallet` and `Blockchain` created, we can now add methods to sync UTXOs and compute balance. + +`Wallet` has a `sync` method to sync all UTXOs belonging to the wallet using the `Blockchain` object. Once the wallet sync is complete balance is computed and `getBalance` can fetch the balance. + +Earlier we have already added a variable for `balance`. Now we will add buttons to call `sync` and `getBalance`. Just below the Create Wallet button let's add two buttons as follows: + +```dart + SubmitButton( text: "Sync Wallet", + callback: () async { await syncWallet(); }, + ), + + SubmitButton( callback: () async { await getBalance(); }, + text: "Get Balance", + ), +``` + +Let's add two internal functions for syncing UTXOs and compute balance. + +```dart + Future getBalance() async { + final balanceObj = await wallet.getBalance(); + final res = "Total Balance: ${balanceObj.total.toString()}"; + print(res); + setState(() { + balance = balanceObj.total.toString(); + displayText = res; + }); + } + + Future syncWallet() async { + wallet.sync(blockchain); + } + +``` + +We should now be able to create a wallet, sync UTXOs, and get the balance + + + +We can use a public testnet faucet to send testnet coins to the wallet and check that the UTXO sync and balance fetch are working correctly. Before we do that add one more method to generate a new address we can then use this address to get testnet coins from a faucet. + +Let's use the `address` variable that was created before for this, we need to add a button for **Get Address** and an internal function to call `Wallet` and create a new address. Let's do the following + +Add a new `getNewAddress` function below the `syncWallet()` function: + +```dart + Future getNewAddress() async { + final res = await wallet.getAddress(addressIndex: const AddressIndex()); + setState(() { + displayText = res.address; + address = res.address; + }); + } +``` + +And a **Get Address** button below the existing **Get Balance** button: + +```dart + SubmitButton( + callback: () async { await getNewAddress(); }, + text: "Get Address" + ), +``` + +We should now have the following, and **Get Address** will be able to display a new address. + + + +Now that we are able to generate a receive address we can get some testnet bitcoin from one of the public [testnet faucets](https://coinfaucet.eu/en/btc-testnet/) + +After we send and after the transaction is confirmed we will need to sync the wallet before we can see the new balance from the received transaction. + +## Restoring a wallet + +The `create` method creates a wallet using a `mnemonic`, to restore we can use the same method, we won't need to call `generateMnemonic` as we will already have a `mnemonic` to restore with. + +This text field below the `Generate Mnemonic` button will also display the mnemonic variable if we click Generate Mnemonic' button. The generated mnemonic will show up in the text field. We can overwrite it with our mnemonic and doing so will also overwrite the mnemonic state variable. This way the mnemonic displayed will be the one used to create the wallet. + +We can now use our mnemonic and use it to restore a wallet. This will come in handy if we have a wallet with testnet bitcoin as these are hard to come by. + + + +## Sending bitcoin + +We are now able to receive bitcoin, now its time to add functionality to send a bitcoin transaction. + +For making a successful bitcoin transaction `bdk-flutter` utilizes a couple of methods. A new unsigned transaction can be created by using TxBuilder](https://github.com/LtbLightning/bdk-flutter#quicksend). + +First, we have to initialize the `TxBuilder` object and call the `addRecipient()` method. +`addRecipient()` takes a `Script` object and the transaction `amount`. + +```dart + final res = await txBuilder.addRecipient(script, amount); +``` + +We can create the`Script` object by using the `Address` class, by specifying the recipient address. + +```dart + final address = await Address.create(address: addressStr); + final script = await address.scriptPubKey(); + final res = await txBuilder.addRecipient(script, amount); +``` + +We can create a `psbt` object by calling the `finish()` method using the response object from `addRecipient()` method. + +```dart + final txBuilder = TxBuilder(); + final address = await Address.create(address: addressStr); + final script = await address.scriptPubKey(); + final psbt = await txBuilder + .addRecipient(script, amount) + .feeRate(1.0) + .finish(wallet); +``` + +This `psbt` can be signed later with [sign()](https://github.com/LtbLightning/bdk-flutter#signtx) method from the `Wallet` and broadcast using [broadcast()](https://github.com/LtbLightning/bdk-flutter#broadcasttx) from the `Blockchain` . + +We will need textfield controllers for the recipient address, amount, and for transaction, these can be added below our existing variable for `mnemonic` + +```dart + TextEditingController recipientAddress = TextEditingController(); + TextEditingController amount = TextEditingController(); +``` + +Let's make an internal function to send a bitcoin transaction, using `Wallet`, `Blockchain` and `TxBuilder `. + +```dart + Future sendTx(String addressStr, int amount) async { + try { + final txBuilder = TxBuilder(); + final address = await Address.create(address: addressStr); + final script = await address.scriptPubKey(); + final txBuilderResult = await txBuilder + .addRecipient(script, amount) + .feeRate(1.0) + .finish(wallet); + final sbt = await wallet.sign(psbt: txBuilderResult.psbt); + final tx = await sbt.extractTx(); + await blockchain.broadcast(tx); + setState(() { + displayText = "Successfully broadcast $amount Sats to $addressStr"; + }); + } on Exception catch (e) { + setState(() { + displayText = "Error: ${e.toString()}"; + }); + } + } + +``` + +Add a new section for send transaction functionality. We will need a `form`, a `TextFormField` for the receiver address and a `TextFormField` for the amount to send. We will also need a button to call the `sendTx` function. + +Before submitting the form we need to make sure all the input fields are valid, for that purpose, we need to initialize a [`GlobalKey`](https://api.flutter.dev/flutter/widgets/GlobalKey-class.html). This can be added above our `Scaffold` + +```dart +final _formKey = GlobalKey(); +``` + +Let's add the send transaction section and UI components below `/* Send Transaction */` + +```dart + StyledContainer( + child: Form( + key: _formKey, + child: Column( + mainAxisAlignment: MainAxisAlignment.start, + crossAxisAlignment: CrossAxisAlignment.center, + children: [ + TextFieldContainer( + child: TextFormField( + controller: recipientAddress, + validator: (value) { + if (value == null || value.isEmpty) { + return 'Please enter your address'; + } + return null; + }, + style: Theme.of(context).textTheme.bodyText1, + decoration: const InputDecoration( + hintText: "Enter Address", + ), + ), + ), + TextFieldContainer( + child: TextFormField( + controller: amount, + validator: (value) { + if (value == null || value.isEmpty) { + return 'Please enter the amount'; + } + return null; + }, + keyboardType: TextInputType.number, + style: Theme.of(context).textTheme.bodyText1, + decoration: const InputDecoration( + hintText: "Enter Amount", + ), + ), + ), + SubmitButton( + text: "Send Bit", + callback: () async { + if (_formKey.currentState!.validate()) { + await sendTx(recipientAddress.text, + int.parse(amount.text)); + } + }, + ) + ] + ), + ) + ) + +``` + +We should now be able to send a transaction as long as there is sufficient balance. + + + +## Conclusion + +The App we created can be built and distributed for both iOS and Android thus sharing a code base and reducing development time. The development and coding focused on application-level code for use cases and we did not have to code intricate internal bitcoin protocol-level code or bitcoin node interactions, and transactions. UTXOs and sync-related functionalities were also not required. All this was managed by `bdk-flutter` allowing us to focus on the product, functionality, and user journey. This is how `bdk` and `bdk-flutter` intend to make Rapid Bitcoin Application Development possible by allowing product and application developers to focus on what they know best while `bdk` handles bitcoin complexity. + +`bdk-flutter` intends to expose functionality and APIs from `bdk` which has a wide variety of APIs with granular details allowing for many interesting use cases to be implemented. `bdk-flutter` and `bdk` are constantly updated and enhanced based on feedback from product teams and developers in the bitcoin community. + +Stay tuned for more APIs and enhancements coming to `bdk-flutter` in the near future. Feature and API requests are most welcome. New blogs and tutorials will be published soon for a more in-depth exploration of `bdk-flutter`. + +In the meantime keep in touch with the project by following us on [GitHub](https://github.com/LtbLightning/bdk-flutter) and [Twitter](https://twitter.com/BitcoinZavior) + +#### References: + +- [bdk](https://github.com/bitcoindevkit) +- [bdk-flutter](https://github.com/LtbLightning/bdk-flutter) +- [bdk-flutter-quickstart GitHub Repository](https://github.com/LtbLightning/bdk-flutter-quickstart) +- [Setup Flutter Development Environment](https://docs.flutter.dev/get-started/install) +- [Mastering Bitcoin(HD Wallet chapter)](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch04.asciidoc) +- [Bitcoin Output Descriptors from bitcoin GitHub](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md) +- Testnet Faucet: [https://coinfaucet.eu/en/btc-testnet/](https://coinfaucet.eu/en/btc-testnet/) or [https://bitcoinfaucet.uo1.net](https://bitcoinfaucet.uo1.net) diff --git a/docs/_blog/exploring_bdk_flutter/assets_section.png b/docs/_blog/exploring_bdk_flutter/assets_section.png new file mode 100644 index 0000000000..1795f43021 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/assets_section.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_complete_app.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_complete_app.png new file mode 100644 index 0000000000..33bf34d094 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_complete_app.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_address.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_address.png new file mode 100644 index 0000000000..2317ab596b Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_address.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_balance.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_balance.png new file mode 100644 index 0000000000..3c19a6c713 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_balance.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_restore.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_restore.png new file mode 100644 index 0000000000..5d0e4fbed6 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_get_restore.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_send.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_send.png new file mode 100644 index 0000000000..1ad60e0174 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_send.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_title.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_title.png new file mode 100644 index 0000000000..f7594d2000 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_title.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_tutorial_screen_createwallet.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_tutorial_screen_createwallet.png new file mode 100644 index 0000000000..09446b4265 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_tutorial_screen_createwallet.png differ diff --git a/docs/_blog/exploring_bdk_flutter/bdk_flutter_tutorial_screen_mnemonic.png b/docs/_blog/exploring_bdk_flutter/bdk_flutter_tutorial_screen_mnemonic.png new file mode 100644 index 0000000000..a42f893b03 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/bdk_flutter_tutorial_screen_mnemonic.png differ diff --git a/docs/_blog/exploring_bdk_flutter/default_flutter_app.png b/docs/_blog/exploring_bdk_flutter/default_flutter_app.png new file mode 100644 index 0000000000..5209c91476 Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/default_flutter_app.png differ diff --git a/docs/_blog/exploring_bdk_flutter/folder_structure.png b/docs/_blog/exploring_bdk_flutter/folder_structure.png new file mode 100644 index 0000000000..f836c67c8e Binary files /dev/null and b/docs/_blog/exploring_bdk_flutter/folder_structure.png differ diff --git a/docs/_blog/exploring_bdk_rn.md b/docs/_blog/exploring_bdk_rn.md new file mode 100644 index 0000000000..c696943304 --- /dev/null +++ b/docs/_blog/exploring_bdk_rn.md @@ -0,0 +1,652 @@ +--- +title: "BDK-RN: Building React Native Apps with BDK" +description: "A tutorial and guide to using bdk-rn for building bitcoin apps" +authors: + - Bitcoin Zavior +date: "2022-08-05" +tags: ["bitcoin", "React Native", "iOS", "Android", "mobile", "bdk-rn", "bdk", "tutorial", "guide", "wallet"] +--- + +## Introduction + +`bdk-rn` is the **Bitcoin Dev kit**'s **React Native** library which enables building bitcoin applications for Android and iOS mobile platforms. Using `bdk-rn` does not require knowledge of the underlying bitcoin or BDK API. Using `bdk-rn` is similar to using any other RN module. Just do `yarn add bdk-rn` and you are ready to code! This is the first tutorial on how to use `bdk-rn`, more coming soon, make sure to [follow](https://twitter.com/BitcoinZavior?ref_src=twsrc%5Etfw) to be notified of new ones. In case you missed it, there is a recorded `bdk-rn` focused Twitch Livestream available on the [Bitcoin Developers](https://www.youtube.com/watch?v=gMpWA875go4) YouTube channel which covers most of this article, make sure to subscribe to Bitcoin Developers [YouTube Channel](https://www.youtube.com/channel/UCUq_ZdezVWKPvkWRicAYxLA/videos) for more bitcoin development videos. + +In this tutorial, we will explore `bdk-rn` usage and the API it provides. This guide will walk through the development process and code for making a bitcoin application. The bitcoin application we create will be a non-custodial HD Wallet. The application will have the functionality to create a new wallet or restore from a known mnemonic seed phrase. This application will also be able to interact with the bitcoin network to sync UTXOs from new blocks and broadcast transactions. + +The tutorial will focus on bitcoin and `bdk-rn` concepts and API. So it will gloss over React Native aspects. The code for this tutorial is available on the [LtbLightning GitHub](https://github.com/LtbLightning/BdkRnQuickStart) + + +BDK RN Quick Start + +### Prerequisites + +In order to use `bdk-rn` in a React Native App, a React Native development environment is required. Please refer to resources out there on the internet if you need to set this up, here is one of many good resources to guide you on [environment setup](https://reactnative.dev/docs/environment-setup) + +### Bitcoin Basics + +The bitcoin concepts used in this blog post are detailed and explained very well in external bitcoin resources. Here are some links for reference: + +[Mastering Bitcoin(HD Wallet chapter)](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch04.asciidoc) + +[Bitcoin Output Descriptors from bitcoin GitHub](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md) + +Now let's jump into Bitcoin Dev Kit + +## Bitcoin Dev Kit and bdk-rn + +`bdk-rn` is a React Native library of Bitcoin Dev Kit(BDK) for building React Native Apps. +It encapsulates all of the low-level APIs and methods for BDK and exposes them in a react native context. To use BDK in React Native(RN) apps only the `bdk-rn` module is required. `Bdk-rn` can be used like any other react native library and is available on [public package managers(npm and yarn)](https://www.npmjs.com/package/bdk-rn). + +## Getting Started + +Although we won't delve deep into RN we will focus more on bitcoin and bdk-rn, however, some rudimentary RN setup is required, especially a basic RN app to add our code. + + start by creating a new RN project. + +`npx react-native init BdkRnQuickStart` + +If this fails in an error on an M1/M2 Mac please use + `arch -x86_64 pod install --repo-update` + +Once done `cd` into the new project directory and run the basic RN app that's created + +```shell +cd BdkRnQuickStart +yarn ios +``` + +This should start building the app and launch the app in a simulator. So far we have created a basic RN project if this doesn't work then refer to the React Native development setup guide to troubleshoot. + + + + + +## Setting up styles and RN app structure + +Let's set up a very basic app structure and some RN scaffolding. Let's create an `src` folder in the project root and inside it add new folders for `assets`, `elements`, `screens` and `styles` + +To make this quick you can download the styles and images used in the tutorial from the repository. The image assets, `Button.tsx` and `styles.js` can be taken from [here](https://github.com/LtbLightning/BdkRnQuickStart/tree/master/src) and moved to the folders as shown. Alternatively, you can write your own styles and use your own images if you intend to style the app in a different way. + + Create a `home.js` file under `screens` folder, this will be where we will be adding most of the code. + +Once done the project structure should look like this: + + + + + +Locate `App.js` in the project root, this will have the default code added by `react-native init`, let's delete all contents of `App.js` and replace it with code to import `home.js` as our main screen. + +```javascript +// App.js + +import React from 'react'; +import Home from './src/screens/home'; + +const App = () => { + return ; +}; + +export default App; +``` + +This will probably crash your app in the simulator but that's fine, it will be fixed in the next step. + +## Installing `bdk-rn` + +With the RN app project in place, we can now add `bdk-rn` using either npm or yarn. + +Using npm: + +```shell +npm i --save bdk-rn +``` + +Using yarn: + +```shell +yarn add bdk-rn +``` + +[iOS Only] Install pods: + +```shell +npx pod-install +or +cd ios && pod install && cd .. +``` + +Verify that `bdk-rn` has been added to `package.json`, once done `bdk-rn` is installed and ready to be used in our **BdkRnQuickStart** App. + +## Importing `bdk-rn` + +Locate `home.js` which we added in the setup section and import `bdk-rn` and also create an RN functional component. + +```javascript +// screens/home.js + +import BdkRn from 'bdk-rn'; + +const Home = () => { +} + +export default Home; +``` + + +Before we start using `bdk-rn` let's add some additional RN component imports, as well as import styles, a button and image assets to create a basic layout to build our home screen. + +```jsx +// screens/home.js + +import BdkRn from 'bdk-rn'; +import React, { Fragment, useState } from 'react'; +import { + ActivityIndicator, + SafeAreaView, + ScrollView, + StatusBar, + Text, + TextInput, + View, + Image, +} from 'react-native'; +import Button from '../elements/Button'; +import { styles } from '../styles/styles'; +const bitcoinLogo = require('../assets/bitcoin_logo.png'); +const bdkLogo = require('../assets/bdk_logo.png'); + +const Home = () => { + // BDK-RN method calls and state variables will be added here + + return ( + + + + {/* Header */} + + + BDK-RN Tutorial + + + + {/* Balance */} + + {/* method call result */} + + {/* buttons for method calls */} + + {/* input boxes and send transaction button */} + + + + ); +}; + +export default Home; +``` + +We now have an app title section and a structure to hold the rest of our app components. + + + + + + + +## Calling `bdk-rn` methods + +All `bdk-rn` methods return a JSON response with data and error properties. All methods return a response as follows: + +```javascript +Promise = { + error: true | false; // success returns true else false. + data: string | object | any; // output data for the method call. +} +``` + +The first step in creating a non-custodial bitcoin app is creating a mnemonic seed phrase for the wallet. + +`bdk-rn` provides `generateMnemonic()` method to create a default 12 word length mnemonic. + +```javascript +import BdkRn from 'bdk-rn'; + +const response = await BdkRn.generateMnemonic(); +const mnemonic = response.data; +``` + +We can specify a longer length or we can also specify the bits of entropy we need by passing the length or entropy arguments. + +To create a mnemonic with an entropy of 256 bits, which will be a 24-word length mnemonic sentence, we can use `{ entropy: 256 }`. +Refer to the readme file on [GitHub](https://github.com/LtbLightning/bdk-rn#generatemnemomic) for more details. + +```javascript +const {data: mnemonic} = await BdkRn.generateMnemonic({ entropy: 256 }); +// here data is destructured and saved as 'mnemonic' +``` + +In order to use this in our RN app let's create a state variable to store the mnemonic and internal `generateMnemonic` method which we can invoke when a button is clicked. We will also need a button which will invoke generateMnemonic when clicked. Adding the following code achieves all of this. + +```jsx +// screens/home.js + +const Home = () => { + // BDK-RN method calls and state variables will be added here + // state variable to store and set mnemonic + const [mnemonic, setMnemonic] = useState(''); + + // internal method to call bdk-rn to generate mnemonic + const getMnemonic = async () => { + // call bdk-rn to generate mnemonic + const {data} = await BdkRn.generateMnemonic({ + length: 12 + }); + // save generated mnemonic to state variable + setMnemonic(data); + }; + +return ( + + + + {/* Header */} + + + BDK-RN Tutorial + + + + {/* Balance */} + + {/* method call result */} + + {/* buttons for method calls */} + +