diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e5daf0..1299c9e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,13 +1,20 @@ # Unreleased -- **Breaking:** `BnbMetric`'s `score`, `bound`, and `drain` take the `target: Target` as a parameter, and `CoinSelector::run_bnb`/`bnb_solutions` gain a leading `target` argument. Consequently `LowestFee` and `Changeless` no longer store a `target` field. This removes the target that `Changeless` previously had to keep in sync with its inner metric, and aligns the metric API with the rest of `CoinSelector`, where `target` is always passed in. +- **Breaking:** Replace `Candidate`'s `input_count` and `is_segwit` fields with `segwit_count` and `legacy_count`, fixing `CoinSelector::input_weight` undercounting candidates that group multiple inputs: in a segwit transaction every legacy input still serializes an empty witness (1 WU), which was previously paid once per candidate instead of once per legacy input, so a group of N legacy inputs came out N-1 WU short. Splitting the count by script type also means a single candidate may now mix legacy and segwit inputs and still be priced exactly. Branch and bound also stops treating candidates of equal value and weight as interchangeable when their segwit/legacy counts differ, which could make it skip a cheaper selection. Replaces `Candidate::new` with `Candidate::new_segwit` and `Candidate::new_legacy`. +- **Breaking:** `CoinSelector` now owns its `Target`. It is set once, through the `SelectionProblem` passed to `CoinSelector::new`, and `CoinSelector::target()` returns it, and it is fixed for the life of the selector. Every method that took a `target: Target` argument no longer does, including `excess`, `missing`, `rate_excess`, `implied_fee`, `is_funded`, `is_within_max_weight`, `drain`, `select_until_target_met`, `select_srd`, `run_bnb`, and `bnb_solutions`. The same goes for arguments that merely restated part of the target: `weight` and `implied_feerate` no longer take `TargetOutputs`, `fee` no longer takes `target_value`, and `effective_value` and `select_all_effective` no longer take a `FeeRate`. `BnbMetric`'s methods read the target from the `CoinSelector` they are given — they are now `fn score(&mut self, cs: &CoinSelector<'_>) -> Option`, `fn bound(&mut self, cs: &CoinSelector<'_>) -> Option` and `fn drain(&mut self, cs: &CoinSelector<'_>) -> Drain` — so `LowestFee` no longer stores a `target` field. +- **Breaking:** Add `SelectionProblem`, which owns the target and candidates for one selection run. `CoinSelector::new` now takes `&SelectionProblem` and borrows it for its lifetime. Build one from prebuilt candidates with `SelectionProblem::new_no_ancestors(target, candidates)`. To measure a selection against a second target, pair `SelectionProblem::with_target(target)` with `CoinSelector::with_problem(&problem)`, which carries the selection, the bans and the candidate order over to the new problem. +- Charge selections for the fee needed to bring the union of their unconfirmed ancestors up to the target feerate (CPFP). Build ancestor-aware problems from `Input`/`InputGroup` and `AncestorToBump` with `SelectionProblem::new`; `SelectionProblem::new_no_ancestors` still takes prebuilt candidates. A shared ancestor is charged once, weight and fee are netted over the union, and `CoinSelector::ancestor_bump` reports the amount. Ancestor weight does not count toward `Target::max_weight`, and RBF rule 4 prices only the child. Each candidate's ancestor set is stored as a sorted `&[u32]` slice, so memory and setup time scale with the number of entries rather than candidates × ancestors. +- Add `CoinSelector::ancestor_bump_lower_bound`, the least bump the selection or any selection extending it could still owe, and `CoinSelector::addable_ancestors`. `LowestFee` uses the lower bound to bound ancestor-aware searches tightly: a funded node credits the ancestor surplus a descendant could still reach, and an unfunded one estimates the least child weight each fee constraint needs. The selector keeps that reachable surplus as a running total, so the bound costs the same at any pool size. +- Hard-prune branch-and-bound nodes whose remaining candidates cannot meet the target feerate, using a running total of what the undecided candidates are worth (a port of Bitcoin Core's `SelectCoinsBnB` lookahead). The relaxation credits still-reachable ancestor surplus, so it holds with unconfirmed ancestors too. +- `CoinSelector::is_fundable` no longer rejects a selection that is already funded. Adding a candidate that is worth more than its own weight can still lower the excess, because the first segwit input adds the witness header. - **Breaking:** `BnbMetric` metrics now decide the change output themselves. The trait gains a `drain(&mut self, cs) -> Drain` method; call it on a branch-and-bound solution (or the `LowestFee` metric directly) to get the change output the metric optimized against, instead of computing a separate `ChangePolicy`. - **Breaking:** `CoinSelector::run_bnb` now returns `(Ordf32, Drain)` instead of just `Ordf32`, handing back the change output the metric decided on for the winning selection. - **Breaking:** `LowestFee` no longer takes a `change_policy`. It now takes `dust_relay_feerate: FeeRate` and `drain_weights: DrainWeights`, and adds change only when doing so lowers the long-term fee and the change would not be dust. - Add `DrainWeights::dust_threshold(dust_relay_feerate)`, the minimum value a change output with these weights must have to not be dust. - Add `CoinSelector::select_srd`, a Single Random Draw selector (port of Bitcoin Core's `SelectCoinsSRD`) that adds candidates in random order until the change reaches `change_lower`, producing a healthy-sized (privacy-friendly) change output instead of minimizing fees. Adds the `CHANGE_LOWER` constant for Core's value. -- **Breaking:** `Changeless` is now `Changeless`, wrapping an inner metric it constrains to changeless solutions (e.g. `Changeless`), replacing the previous tuple-composition approach. -- **Breaking:** Removed the `BnbMetric` tuple implementations (`impl BnbMetric for ((A, f32), ...)`). Weighted composition of independent metrics is no longer supported; the only composition still provided is the changeless constraint, now expressed as `Changeless`. If you relied on tuples to blend multiple objectives, there is no drop-in replacement. +- Search branch and bound depth-first (better-bound child first, backtracking in place) instead of best-first over a heap of cloned branches. Only the current path is held in memory, and under a round cap it reaches complete selections on large pools where the old frontier often ran out of rounds first. +- Seed branch and bound with the greedy selection, so a search that runs out of rounds returns the best selection it has instead of `NoBnbSolution::RoundLimit`. `RoundLimit` now means the round budget ran out before even the greedy selection was scored (e.g. `max_rounds` is 0), or the metric rejected it. +- **Breaking:** Remove the `Changeless` metric and the `BnbMetric` tuple implementations (`impl BnbMetric for ((A, f32), ...)`). Generic metric composition is no longer supported. `LowestFee` decides for itself whether a selection should carry change (adding one only when it lowers the long-term fee, clears the dust threshold, and fits `Target::max_weight`), so a separate changeless objective duplicates that decision and then constrains it. Callers that required a changeless transaction should use `LowestFee` and inspect the returned `Drain`. - **Breaking:** `CoinSelector::selected_indices` and `CoinSelector::banned` now return `&Bitset` instead of `&BTreeSet`. `Bitset` exposes `contains`/`len`/`is_empty`/`iter` (#46) - Replace the internal `Cow`/`Cow<[usize]>` selection state with a `Bitset` and an `Arc`-shared candidate order, making the per-branch clones in branch-and-bound substantially cheaper (#46) - Fix compilation error when building with `--no-default-features` (#36) diff --git a/README.md b/README.md index 4335d4b..5a2bf3f 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ ```rust use std::str::FromStr; -use bdk_coin_select::{ CoinSelector, Candidate, TR_KEYSPEND_TXIN_WEIGHT, Drain, FeeRate, Target, ChangePolicy, TargetOutputs, TargetFee, DrainWeights}; +use bdk_coin_select::{ CoinSelector, Candidate, SelectionProblem, TR_KEYSPEND_TXIN_WEIGHT, Drain, FeeRate, Target, ChangePolicy, TargetOutputs, TargetFee, DrainWeights}; use bitcoin::{ Amount, Address, Network, Transaction, TxIn, TxOut }; let recipient_addr: Address = "tb1pvjf9t34fznr53u5tqhejz4nr69luzkhlvsdsdfq9pglutrpve2xq7hps46" @@ -33,34 +33,34 @@ let candidates = vec![ Candidate { // How many inputs does this candidate represents. Needed so we can // figure out the weight of the varint that encodes the number of inputs - input_count: 1, + // and whether segwit transaction fields need to be counted in. + segwit_count: 1, + legacy_count: 0, // the value of the input value: 1_000_000, // the total weight of the input(s) including their witness/scriptSig // you may need to use miniscript to figure out the correct value here. weight: TR_KEYSPEND_TXIN_WEIGHT, - // wether it's a segwit input. Needed so we know whether to include the - // segwit header in total weight calculations. - is_segwit: true }, Candidate { // A candidate can represent multiple inputs in the case where you // always want some inputs to be spent together. - input_count: 2, + segwit_count: 2, + legacy_count: 0, weight: 2*TR_KEYSPEND_TXIN_WEIGHT, value: 3_000_000, - is_segwit: true } ]; // You can now select coins! -let mut coin_selector = CoinSelector::new(&candidates); +let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); +let mut coin_selector = CoinSelector::new(&problem); coin_selector.select(0); -assert!(!coin_selector.is_funded(target), "we didn't select enough"); -println!("we didn't select enough yet we're missing: {}", coin_selector.missing(target)); +assert!(!coin_selector.is_funded(), "we didn't select enough"); +println!("we didn't select enough yet we're missing: {}", coin_selector.missing()); coin_selector.select(1); -assert!(coin_selector.is_funded(target), "we should have enough now"); +assert!(coin_selector.is_funded(), "we should have enough now"); // Now we need to know if we need a change output to drain the excess if we overshot too much // @@ -69,7 +69,7 @@ assert!(coin_selector.is_funded(target), "we should have enough now"); let drain_weights = DrainWeights::TR_KEYSPEND; // Our policy is to only add a change output if the value is over 1_000 sats let change_policy = ChangePolicy::min_value(drain_weights, 1_000); -let change = coin_selector.drain(target, change_policy); +let change = coin_selector.drain(change_policy); if change.is_some() { println!("We need to add our change output to the transaction with {} value", change.value); } else { @@ -89,7 +89,7 @@ metric by implementing the [`BnbMetric`] yourself but we don't recommend this. ```rust use std::str::FromStr; -use bdk_coin_select::{ BnbMetric, Candidate, CoinSelector, FeeRate, Target, TargetFee, TargetOutputs, TR_KEYSPEND_TXIN_WEIGHT}; +use bdk_coin_select::{ BnbMetric, Candidate, CoinSelector, FeeRate, SelectionProblem, Target, TargetFee, TargetOutputs, TR_KEYSPEND_TXIN_WEIGHT}; use bdk_coin_select::metrics::LowestFee; use bitcoin::{ Address, Amount, Network, Transaction, TxIn, TxOut }; @@ -105,36 +105,38 @@ let outputs = vec![TxOut { let candidates = [ Candidate { - input_count: 1, + segwit_count: 1, + legacy_count: 0, value: 400_000, weight: TR_KEYSPEND_TXIN_WEIGHT, - is_segwit: true }, Candidate { - input_count: 1, + segwit_count: 1, + legacy_count: 0, value: 200_000, weight: TR_KEYSPEND_TXIN_WEIGHT, - is_segwit: true }, Candidate { - input_count: 1, + segwit_count: 1, + legacy_count: 0, value: 11_000, weight: TR_KEYSPEND_TXIN_WEIGHT, - is_segwit: true } ]; let drain_weights = bdk_coin_select::DrainWeights::default(); // You could determine this by looking at the user's transaction history and taking an average of the feerate. let long_term_feerate = FeeRate::from_sat_per_vb(10.0); -let mut coin_selector = CoinSelector::new(&candidates); - let target = Target { fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(15.0)), outputs: TargetOutputs::fund_outputs(outputs.iter().map(|output| (output.weight().to_wu(), output.value.to_sat()))), max_weight: None, }; +let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + +let mut coin_selector = CoinSelector::new(&problem); + // The feerate used to work out whether a change output would be dust (and so shouldn't be added). // The standard dust relay feerate is 3 sat/vb. let dust_relay_feerate = FeeRate::from_sat_per_vb(3.0); @@ -150,13 +152,13 @@ let mut metric = LowestFee { // We run the branch and bound algorithm with a max round limit of 100,000. // On success it returns the score along with the change output the metric decided on. -let change = match coin_selector.run_bnb(target, metric, 100_000) { +let change = match coin_selector.run_bnb(metric, 100_000) { Err(err) => { println!("failed to find a solution: {}", err); // fall back to naive selection - coin_selector.select_until_target_met(target).expect("a selection was impossible!"); + coin_selector.select_until_target_met().expect("a selection was impossible!"); // the metric still decides the change output for whatever we end up selecting - metric.drain(&coin_selector, target) + metric.drain(&coin_selector) } Ok((score, change)) => { println!("we found a solution with score {}", score); @@ -175,6 +177,53 @@ println!("We are including a change output of {} value (0 means not change)", ch ``` +## Unconfirmed ancestors + +Use `SelectionProblem::new` when spending unconfirmed UTXOs. Supply every unconfirmed transaction +that created an input and all of its transitive unconfirmed ancestors; missing transaction ids are +treated as confirmed and can make the required CPFP fee too low. Parent lists contain direct parents +only. Ancestors shared by several selected inputs are charged once over their union. + +```rust +use bdk_coin_select::{ + AncestorToBump, FeeRate, Input, SelectionProblem, Target, TargetFee, TargetOutputs, +}; + +let target = Target { + fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(5.0)), + outputs: TargetOutputs::fund_outputs([(136, 50_000)]), + max_weight: None, +}; +let inputs = [Input { + value: 100_000, + weight: 272, + is_segwit: true, + residing_txid: "child", +}]; +let ancestors = [ + AncestorToBump { + txid: "parent", + weight: 400, + fee: 100, + parents: vec![], + }, + AncestorToBump { + txid: "child", + weight: 600, + fee: 200, + parents: vec!["parent"], + }, +]; +let problem = SelectionProblem::new(target, inputs, ancestors); +let mut coin_selector = problem.selector(); +coin_selector.select(0); +// 1000 wu of ancestors at 1.25 sat/wu owe 1250 sats, of which they already pay 300. +assert_eq!(coin_selector.ancestor_bump(), 950); +``` + +Adding an input may drag in more fee debt than value, so funding is not necessarily monotone for +ancestor-aware problems. `run_bnb` accounts for this and de-duplicates shared ancestors. + # Minimum Supported Rust Version (MSRV) This library is compiles on rust v1.54 and above diff --git a/benches/coin_selector.rs b/benches/coin_selector.rs index c05eabb..56f3e30 100644 --- a/benches/coin_selector.rs +++ b/benches/coin_selector.rs @@ -1,10 +1,15 @@ //! Benchmarks for `CoinSelector`. //! -//! Two groups: //! - `clone`: direct cost of `CoinSelector::clone()`, the operation `Bitset` //! was introduced to make cheap. //! - `run_bnb_lowest_fee`: end-to-end Branch-and-Bound throughput on a //! deterministic synthetic pool using the `LowestFee` metric. +//! - `run_bnb_lowest_fee_ancestors`: the same, but where a third of the coins sit on unconfirmed +//! ancestors that need bumping — covering both the private and shared ancestor paths, which cost +//! different amounts per fee calculation. +//! - `new_with_ancestors`: building a `SelectionProblem` with ancestors and a `CoinSelector` over it +//! at wallet-to-exchange pool sizes. Setup has to scale with the ancestor entries, not candidates +//! times ancestors, or a large pool loses its time budget before the search starts. //! //! Run with `cargo bench`. Filter with `cargo bench -- `. @@ -14,8 +19,9 @@ #![allow(clippy::incompatible_msrv)] use bdk_coin_select::{ - metrics::LowestFee, Candidate, CoinSelector, DrainWeights, FeeRate, Target, TargetFee, - TargetOutputs, TR_SPK_WEIGHT, TXIN_BASE_WEIGHT, TXOUT_BASE_WEIGHT, + metrics::LowestFee, AncestorToBump, Candidate, CoinSelector, DrainWeights, FeeRate, Input, + SelectionProblem, Target, TargetFee, TargetOutputs, TR_SPK_WEIGHT, TXIN_BASE_WEIGHT, + TXOUT_BASE_WEIGHT, }; use criterion::{criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion}; use std::hint::black_box; @@ -33,8 +39,8 @@ fn make_candidates(n: usize) -> Vec { Candidate { value, weight: TXIN_BASE_WEIGHT + P2WPKH_SAT_W, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, } }) .collect() @@ -56,7 +62,9 @@ fn bench_coin_selector_clone(c: &mut Criterion) { let mut group = c.benchmark_group("clone"); for &n in &[64usize, 256, 1024, 4096] { let candidates = make_candidates(n); - let mut selector = CoinSelector::new(&candidates); + let (target, _) = make_bnb_inputs(&candidates); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut selector = CoinSelector::new(&problem); // Select ~10% of candidates so `selected` is non-trivial to copy. for i in (0..n).step_by(10) { selector.select(i); @@ -74,8 +82,9 @@ fn bench_run_bnb_lowest_fee(c: &mut Criterion) { group.sample_size(20); for &n in &[20usize, 50, 100, 200] { let candidates = make_candidates(n); - let selector = CoinSelector::new(&candidates); let (target, long_term_feerate) = make_bnb_inputs(&candidates); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let selector = CoinSelector::new(&problem); group.bench_with_input(BenchmarkId::from_parameter(n), &n, |b, _| { b.iter_batched( || selector.clone(), @@ -85,7 +94,7 @@ fn bench_run_bnb_lowest_fee(c: &mut Criterion) { dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), drain_weights: DrainWeights::TR_KEYSPEND, }; - let _ = sel.run_bnb(target, metric, black_box(100_000)); + let _ = sel.run_bnb(metric, black_box(100_000)); sel }, BatchSize::SmallInput, @@ -95,5 +104,112 @@ fn bench_run_bnb_lowest_fee(c: &mut Criterion) { group.finish(); } -criterion_group!(benches, bench_coin_selector_clone, bench_run_bnb_lowest_fee); +/// A third of the coins sit on a two-long unconfirmed chain: an unpaid parent and a tip that pays a +/// little. Without `share`, every such coin has a chain of its own (private ancestors). With +/// `share`, they all sit on the *same* chain, which is the case that cannot be folded into the +/// candidates up front and has to be de-duplicated per selection. +fn make_ancestor_problem(n: usize, share: bool) -> SelectionProblem { + const P2WPKH_SAT_W: u64 = 107; + const CONFIRMED: usize = usize::MAX; + + let mut ancestors = Vec::new(); + let mut residing = Vec::with_capacity(n); + let mut shared_tip = None; + for i in 0..n { + if i % 3 != 0 { + residing.push(CONFIRMED); + continue; + } + match (share, shared_tip) { + (true, Some(tip)) => residing.push(tip), + _ => { + let parent = ancestors.len(); + ancestors.push(AncestorToBump { + txid: parent, + weight: 800, + fee: 0, + parents: vec![], + }); + let tip = ancestors.len(); + ancestors.push(AncestorToBump { + txid: tip, + weight: 800, + fee: 200, + parents: vec![parent], + }); + residing.push(tip); + shared_tip = Some(tip); + } + } + } + + let value = |i: usize| 1_000 + i as u64 * 137 + (i * i) as u64; + let inputs = (0..n).map(|i| Input { + value: value(i), + weight: TXIN_BASE_WEIGHT + P2WPKH_SAT_W, + is_segwit: true, + residing_txid: residing[i], + }); + let total: u64 = (0..n).map(value).sum(); + let target = Target { + fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(2.0)), + outputs: TargetOutputs::fund_outputs([(TXOUT_BASE_WEIGHT + TR_SPK_WEIGHT, total / 2)]), + max_weight: None, + }; + SelectionProblem::new(target, inputs, ancestors) +} + +fn bench_run_bnb_lowest_fee_ancestors(c: &mut Criterion) { + let mut group = c.benchmark_group("run_bnb_lowest_fee_ancestors"); + group.sample_size(20); + for &share in &[false, true] { + let kind = match share { + false => "private", + true => "shared", + }; + for &n in &[20usize, 50, 100] { + let problem = make_ancestor_problem(n, share); + let selector = CoinSelector::new(&problem); + group.bench_with_input(BenchmarkId::new(kind, n), &n, |b, _| { + b.iter_batched( + || selector.clone(), + |mut sel| { + let metric = LowestFee { + long_term_feerate: FeeRate::from_sat_per_vb(10.0), + dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), + drain_weights: DrainWeights::TR_KEYSPEND, + }; + let _ = sel.run_bnb(metric, black_box(100_000)); + sel + }, + BatchSize::SmallInput, + ); + }); + } + } + group.finish(); +} + +fn bench_new_with_ancestors(c: &mut Criterion) { + let mut group = c.benchmark_group("new_with_ancestors"); + group.sample_size(10); + for &n in &[20_000usize, 200_000] { + group.bench_with_input(BenchmarkId::from_parameter(n), &n, |b, &n| { + b.iter(|| { + let problem = make_ancestor_problem(black_box(n), false); + let selector = CoinSelector::new(&problem); + black_box(selector.ancestor_bump_lower_bound()); + }); + }); + } + group.finish(); +} + +criterion_group!( + benches, + bench_coin_selector_clone, + bench_run_bnb_lowest_fee, + bench_run_bnb_lowest_fee_ancestors, + bench_new_with_ancestors +); criterion_main!(benches); diff --git a/src/bnb.rs b/src/bnb.rs index 0498e5c..00bfda5 100644 --- a/src/bnb.rs +++ b/src/bnb.rs @@ -1,226 +1,417 @@ -use core::cmp::Reverse; - -use crate::{float::Ordf32, Drain, Target}; +use crate::{float::Ordf32, Drain}; use super::CoinSelector; -use alloc::collections::BinaryHeap; +use alloc::vec::Vec; /// An [`Iterator`] that iterates over rounds of branch and bound to minimize the score of the /// provided [`BnbMetric`]. +/// +/// The tree is searched depth-first, visiting the child with the better bound first and +/// backtracking in place, so only the current path is held in memory. #[derive(Debug)] pub(crate) struct BnbIter<'a, M: BnbMetric> { - queue: BinaryHeap>, + selector: CoinSelector<'a>, + stack: Vec, best: Option, - /// The target the metric scores selections against. - pub(crate) target: Target, + /// The greedy selection, yielded before the first node is expanded. Its score is `best`: + /// nothing else can have run yet, so the two are set together. See + /// [`seed_greedy_incumbent`](BnbIter::seed_greedy_incumbent). + seed: Option>, + exhausted: bool, /// The `BnBMetric` that will score each selection pub(crate) metric: M, } +/// A decision on the current path: either `index` was selected, or `banned` (`index` and the +/// candidates interchangeable with it) were banned. +#[derive(Debug)] +struct Frame { + is_inclusion: bool, + index: usize, + /// Position of `index` in the candidate order. + cursor: usize, + /// Position to resume scanning for the next undecided candidate below this frame. + next_cursor: usize, + banned: Vec, + /// Whether the other child of this frame's parent still needs visiting. + sibling_pending: bool, +} + impl<'a, M: BnbMetric> Iterator for BnbIter<'a, M> { type Item = Option<(CoinSelector<'a>, Ordf32)>; fn next(&mut self) -> Option { + if let Some(seed) = self.seed.take() { + let score = self.best.expect("the seed and `best` are set together"); + return Some(Some((seed, score))); + } + + if self.exhausted { + return None; + } + // { // println!("=========================== {:?}", self.best); - // for thing in self.queue.iter() { - // println!("{} {:?}", &thing.selector, thing.lower_bound); + // println!("{} {:?}", &self.selector, self.bound_of_current(self.cursor())); + // for frame in self.stack.iter() { + // println!( + // "\t{} [{}] cursor={} sibling_pending={}", + // if frame.is_inclusion { "IN " } else { "EX " }, + // frame.index, + // frame.cursor, + // frame.sibling_pending, + // ); // } // let _ = std::io::stdin().read_line(&mut alloc::string::String::new()); // } - let branch = self.queue.pop()?; - if let Some(best) = &self.best { - // If the next thing in queue is not better than our best we're done. - if *best < branch.lower_bound { - // println!( - // "\t\t(SKIP) branch={} inclusion={} lb={:?}, score={:?}", - // branch.selector, - // !branch.is_exclusion, - // branch.lower_bound, - // self.metric.score(&branch.selector), - // ); - return None; - } - } - // println!( - // "\t\t( POP) branch={} inclusion={} lb={:?}, score={:?}", - // branch.selector, - // !branch.is_exclusion, - // branch.lower_bound, - // self.metric.score(&branch.selector), - // ); - - let selector = branch.selector; + // An exclusion node has the same selection as its parent, which was already scored. + let return_val = if !self.is_exclusion_node() { + self.try_record_best() + .map(|score| (self.selector.clone(), score)) + } else { + None + }; - let mut return_val = None; - if !branch.is_exclusion { - if let Some(score) = self.metric.score(&selector, self.target) { - let better = match self.best { - Some(best_score) => score < best_score, - None => true, - }; - if better { - self.best = Some(score); - return_val = Some(score); - } - }; + if !self.descend() && !self.backtrack_to_next_branch() { + self.exhausted = true; } - self.insert_new_branches(&selector); - Some(return_val.map(|score| (selector, score))) + Some(return_val) } } impl<'a, M: BnbMetric> BnbIter<'a, M> { - pub(crate) fn new(mut selector: CoinSelector<'a>, target: Target, metric: M) -> Self { + pub(crate) fn new(mut selector: CoinSelector<'a>, metric: M) -> Self { + if metric.requires_ordering_by_descending_value_pwu() { + selector.sort_candidates_by_descending_value_pwu(); + } + let mut iter = BnbIter { - queue: BinaryHeap::default(), + selector, + stack: Vec::new(), best: None, - target, + seed: None, + exhausted: false, metric, }; - if iter.metric.requires_ordering_by_descending_value_pwu() { - selector.sort_candidates_by_descending_value_pwu(); - } + iter.seed_greedy_incumbent(); - iter.consider_adding_to_queue(&selector, false); + if !iter.bound_is_promising(0) { + iter.exhausted = true; + } iter } - fn consider_adding_to_queue(&mut self, cs: &CoinSelector<'a>, is_exclusion: bool) { - let bound = self.metric.bound(cs, self.target); - if let Some(bound) = bound { - let is_good_enough = match self.best { - Some(best) => best > bound, - None => true, - }; - if is_good_enough { - let branch = Branch { - lower_bound: bound, - selector: cs.clone(), - is_exclusion, - }; - /*println!( - "\t\t(PUSH) branch={} inclusion={} lb={:?} score={:?}", - branch.selector, - !branch.is_exclusion, - branch.lower_bound, - self.metric.score(&branch.selector), - );*/ - self.queue.push(branch); - } /* else { - println!( - "\t\t( REJ) branch={} inclusion={} lb={:?} score={:?}", - cs, - !is_exclusion, - bound, - self.metric.score(cs), - ); - }*/ - } /*else { - println!( - "\t\t(NO B) branch={} inclusion={} score={:?}", - cs, - !is_exclusion, - self.metric.score(cs), - ); - }*/ - } - - fn insert_new_branches(&mut self, cs: &CoinSelector<'a>) { - let (next_index, next) = match cs.unselected().next() { - Some(c) => c, - None => return, // exhausted + /// Score the greedy prefix and adopt it as the incumbent. + /// + /// Without this the search is not anytime: a caller that runs out of rounds before the first + /// complete selection gets nothing back and falls through to whatever fallback it has, which on + /// a large pool is far worse than the selection a single greedy pass would have handed it. The + /// seed costs one round and one scored selection, and since it is only an incumbent — the bound + /// is unchanged and still admissible — the optimum stays reachable. + /// + /// It yields nothing for a metric that rejects the greedy prefix outright. + fn seed_greedy_incumbent(&mut self) { + let mut seed = self.selector.clone(); + if seed.select_until_target_met().is_err() { + return; + } + if let Some(score) = self.metric.score(&seed) { + self.best = Some(score); + self.seed = Some(seed); + } + } + + fn is_exclusion_node(&self) -> bool { + self.stack.last().map_or(false, |frame| !frame.is_inclusion) + } + + fn try_record_best(&mut self) -> Option { + let decided_before = self.cursor(); + self.selector.set_decided_before(decided_before); + let score = self.metric.score(&self.selector)?; + let better = match self.best { + Some(best_score) => score < best_score, + None => true, }; + if better { + self.best = Some(score); + Some(score) + } else { + None + } + } + + fn is_promising(&self, bound: Option) -> bool { + match (bound, self.best) { + (Some(bound), Some(best)) => best > bound, + (Some(_), None) => true, + (None, _) => false, + } + } + + /// Bound the current node, telling the selector how much of the candidate order it can skip. + /// + /// Every candidate before `decided_before` has already been decided — included by an inclusion + /// frame, or banned by an exclusion one — so a metric asking about undecided candidates never + /// has to look at them. That is what keeps the cost of a node proportional to the answer rather + /// than to the depth it was found at. + fn bound_of_current(&mut self, decided_before: usize) -> Option { + self.selector.set_decided_before(decided_before); + self.metric.bound(&self.selector) + } + + fn bound_is_promising(&mut self, decided_before: usize) -> bool { + let bound = self.bound_of_current(decided_before); + self.is_promising(bound) + } + + fn cursor(&self) -> usize { + self.stack.last().map_or(0, |frame| frame.next_cursor) + } - let mut inclusion_cs = cs.clone(); - inclusion_cs.select(next_index); - self.consider_adding_to_queue(&inclusion_cs, false); + /// The first undecided candidate at or after `start` in the candidate order, as + /// `(index, cursor)`. + fn next_candidate(&self, start: usize) -> Option<(usize, usize)> { + for (cursor, (index, _)) in (start..).zip(self.selector.candidates_from(start)) { + if !self.selector.is_selected(index) && !self.selector.banned().contains(index) { + return Some((index, cursor)); + } + } + None + } - // for the exclusion branch, we keep banning if candidates have the same weight and value - let mut is_first_ban = true; - let mut exclusion_cs = cs.clone(); - let to_ban = (next.value, next.weight); - for (next_index, next) in cs.unselected() { - if (next.value, next.weight) != to_ban { + /// The candidates to ban when excluding `index`, and the cursor to resume from. + /// + /// For the exclusion branch, we keep banning candidates that are interchangeable with the one + /// we exclude: same value, weight and input counts, and dragging in exactly the same unconfirmed + /// ancestors. The counts matter because a segwit and a legacy input of equal weight change the + /// tx weight differently, and two coins of equal value and weight are not interchangeable if one + /// of them drags in an ancestor that needs bumping. Candidates are only compared until the first + /// mismatch, since this exploits them being adjacent in the sorted order. + fn exclusion_plan(&self, index: usize, cursor: usize) -> (Vec, usize) { + let next = self.selector.candidate(index); + let to_ban = ( + next.value, + next.weight, + next.segwit_count, + next.legacy_count, + ); + let to_ban_drags_in = self.selector.problem().drags_in(index); + let mut banned = alloc::vec![index]; + let mut next_cursor = cursor + 1; + for (next_index, next) in self.selector.candidates_from(cursor + 1) { + if self.selector.is_selected(next_index) || self.selector.banned().contains(next_index) + { + next_cursor += 1; + continue; + } + if ( + next.value, + next.weight, + next.segwit_count, + next.legacy_count, + ) != to_ban + || self.selector.problem().drags_in(next_index) != to_ban_drags_in + { break; } - let (_index, _candidate) = exclusion_cs - .candidates() - .find(|(i, _)| *i == next_index) - .expect("must have index since we are planning to ban it"); - if is_first_ban { - is_first_ban = false; - } /*else { - println!("banning: [{}] {:?}", _index, _candidate); - }*/ - exclusion_cs.ban(next_index); + // println!("banning: [{}] {:?}", next_index, next); + banned.push(next_index); + next_cursor += 1; } - self.consider_adding_to_queue(&exclusion_cs, true); + (banned, next_cursor) } -} -#[derive(Debug, Clone)] -struct Branch<'a> { - lower_bound: Ordf32, - selector: CoinSelector<'a>, - is_exclusion: bool, -} + fn apply_exclude(&mut self, banned: &[usize]) { + for &index in banned { + self.selector.ban(index); + } + } -impl Ord for Branch<'_> { - fn cmp(&self, other: &Self) -> core::cmp::Ordering { - // NOTE: Reverse comparision `lower_bound` because we want a min-heap (by default BinaryHeap - // is a max-heap). - // NOTE: We tiebreak equal scores based on whether it's exlusion or not (preferring - // inclusion). We do this because we want to try and get to evaluating complete selection - // returning actual scores as soon as possible. - core::cmp::Ord::cmp( - &(Reverse(&self.lower_bound), !self.is_exclusion), - &(Reverse(&other.lower_bound), !other.is_exclusion), - ) + fn undo_exclude(&mut self, banned: &[usize]) { + for &index in banned { + self.selector.unban(index); + } } -} -impl PartialOrd for Branch<'_> { - fn partial_cmp(&self, other: &Self) -> Option { - Some(self.cmp(other)) + fn push_include(&mut self, index: usize, cursor: usize, sibling_pending: bool) { + self.selector.select(index); + self.stack.push(Frame { + is_inclusion: true, + index, + cursor, + next_cursor: cursor + 1, + banned: Vec::new(), + sibling_pending, + }); } -} -impl PartialEq for Branch<'_> { - fn eq(&self, other: &Self) -> bool { - self.lower_bound == other.lower_bound + fn push_exclude( + &mut self, + index: usize, + cursor: usize, + banned: Vec, + next_cursor: usize, + sibling_pending: bool, + ) { + self.apply_exclude(&banned); + self.stack.push(Frame { + is_inclusion: false, + index, + cursor, + next_cursor, + banned, + sibling_pending, + }); } -} -impl Eq for Branch<'_> {} + /// Step into the more promising child of the current node. Returns `false` if neither child + /// can beat the incumbent (or there are no undecided candidates left). + fn descend(&mut self) -> bool { + let (index, cursor) = match self.next_candidate(self.cursor()) { + Some(next) => next, + None => return false, + }; + + self.selector.select(index); + let inc_bound = self.bound_of_current(cursor + 1); + let inc_ok = self.is_promising(inc_bound); + self.selector.deselect(index); + + let (banned, exc_next_cursor) = self.exclusion_plan(index, cursor); + self.apply_exclude(&banned); + let exc_bound = self.bound_of_current(exc_next_cursor); + let exc_ok = self.is_promising(exc_bound); + self.undo_exclude(&banned); + + // println!( + // "\t\t(DESC) branch={} next=[{}] inc_lb={:?}{} exc_lb={:?}{}", + // self.selector, + // index, + // inc_bound, + // if inc_ok { "" } else { " (REJ)" }, + // exc_bound, + // if exc_ok { "" } else { " (REJ)" }, + // ); + + match (inc_ok, exc_ok) { + (false, false) => false, + (true, false) => { + self.push_include(index, cursor, false); + true + } + (false, true) => { + self.push_exclude(index, cursor, banned, exc_next_cursor, false); + true + } + (true, true) => { + // NOTE: We tiebreak equal bounds by preferring inclusion. We do this because we + // want to try and get to evaluating complete selections as soon as possible. + let include_first = match (inc_bound, exc_bound) { + (Some(inc), Some(exc)) => inc <= exc, + _ => true, + }; + if include_first { + self.push_include(index, cursor, true); + } else { + self.push_exclude(index, cursor, banned, exc_next_cursor, true); + } + true + } + } + } + + /// Unwind the path until a frame whose pending sibling can still beat the incumbent, and step + /// into that sibling. Returns `false` once the whole tree is exhausted. + /// + /// The sibling's bound is recomputed here rather than reused from [`descend`](Self::descend): + /// the incumbent may have improved since. + fn backtrack_to_next_branch(&mut self) -> bool { + while let Some(frame) = self.stack.pop() { + // println!( + // "\t\t(BACK) undo {} [{}] sibling_pending={}", + // if frame.is_inclusion { "IN " } else { "EX " }, + // frame.index, + // frame.sibling_pending, + // ); + if frame.is_inclusion { + self.selector.deselect(frame.index); + if frame.sibling_pending { + let (banned, next_cursor) = self.exclusion_plan(frame.index, frame.cursor); + self.apply_exclude(&banned); + if self.bound_is_promising(next_cursor) { + self.stack.push(Frame { + is_inclusion: false, + index: frame.index, + cursor: frame.cursor, + next_cursor, + banned, + sibling_pending: false, + }); + return true; + } + self.undo_exclude(&banned); + } + } else { + self.undo_exclude(&frame.banned); + if frame.sibling_pending { + self.selector.select(frame.index); + if self.bound_is_promising(frame.cursor + 1) { + self.stack.push(Frame { + is_inclusion: true, + index: frame.index, + cursor: frame.cursor, + next_cursor: frame.cursor + 1, + banned: Vec::new(), + sibling_pending: false, + }); + return true; + } + self.selector.deselect(frame.index); + } + } + } + false + } +} /// A branch and bound metric where we minimize the [`Ordf32`] score. /// /// This is to be used as input for [`CoinSelector::run_bnb`] or [`CoinSelector::bnb_solutions`]. +/// +/// Every selection passed to these methods carries its own [`Target`](crate::Target), reachable via +/// [`CoinSelector::target`]. [`bound`] is only a valid lower bound on the [`score`] of the +/// descendants of `cs` because the search clones `cs` to build them, so every node of a single +/// search is guaranteed to have the same target. +/// +/// [`bound`]: BnbMetric::bound +/// [`score`]: BnbMetric::score pub trait BnbMetric { - /// Get the score of a given selection for `target`. + /// Get the score of the selection `cs` against [`cs.target()`](CoinSelector::target). /// /// If this returns `None`, the selection is invalid. - fn score(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option; + fn score(&mut self, cs: &CoinSelector<'_>) -> Option; - /// Get the lower bound score using a heuristic for `target`. + /// Get the lower bound score, using a heuristic, against + /// [`cs.target()`](CoinSelector::target). /// /// This represents the best possible score of all descendant branches (according to the /// heuristic). /// /// If this returns `None`, the current branch and all descendant branches will not have valid /// solutions. - fn bound(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option; + fn bound(&mut self, cs: &CoinSelector<'_>) -> Option; - /// The change output (a.k.a. drain) this metric decides on for the given selection and `target`, - /// or [`Drain::NONE`] if it decides there should be no change. + /// The change output (a.k.a. drain) this metric decides on for the selection `cs` and + /// [`cs.target()`](CoinSelector::target), or [`Drain::NONE`] if it decides there should be no + /// change. /// /// Call this on a branch-and-bound solution to get the change output the metric optimized against. - fn drain(&mut self, cs: &CoinSelector<'_>, target: Target) -> Drain; + fn drain(&mut self, cs: &CoinSelector<'_>) -> Drain; /// Returns whether the metric requies we order candidates by descending value per weight unit. fn requires_ordering_by_descending_value_pwu(&self) -> bool { diff --git a/src/coin_selector.rs b/src/coin_selector.rs index 604abd8..50b2614 100644 --- a/src/coin_selector.rs +++ b/src/coin_selector.rs @@ -1,7 +1,9 @@ use super::*; #[allow(unused)] // some bug in <= 1.48.0 sees this as unused when it isn't use crate::float::FloatExt; -use crate::{bitset::Bitset, bnb::BnbMetric, float::Ordf32, ChangePolicy, FeeRate, Target}; +use crate::{ + bitset::Bitset, bnb::BnbMetric, float::Ordf32, ChangePolicy, FeeRate, SelectionProblem, Target, +}; use alloc::{sync::Arc, vec::Vec}; /// The minimum change amount Bitcoin Core's `SelectCoinsSRD` targets; a sensible default for the @@ -17,30 +19,278 @@ pub const CHANGE_LOWER: u64 = 50_000; /// [`bnb_solutions`]: CoinSelector::bnb_solutions #[derive(Debug, Clone)] pub struct CoinSelector<'a> { - candidates: &'a [Candidate], + problem: &'a SelectionProblem, selected: Bitset, banned: Bitset, candidate_order: Arc>, + /// Running sums over the selected candidates, kept up to date by [`select`](Self::select) and + /// [`deselect`](Self::deselect) so the aggregate queries don't rescan the selection. + selected_value: u64, + selected_weight: u64, + selected_segwit_count: usize, + selected_legacy_count: usize, + /// Running sums over the unconfirmed ancestors the selection drags in. See + /// [`ancestor_bump`](Self::ancestor_bump). + ancestors: SelectionTotals, + /// Position in the candidate order before which every candidate is already decided — selected + /// or banned. See [`set_decided_before`](Self::set_decided_before). + decided_before: usize, +} + +/// Running totals over the unconfirmed ancestors the selected candidates drag in, over the surplus +/// the reachable ones (neither selected nor banned) could still bring, and over what those +/// reachable candidates are worth. +/// +/// [`CoinSelector`] decides *when* a candidate's ancestors arrive or leave (when its selected or +/// banned bit actually changes); the bookkeeping for *what* that changes lives here. +#[derive(Debug, Clone, PartialEq, Eq)] +struct SelectionTotals { + /// `(weight, fee)` of the selected candidates' private ancestors. Each is reachable through one + /// candidate only, so a plain sum never counts one twice. + private: (u64, u64), + /// How many selected candidates drag in each shared ancestor. Empty unless the problem has + /// shared ancestors. + shared_refcounts: Vec, + /// `(weight, fee)` of the shared ancestors with a non-zero refcount, each counted once. + shared: (u64, u64), + /// Summed [`SelectionProblem::ancestor_surplus`] of the private ancestors of every reachable + /// candidate, each candidate's group netted as one. + reachable_private_surplus: u64, + /// How many reachable candidates drag in each shared ancestor. Empty unless the problem has + /// shared ancestors. + reachable_shared_refcounts: Vec, + /// Summed [`SelectionProblem::ancestor_surplus`] of the shared ancestors that some reachable + /// candidate drags in and no selected candidate does yet. + reachable_shared_surplus: u64, + /// Value and weight of the reachable candidates worth selecting, i.e. those whose standalone + /// effective value is positive. Candidates that cost more weight than they bring are left out + /// because they only ever lower the total, so this stays an upper bound on what the rest of + /// this branch can still contribute. + undecided: (u64, u64), +} + +impl SelectionTotals { + fn new(problem: &SelectionProblem) -> Self { + let shared_len = if problem.has_shared_ancestors() { + problem.ancestors().len() + } else { + 0 + }; + let mut totals = Self { + private: (0, 0), + shared_refcounts: alloc::vec![0; shared_len], + shared: (0, 0), + reachable_private_surplus: 0, + reachable_shared_refcounts: alloc::vec![0; shared_len], + reachable_shared_surplus: 0, + undecided: (0, 0), + }; + // Nothing is selected or banned yet, so every candidate is reachable. + for index in 0..problem.len() { + totals.add_reachable(problem, index); + } + totals + } + + /// Summed surplus of the ancestors reachable candidates could still bring in. + fn reachable_surplus(&self) -> u64 { + self.reachable_private_surplus + self.reachable_shared_surplus + } + + /// Summed `(weight, fee)` of every ancestor the selection drags in, each counted once. + fn selected(&self) -> (u64, u64) { + ( + self.private.0 + self.shared.0, + self.private.1 + self.shared.1, + ) + } + + /// Candidate `index` was selected. + fn add_selected(&mut self, problem: &SelectionProblem, index: usize) { + if problem.has_private_ancestors() { + let (weight, fee) = problem.private_ancestors(index); + self.private.0 += weight; + self.private.1 += fee; + } + if problem.has_shared_ancestors() { + for &anc_index in problem.shared_drags_in(index) { + let anc_index = anc_index as usize; + let refcount = &mut self.shared_refcounts[anc_index]; + if *refcount == 0 { + let (weight, fee) = problem.ancestors()[anc_index]; + self.shared.0 += weight; + self.shared.1 += fee; + // Now selected, so no longer something a descendant could still add. + if self.reachable_shared_refcounts[anc_index] > 0 { + self.reachable_shared_surplus -= problem.ancestor_surplus((weight, fee)); + } + } + *refcount += 1; + } + } + } + + /// Candidate `index` was deselected. + fn sub_selected(&mut self, problem: &SelectionProblem, index: usize) { + if problem.has_private_ancestors() { + let (weight, fee) = problem.private_ancestors(index); + self.private.0 -= weight; + self.private.1 -= fee; + } + if problem.has_shared_ancestors() { + for &anc_index in problem.shared_drags_in(index) { + let anc_index = anc_index as usize; + let refcount = &mut self.shared_refcounts[anc_index]; + *refcount -= 1; + if *refcount == 0 { + let (weight, fee) = problem.ancestors()[anc_index]; + self.shared.0 -= weight; + self.shared.1 -= fee; + if self.reachable_shared_refcounts[anc_index] > 0 { + self.reachable_shared_surplus += problem.ancestor_surplus((weight, fee)); + } + } + } + } + } + + /// Whether a candidate brings in more value than its own weight costs at the target feerate. + /// One that doesn't can only ever lower a running total, so [`undecided`](Self::undecided) + /// leaves it out and stays an upper bound. + fn is_worth_selecting(problem: &SelectionProblem, index: usize) -> bool { + problem + .candidate(index) + .effective_value(problem.target().fee.rate) + > 0.0 + } + + /// Candidate `index` became reachable (neither selected nor banned). + fn add_reachable(&mut self, problem: &SelectionProblem, index: usize) { + if Self::is_worth_selecting(problem, index) { + let candidate = problem.candidate(index); + self.undecided.0 += candidate.value; + self.undecided.1 += candidate.weight; + } + if problem.has_private_ancestors() { + self.reachable_private_surplus += + problem.ancestor_surplus(problem.private_ancestors(index)); + } + if problem.has_shared_ancestors() { + for &anc_index in problem.shared_drags_in(index) { + let anc_index = anc_index as usize; + let refcount = &mut self.reachable_shared_refcounts[anc_index]; + if *refcount == 0 && self.shared_refcounts[anc_index] == 0 { + self.reachable_shared_surplus += + problem.ancestor_surplus(problem.ancestors()[anc_index]); + } + *refcount += 1; + } + } + } + + /// Candidate `index` stopped being reachable (it was selected or banned). + fn remove_reachable(&mut self, problem: &SelectionProblem, index: usize) { + if Self::is_worth_selecting(problem, index) { + let candidate = problem.candidate(index); + self.undecided.0 -= candidate.value; + self.undecided.1 -= candidate.weight; + } + if problem.has_private_ancestors() { + self.reachable_private_surplus -= + problem.ancestor_surplus(problem.private_ancestors(index)); + } + if problem.has_shared_ancestors() { + for &anc_index in problem.shared_drags_in(index) { + let anc_index = anc_index as usize; + let refcount = &mut self.reachable_shared_refcounts[anc_index]; + *refcount -= 1; + if *refcount == 0 && self.shared_refcounts[anc_index] == 0 { + self.reachable_shared_surplus -= + problem.ancestor_surplus(problem.ancestors()[anc_index]); + } + } + } + } } impl<'a> CoinSelector<'a> { - /// Creates a new coin selector from some candidate inputs and a `base_weight`. + /// Creates a new coin selector for `problem`. /// - /// The `base_weight` is the weight of the transaction without any inputs and without a change - /// output. + /// The [`SelectionProblem`] is fixed for the life of the selector: its target and candidates. + /// Everything the selector reports is measured against that one target. Methods refer to + /// candidates by index into [`SelectionProblem::candidates`]. /// /// The `CoinSelector` does not keep track of the final transaction's output count. The caller /// is responsible for including the potential output-count varint weight change in the /// corresponding [`DrainWeights`]. - /// - /// Note that methods in `CoinSelector` will refer to inputs by the index in the `candidates` - /// slice you pass in. - pub fn new(candidates: &'a [Candidate]) -> Self { + pub fn new(problem: &'a SelectionProblem) -> Self { + let n = problem.len(); Self { - candidates, - selected: Bitset::with_capacity(candidates.len()), - banned: Bitset::with_capacity(candidates.len()), - candidate_order: Arc::new((0..candidates.len()).collect::>()), + problem, + selected: Bitset::with_capacity(n), + banned: Bitset::with_capacity(n), + candidate_order: Arc::new((0..n).collect::>()), + selected_value: 0, + selected_weight: 0, + selected_segwit_count: 0, + selected_legacy_count: 0, + ancestors: SelectionTotals::new(problem), + decided_before: 0, + } + } + + /// What this selector is funding. + pub fn target(&self) -> Target { + self.problem.target() + } + + /// The selection problem this selector is solving. + pub fn problem(&self) -> &'a SelectionProblem { + self.problem + } + + /// A copy of this selector — same selection, bans and candidate order — over `problem` + /// instead. + /// + /// Use this with [`SelectionProblem::with_target`] to measure a selection against a second + /// target, for example to check whether a fee bump needs more inputs. + /// + /// # Panics + /// + /// If `problem` does not have the same number of candidates as this selector's problem, since + /// the selection refers to candidates by index. + /// + /// ``` + /// # use bdk_coin_select::{Candidate, CoinSelector, FeeRate, SelectionProblem, Target, TargetFee, TargetOutputs}; + /// # let candidates = [Candidate::new_tr_keyspend(100_000), Candidate::new_tr_keyspend(100_000)]; + /// let target = Target { + /// outputs: TargetOutputs::fund_outputs([(46 * 4, 90_000)]), + /// fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(1.0)), + /// max_weight: None, + /// }; + /// let problem = SelectionProblem::new_no_ancestors(target, candidates); + /// let mut selector = problem.selector(); + /// selector.select(0); + /// assert!(selector.is_funded()); + /// + /// // Would that same selection still fund the transaction at a much higher feerate? + /// let bumped_problem = problem.with_target(Target { + /// fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(500.0)), + /// ..target + /// }); + /// let bumped = selector.with_problem(&bumped_problem); + /// assert_eq!(bumped.selected_indices(), selector.selected_indices()); + /// assert!(!bumped.is_funded(), "the bump needs another input"); + /// ``` + pub fn with_problem(&self, problem: &'a SelectionProblem) -> CoinSelector<'a> { + assert_eq!( + problem.len(), + self.problem.len(), + "the selection refers to candidates by index, so both problems must have the same candidates" + ); + CoinSelector { + problem, + ..self.clone() } } @@ -49,35 +299,96 @@ impl<'a> CoinSelector<'a> { pub fn candidates( &self, ) -> impl DoubleEndedIterator + ExactSizeIterator + '_ { + let candidates = self.problem.candidates(); self.candidate_order .iter() - .map(move |i| (*i, self.candidates[*i])) + .map(move |i| (*i, candidates[*i])) } - /// Get the candidate at `index`. `index` refers to its position in the original `candidates` - /// slice passed into [`CoinSelector::new`]. + /// [`candidates`](Self::candidates), skipping the first `from_position` of the sorted order. + /// + /// `from_position` is a position in that order, not an index into + /// [`SelectionProblem::candidates`] — unlike the `index` each item carries. It may equal the + /// candidate count, which yields nothing; past that is a caller bug and panics. + pub(crate) fn candidates_from( + &self, + from_position: usize, + ) -> impl DoubleEndedIterator + ExactSizeIterator + '_ { + let cands = self.problem.candidates(); + self.candidate_order[from_position..] + .iter() + .map(move |i| (*i, cands[*i])) + } + + /// Promise that every candidate before `position` in the candidate order is already selected or + /// banned, so scans for undecided candidates may start there. + /// + /// Branch and bound decides candidates in order, so at depth `d` the first `d` positions are + /// all decided. Without this, every query for the undecided candidates walks those `d` entries + /// first, which makes the cost of a node grow with the pool rather than with the answer. Zero + /// is always correct and is where every selector starts; anything that can make an earlier + /// candidate undecided again ([`deselect`](Self::deselect), unbanning, re-sorting) puts it + /// back there. + pub(crate) fn set_decided_before(&mut self, position: usize) { + debug_assert!( + self.candidates() + .take(position) + .all(|(index, _)| self.selected.contains(index) || self.banned.contains(index)), + "an undecided candidate sits before `position`, so skipping the prefix would hide it", + ); + self.decided_before = position; + } + + /// Get the candidate at `index`. `index` refers to its position in + /// [`SelectionProblem::candidates`]. pub fn candidate(&self, index: usize) -> Candidate { - self.candidates[index] + self.problem.candidates()[index] } - /// Deselect a candidate at `index`. `index` refers to its position in the original `candidates` - /// slice passed into [`CoinSelector::new`]. + /// Deselect a candidate at `index`. `index` refers to its position in + /// [`SelectionProblem::candidates`]. pub fn deselect(&mut self, index: usize) -> bool { - self.selected.remove(index) + // This can make a candidate before the decided prefix undecided again. + self.decided_before = 0; + let removed = self.selected.remove(index); + if removed { + let candidate = self.problem.candidates()[index]; + self.selected_value -= candidate.value; + self.selected_weight -= candidate.weight; + self.selected_segwit_count -= candidate.segwit_count; + self.selected_legacy_count -= candidate.legacy_count; + self.ancestors.sub_selected(self.problem, index); + if !self.banned.contains(index) { + self.ancestors.add_reachable(self.problem, index); + } + } + removed } /// Convienince method to pick elements of a slice by the indexes that are currently selected. - /// Obviously the slice must represent the inputs ordered in the same way as when they were - /// passed to `Candidates::new`. + /// Obviously the slice must represent the inputs ordered in the same way as + /// [`SelectionProblem::candidates`]. pub fn apply_selection(&self, candidates: &'a [T]) -> impl Iterator + '_ { self.selected.iter().map(move |i| &candidates[i]) } - /// Select the input at `index`. `index` refers to its position in the original `candidates` - /// slice passed into [`CoinSelector::new`]. + /// Select the input at `index`. `index` refers to its position in + /// [`SelectionProblem::candidates`]. pub fn select(&mut self, index: usize) -> bool { - assert!(index < self.candidates.len()); - self.selected.insert(index) + assert!(index < self.problem.len()); + let inserted = self.selected.insert(index); + if inserted { + let candidate = self.problem.candidates()[index]; + self.selected_value += candidate.value; + self.selected_weight += candidate.weight; + self.selected_segwit_count += candidate.segwit_count; + self.selected_legacy_count += candidate.legacy_count; + if !self.banned.contains(index) { + self.ancestors.remove_reachable(self.problem, index); + } + self.ancestors.add_selected(self.problem, index); + } + inserted } /// Select the next unselected candidate in the sorted order fo the candidates. @@ -94,12 +405,21 @@ impl<'a> CoinSelector<'a> { /// Ban an input from being selected. Banning the input means it won't show up in [`unselected`] /// or [`unselected_indices`]. Note it can still be manually selected. /// - /// `index` refers to its position in the original `candidates` slice passed into [`CoinSelector::new`]. + /// `index` refers to its position in [`SelectionProblem::candidates`]. /// /// [`unselected`]: Self::unselected /// [`unselected_indices`]: Self::unselected_indices pub fn ban(&mut self, index: usize) { - self.banned.insert(index); + if self.banned.insert(index) && !self.selected.contains(index) { + self.ancestors.remove_reachable(self.problem, index); + } + } + + pub(crate) fn unban(&mut self, index: usize) { + self.decided_before = 0; + if self.banned.remove(index) && !self.selected.contains(index) { + self.ancestors.add_reachable(self.problem, index); + } } /// Gets the list of inputs that have been banned by [`ban`]. @@ -109,29 +429,40 @@ impl<'a> CoinSelector<'a> { &self.banned } - /// Is the input at `index` selected. `index` refers to its position in the original - /// `candidates` slice passed into [`CoinSelector::new`]. + /// Is the input at `index` selected. `index` refers to its position in + /// [`SelectionProblem::candidates`]. pub fn is_selected(&self, index: usize) -> bool { self.selected.contains(index) } - /// Whether the candidates can cover this `target`'s **value** (net of input fees) — i.e. whether - /// enough value is reachable for [`is_funded`] to hold. Respects [`ban`]ned candidates. + /// Whether the candidates can cover the [`target`](Self::target)'s **value** (net of input + /// fees) — i.e. whether enough value is reachable for [`is_funded`] to hold. Respects + /// [`ban`]ned candidates. /// - /// Selecting *all* effective inputs maximizes the value available, so if that can't meet the - /// target value, nothing can. Monotone, hence exact. + /// The current selection is checked first, then the current selection plus every remaining + /// candidate with positive effective value. The first check matters because transaction + /// framing can make adding a candidate that is worth more than its own weight lower the excess: + /// the first segwit input adds the witness header. /// /// NOTE: this does **not** account for [`Target::max_weight`] — a `true` result can still be /// infeasible under the weight cap. Use [`select_until_target_met`] or branch and bound (both of /// which enforce the cap) to actually build a selection. /// + /// NOTE: with unconfirmed ancestors ([`SelectionProblem::has_ancestors`]) this is a heuristic + /// and can answer either way. Funding is not monotone then: an input can drag in an ancestor + /// that costs more than the input is worth, and inputs sharing an ancestor pay for it once + /// between them. Use branch and bound to decide feasibility exactly. + /// /// [`ban`]: Self::ban /// [`is_funded`]: Self::is_funded /// [`select_until_target_met`]: Self::select_until_target_met - pub fn is_fundable(&self, target: Target) -> bool { + pub fn is_fundable(&self) -> bool { + if self.is_funded() { + return true; + } let mut test = self.clone(); - test.select_all_effective(target.fee.rate); - test.is_funded(target) + test.select_all_effective(); + test.is_funded() } /// Returns true if no candidates have been selected. @@ -142,59 +473,185 @@ impl<'a> CoinSelector<'a> { /// The weight of the inputs including the witness header and the varint for the number of /// inputs. pub fn input_weight(&self) -> u64 { - let is_segwit_tx = self.selected().any(|(_, wv)| wv.is_segwit); - let witness_header_extra_weight = is_segwit_tx as u64 * 2; - - let input_count = self.selected().map(|(_, wv)| wv.input_count).sum::(); + let input_count = self.selected_segwit_count + self.selected_legacy_count; let input_varint_weight = varint_size(input_count) * 4; - let selected_weight: u64 = self - .selected() - .map(|(_, candidate)| { - let mut weight = candidate.weight; - if is_segwit_tx && !candidate.is_segwit { - // non-segwit candidates do not have the witness length field included in their - // weight field so we need to add 1 here if it's in a segwit tx. - weight += 1; - } - weight - }) - .sum(); + let segwit_extra_weight = if self.selected_segwit_count > 0 { + // The witness header, plus: legacy inputs do not have the witness length included in + // their weight field so we need to add 1 to each if it's a segwit tx. + 2 + self.selected_legacy_count as u64 + } else { + 0 + }; - input_varint_weight + selected_weight + witness_header_extra_weight + input_varint_weight + self.selected_weight + segwit_extra_weight } /// Absolute value sum of all selected inputs. pub fn selected_value(&self) -> u64 { - self.selected - .iter() - .map(|index| self.candidates[index].value) - .sum() + self.selected_value + } + + /// The unconfirmed ancestors the current selection drags in (indices into + /// [`SelectionProblem::ancestors`]). + /// + /// This is the **union** over the selected candidates, so an ancestor shared by several of them + /// appears once. Deselecting a candidate keeps an ancestor that another selected candidate + /// still drags in. + pub fn selected_ancestors(&self) -> Bitset { + let mut union = Bitset::with_capacity(self.problem.ancestors().len()); + if self.problem.has_ancestors() { + for cand_index in self.selected.iter() { + for &anc_index in self.problem.drags_in(cand_index) { + union.insert(anc_index as usize); + } + } + } + union + } + + /// The fee (sats) this selection must pay *on top of* its own feerate obligation so the + /// unconfirmed ancestors it drags in reach `target.fee.rate` (CPFP). + /// + /// Charged over the ancestors this selection drags in, taken **once each** — never by summing + /// [`SelectionProblem::local_bump`], which would charge a shared ancestor once per candidate. + /// Weight and fee are netted across them, so an ancestor paying above the rate offsets one + /// paying below it, and the result saturates at 0 (an ancestor that overpays never funds the + /// child). + /// + /// Note this makes funding **non-monotone**: selecting a candidate that drags in an + /// underpaying ancestor can lower [`excess`](Self::excess). It also means the bump is not + /// additive over candidates, and a descendant selection can owe *less* than its parent (by + /// dragging in an ancestor that already overpays). + pub fn ancestor_bump(&self) -> u64 { + if !self.problem.has_ancestors() { + return 0; + } + crate::selection_problem::ancestor_shortfall( + self.target().fee.rate, + self.ancestors.selected(), + ) + } + + /// The unconfirmed ancestors that are not dragged in yet but could still be, i.e. those of the + /// [`unselected`](Self::unselected) candidates. Respects [`ban`](Self::ban). + /// + /// These are exactly the ancestors a descendant of this selection can add. + pub fn addable_ancestors(&self) -> Bitset { + let mut union = Bitset::with_capacity(self.problem.ancestors().len()); + if self.problem.has_ancestors() { + let already = self.selected_ancestors(); + for cand_index in self.unselected_indices() { + for &anc_index in self.problem.drags_in(cand_index) { + let anc_index = anc_index as usize; + if !already.contains(anc_index) { + union.insert(anc_index); + } + } + } + } + union + } + + /// The least [`ancestor_bump`](Self::ancestor_bump) this selection — or any selection extending + /// it — could still owe. + /// + /// This is **not** the bump of the current selection. A later coin can drag in an ancestor that + /// already overpays the target rate; that surplus nets against the deficit, so a descendant can + /// owe *less*. This method credits every still-reachable surplus and floors at zero: + /// + /// ```text + /// bump of this selection, and of every selection that adds more coins + /// >= max(0, currently_owed − reachable_surplus) + /// ``` + /// + /// where `currently_owed` is `rate · ancestor_weight − ancestor_fee` of this selection, and + /// `reachable_surplus` is how much still-addable ancestors overpay the target rate. + /// + /// Surplus cannot be picked up ancestor by ancestor: ancestors arrive by selecting a + /// *candidate*, which drags in its whole transitive set. So `reachable_surplus` is accumulated + /// per group that must arrive together — the split [`SelectionProblem`] already computed: + /// + /// - Ancestors only one candidate can reach ([`private_ancestors`]) are netted as a group, and + /// contribute only if the group as a whole is in surplus. A chain whose tip overpays but which + /// nets to a deficit therefore offers nothing. + /// - Ancestors several candidates can reach ([`shared_drags_in`]) are credited individually, + /// since which candidate brings them — and what else it brings — is not pinned down. + /// + /// This is still a relaxation: those groups may not be reachable *together*, and reaching them at + /// all means adding candidates (and their child weight). Both only push the real figure up. When + /// nothing reachable overpays, the bound equals the current bump. + /// + /// Constant time: the selector keeps the reachable surplus as a running total, in whole + /// satoshis rounded up per group. What is owed is computed exactly in `f64`, as + /// [`ancestor_bump`](Self::ancestor_bump) is, so the two need no rounding allowance between + /// them; the result can only sit below the exact value, which is the safe direction. + /// + /// [`private_ancestors`]: SelectionProblem::private_ancestors + /// [`shared_drags_in`]: SelectionProblem::shared_drags_in + pub fn ancestor_bump_lower_bound(&self) -> u64 { + if !self.problem.has_ancestors() { + return 0; + } + let (weight, fee) = self.ancestors.selected(); + let owed = weight as f64 * self.target().fee.rate.spwu() as f64 - fee as f64; + let bound = owed - self.ancestors.reachable_surplus() as f64; + if bound <= 0.0 { + 0 + } else { + // Truncating a positive float rounds down, which is the safe direction. + bound as u64 + } + } + + /// The most any descendant of this branch could still improve the feerate constraint. + /// + /// This is Bitcoin Core's `SelectCoinsBnB` lookahead (`curr_available_value`): the selector + /// keeps a running total of what the reachable candidates can contribute, and a node whose + /// total still cannot close the gap has an empty subtree. Constant time. + /// + /// Every term is one-sided, so the result is an over-estimate and never prunes a branch that + /// holds a solution. The undecided pair counts only candidates worth selecting, and the current + /// ancestor bump is swapped for [`ancestor_bump_lower_bound`](Self::ancestor_bump_lower_bound), + /// which holds for this branch and every descendant — so a subsidizing ancestor that a + /// descendant might drag in is credited here rather than assumed away. The input-count varint + /// and witness overhead those candidates would add is ignored for the same reason: leaving it + /// out can only make this larger. + pub(crate) fn best_reachable_rate_excess_wu(&self) -> i64 { + self.rate_excess_wu(Drain::NONE) + self.ancestor_bump() as i64 + - self.ancestor_bump_lower_bound() as i64 + + self.ancestors.undecided.0 as i64 + - self + .target() + .fee + .rate + .implied_fee_wu(self.ancestors.undecided.1) as i64 } /// Current weight of transaction implied by the selection. /// /// If you don't have any drain outputs (only target outputs) just set drain_weights to /// [`DrainWeights::NONE`]. - pub fn weight(&self, target_ouputs: TargetOutputs, drain_weight: DrainWeights) -> u64 { + pub fn weight(&self, drain_weight: DrainWeights) -> u64 { TX_FIXED_FIELD_WEIGHT + self.input_weight() - + target_ouputs.output_weight_with_drain(drain_weight) + + self.target().outputs.output_weight_with_drain(drain_weight) } - /// How much the current selection overshoots the value needed to achieve `target`. + /// How much the current selection overshoots the value needed to achieve the + /// [`target`](Self::target). /// /// In order for the resulting transaction to be valid this must be 0 or above. If it's above 0 - /// this means the transaction will overpay for what it needs to reach `target`. - pub fn excess(&self, target: Target, drain: Drain) -> i64 { - self.rate_excess(target, drain) - .min(self.absolute_excess(target, drain)) - .min(self.replacement_excess(target, drain)) + /// this means the transaction will overpay for what it needs to reach the target. + pub fn excess(&self, drain: Drain) -> i64 { + self.rate_excess(drain) + .min(self.absolute_excess(drain)) + .min(self.replacement_excess(drain)) } /// How much extra value needs to be selected to reach the target. - pub fn missing(&self, target: Target) -> u64 { - let excess = self.excess(target, Drain::NONE); + pub fn missing(&self) -> u64 { + let excess = self.excess(Drain::NONE); if excess < 0 { excess.unsigned_abs() } else { @@ -202,56 +659,58 @@ impl<'a> CoinSelector<'a> { } } - /// How much the current selection overshoots the value need to satisfy `target.fee.rate` and - /// `target.value` (while ignoring `target.fee.absolute`). - pub fn rate_excess(&self, target: Target, drain: Drain) -> i64 { + /// How much the current selection overshoots the value need to satisfy `self.target().fee.rate` and + /// `self.target().value` (while ignoring `self.target().fee.absolute`). + /// + /// The feerate obligation includes the [`ancestor_bump`](Self::ancestor_bump). + pub fn rate_excess(&self, drain: Drain) -> i64 { self.selected_value() as i64 - - target.value() as i64 + - self.target().value() as i64 - drain.value as i64 - - self.implied_fee_from_feerate(target, drain.weights) as i64 + - self.implied_fee_from_feerate(drain.weights) as i64 } - /// Same as [rate_excess](Self::rate_excess) except `target.fee.rate` is applied to the + /// Same as [rate_excess](Self::rate_excess) except `self.target().fee.rate` is applied to the /// implied transaction's weight units directly without any conversion to vbytes. - pub fn rate_excess_wu(&self, target: Target, drain: Drain) -> i64 { + pub fn rate_excess_wu(&self, drain: Drain) -> i64 { self.selected_value() as i64 - - target.value() as i64 + - self.target().value() as i64 - drain.value as i64 - - self.implied_fee_from_feerate_wu(target, drain.weights) as i64 + - self.implied_fee_from_feerate_wu(drain.weights) as i64 } - /// How much the current selection overshoots the value needed to satisfy `target.fee.absolute` - /// and `target.value` (while ignoring `target.fee.rate`). - pub fn absolute_excess(&self, target: Target, drain: Drain) -> i64 { + /// How much the current selection overshoots the value needed to satisfy `self.target().fee.absolute` + /// and `self.target().value` (while ignoring `self.target().fee.rate`). + pub fn absolute_excess(&self, drain: Drain) -> i64 { self.selected_value() as i64 - - target.value() as i64 + - self.target().value() as i64 - drain.value as i64 - - target.fee.absolute as i64 + - self.target().fee.absolute as i64 } /// How much the current selection overshoots the value needed to satisfy RBF's rule 4. - pub fn replacement_excess(&self, target: Target, drain: Drain) -> i64 { + pub fn replacement_excess(&self, drain: Drain) -> i64 { let mut replacement_excess_needed = 0; - if let Some(replace) = target.fee.replace { + if let Some(replace) = self.target().fee.replace { replacement_excess_needed = - replace.min_fee_to_do_replacement(self.weight(target.outputs, drain.weights)) + replace.min_fee_to_do_replacement(self.weight(drain.weights)) } self.selected_value() as i64 - - target.value() as i64 + - self.target().value() as i64 - drain.value as i64 - replacement_excess_needed as i64 } /// Same as [replacement_excess](Self::replacement_excess) except the replacement fee /// is calculated using weight units directly without any conversion to vbytes. - pub fn replacement_excess_wu(&self, target: Target, drain: Drain) -> i64 { + pub fn replacement_excess_wu(&self, drain: Drain) -> i64 { let mut replacement_excess_needed = 0; - if let Some(replace) = target.fee.replace { + if let Some(replace) = self.target().fee.replace { replacement_excess_needed = - replace.min_fee_to_do_replacement_wu(self.weight(target.outputs, drain.weights)) + replace.min_fee_to_do_replacement_wu(self.weight(drain.weights)) } self.selected_value() as i64 - - target.value() as i64 + - self.target().value() as i64 - drain.value as i64 - replacement_excess_needed as i64 } @@ -259,68 +718,107 @@ impl<'a> CoinSelector<'a> { /// The feerate the transaction would have if we were to use this selection of inputs to achieve /// the `target`'s value and weight. It is essentially telling you what target feerate you currently have. /// + /// This is the *child* transaction's feerate: the fee and weight of any unconfirmed ancestors + /// this selection drags in are not included, so it is not the package feerate. + /// /// Returns `None` if the feerate would be negative or infinity. - pub fn implied_feerate(&self, target_outputs: TargetOutputs, drain: Drain) -> Option { - let numerator = - self.selected_value() as i64 - target_outputs.value_sum as i64 - drain.value as i64; - let denom = self.weight(target_outputs, drain.weights); + pub fn implied_feerate(&self, drain: Drain) -> Option { + let numerator = self.selected_value() as i64 + - self.target().outputs.value_sum as i64 + - drain.value as i64; + let denom = self.weight(drain.weights); if numerator < 0 || denom == 0 { return None; } Some(FeeRate::from_sat_per_wu(numerator as f32 / denom as f32)) } - /// The fee the current selection and `drain_weight` should pay to satisfy `target_fee`. + /// The fee the current selection and `drain_weight` should pay to satisfy the + /// [`target`](Self::target)'s [`TargetFee`]. /// /// This compares the fee calculated from the target feerate with the fee calculated from the /// [`Replace`] constraints and returns the larger of the two. /// + /// The feerate component includes the [`ancestor_bump`](Self::ancestor_bump); the absolute and + /// replacement components are child-transaction constraints and are left alone. + /// /// `drain_weight` can be 0 to indicate no draining output. - pub fn implied_fee(&self, target: Target, drain_weights: DrainWeights) -> u64 { + pub fn implied_fee(&self, drain_weights: DrainWeights) -> u64 { let mut implied_fee = self - .implied_fee_from_feerate(target, drain_weights) - .max(target.fee.absolute); + .implied_fee_from_feerate(drain_weights) + .max(self.target().fee.absolute); - if let Some(replace) = target.fee.replace { + if let Some(replace) = self.target().fee.replace { implied_fee = Ord::max( implied_fee, - replace.min_fee_to_do_replacement(self.weight(target.outputs, drain_weights)), + replace.min_fee_to_do_replacement(self.weight(drain_weights)), ); } implied_fee } - fn implied_fee_from_feerate(&self, target: Target, drain_weights: DrainWeights) -> u64 { - target + fn implied_fee_from_feerate(&self, drain_weights: DrainWeights) -> u64 { + self.target() .fee .rate - .implied_fee(self.weight(target.outputs, drain_weights)) + .implied_fee(self.weight(drain_weights)) + + self.ancestor_bump() } - fn implied_fee_from_feerate_wu(&self, target: Target, drain_weights: DrainWeights) -> u64 { - target + fn implied_fee_from_feerate_wu(&self, drain_weights: DrainWeights) -> u64 { + self.target() .fee .rate - .implied_fee_wu(self.weight(target.outputs, drain_weights)) + .implied_fee_wu(self.weight(drain_weights)) + + self.ancestor_bump() + } + + /// A lower bound on the child fee this selection, or any selection extending it, must pay. + /// + /// It prices the child weight so far (at whichever of the vbyte and weight-unit roundings is + /// lower) against the rate, absolute, and replacement constraints, and adds the + /// [`ancestor_bump_lower_bound`](Self::ancestor_bump_lower_bound) to the rate constraint, since + /// every descendant owes at least that much for its ancestors. + pub(crate) fn fee_floor(&self) -> u64 { + let target = self.target(); + let weight = self.weight(DrainWeights::NONE); + let rate_floor = target + .fee + .rate + .implied_fee_wu(weight) + .min(target.fee.rate.implied_fee(weight)); + let mut floor = (rate_floor + self.ancestor_bump_lower_bound()).max(target.fee.absolute); + if let Some(replace) = target.fee.replace { + floor = floor.max( + replace + .min_fee_to_do_replacement_wu(weight) + .min(replace.min_fee_to_do_replacement(weight)), + ); + } + floor } /// The actual fee the selection would pay if it was used in a transaction that had /// `target_value` value for outputs and change output of `drain_value`. /// /// This can be negative when the selection is invalid (outputs are greater than inputs). - pub fn fee(&self, target_value: u64, drain_value: u64) -> i64 { - self.selected_value() as i64 - target_value as i64 - drain_value as i64 + pub fn fee(&self, drain_value: u64) -> i64 { + self.selected_value() as i64 - self.target().value() as i64 - drain_value as i64 } /// The value of the current selected inputs minus the fee needed to pay for the selected inputs - pub fn effective_value(&self, feerate: FeeRate) -> i64 { - self.selected_value() as i64 - (self.input_weight() as f32 * feerate.spwu()).ceil() as i64 + /// + /// Only the selected inputs' own weight is charged; any [`ancestor_bump`](Self::ancestor_bump) + /// they drag in is not. + pub fn effective_value(&self) -> i64 { + self.selected_value() as i64 + - (self.input_weight() as f32 * self.target().fee.rate.spwu()).ceil() as i64 } // /// Waste sum of all selected inputs. - fn input_waste(&self, feerate: FeeRate, long_term_feerate: FeeRate) -> f32 { - self.input_weight() as f32 * (feerate.spwu() - long_term_feerate.spwu()) + fn input_waste(&self, long_term_feerate: FeeRate) -> f32 { + self.input_weight() as f32 * (self.target().fee.rate.spwu() - long_term_feerate.spwu()) } /// Sorts the candidates by the comparision function. @@ -336,7 +834,9 @@ impl<'a> CoinSelector<'a> { where F: FnMut((usize, Candidate), (usize, Candidate)) -> core::cmp::Ordering, { - let candidates = &self.candidates; + // Positions change, so what was decided before one of them no longer means anything. + self.decided_before = 0; + let candidates = self.problem.candidates(); Arc::make_mut(&mut self.candidate_order) .sort_by(|a, b| cmp((*a, candidates[*a]), (*b, candidates[*b]))) } @@ -381,29 +881,24 @@ impl<'a> CoinSelector<'a> { /// You can pass in an `excess_discount` which must be between `0.0..1.0`. Passing in `1.0` gives you no discount /// /// [waste metric]: https://bitcoin.stackexchange.com/questions/113622/what-does-waste-metric-mean-in-the-context-of-coin-selection - pub fn waste( - &self, - target: Target, - long_term_feerate: FeeRate, - drain: Drain, - excess_discount: f32, - ) -> f32 { + pub fn waste(&self, long_term_feerate: FeeRate, drain: Drain, excess_discount: f32) -> f32 { debug_assert!((0.0..=1.0).contains(&excess_discount)); - let mut waste = self.input_waste(target.fee.rate, long_term_feerate); + let mut waste = self.input_waste(long_term_feerate); if drain.is_none() { // We don't allow negative excess waste since negative excess just means you haven't // satisified target yet in which case you probably shouldn't be calling this function. - let mut excess_waste = self.excess(target, drain).max(0) as f32; + let mut excess_waste = self.excess(drain).max(0) as f32; // we allow caller to discount this waste depending on how wasteful excess actually is // to them. excess_waste *= excess_discount.clamp(0.0, 1.0); waste += excess_waste; } else { - waste += - drain - .weights - .waste(target.fee.rate, long_term_feerate, target.outputs.n_outputs); + waste += drain.weights.waste( + self.target().fee.rate, + long_term_feerate, + self.target().outputs.n_outputs, + ); } waste @@ -415,7 +910,7 @@ impl<'a> CoinSelector<'a> { ) -> impl ExactSizeIterator + DoubleEndedIterator + '_ { self.selected .iter() - .map(move |index| (index, self.candidates[index])) + .map(move |index| (index, self.problem.candidates()[index])) } /// The unselected candidates with their index. @@ -425,7 +920,7 @@ impl<'a> CoinSelector<'a> { /// [`sort_candidates_by`]: Self::sort_candidates_by pub fn unselected(&self) -> impl DoubleEndedIterator + '_ { self.unselected_indices() - .map(move |i| (i, self.candidates[i])) + .map(move |i| (i, self.problem.candidates()[i])) } /// The weight of the lightest unselected (addable) candidate, or `None` when nothing is left to @@ -449,8 +944,12 @@ impl<'a> CoinSelector<'a> { /// This excludes candidates that have been selected or [`banned`]. /// /// [`banned`]: Self::ban + /// + /// Branch and bound tells the selector how much of the candidate order it has already decided, + /// and this starts past that prefix. Those candidates are selected or banned either way, so the + /// answer is the same. pub fn unselected_indices(&self) -> impl DoubleEndedIterator + '_ { - self.candidate_order + self.candidate_order[self.decided_before..] .iter() .copied() .filter(move |&index| !(self.selected.contains(index) || self.banned.contains(index))) @@ -467,9 +966,9 @@ impl<'a> CoinSelector<'a> { /// Always `true` when `max_weight` is `None`. Note this is the *anti-monotone* half of /// feasibility (adding inputs adds weight), so it is kept separate from the monotone /// value-only [`is_funded`](Self::is_funded). - pub fn is_within_max_weight(&self, target: Target, drain_weights: DrainWeights) -> bool { - match target.max_weight { - Some(max_weight) => self.weight(target.outputs, drain_weights) <= max_weight, + pub fn is_within_max_weight(&self, drain_weights: DrainWeights) -> bool { + match self.target().max_weight { + Some(max_weight) => self.weight(drain_weights) <= max_weight, None => true, } } @@ -479,8 +978,8 @@ impl<'a> CoinSelector<'a> { /// /// This is **monotone**: selecting more never un-meets it. It deliberately does *not* include /// the weight cap — see [`is_within_max_weight`](Self::is_within_max_weight). - pub fn is_funded_with_drain(&self, target: Target, drain: Drain) -> bool { - self.excess(target, drain) >= 0 + pub fn is_funded_with_drain(&self, drain: Drain) -> bool { + self.excess(drain) >= 0 } /// Whether the selection covers the target **value** (net of input fees), i.e. [`excess`] is @@ -492,8 +991,8 @@ impl<'a> CoinSelector<'a> { /// [`excess`]: Self::excess /// [`is_within_max_weight`]: Self::is_within_max_weight /// [`is_funded_with_drain`]: Self::is_funded_with_drain - pub fn is_funded(&self, target: Target) -> bool { - self.is_funded_with_drain(target, Drain::NONE) + pub fn is_funded(&self) -> bool { + self.is_funded_with_drain(Drain::NONE) } /// Select all unselected candidates @@ -506,27 +1005,21 @@ impl<'a> CoinSelector<'a> { } /// The value of the change output should have to drain the excess value while maintaining the - /// constraints of `target` and respecting `change_policy`. + /// constraints of the [`target`](Self::target) and respecting `change_policy`. /// /// If not change output should be added according to policy then it will return `None`. - pub fn drain_value(&self, target: Target, change_policy: ChangePolicy) -> Option { - let excess = self.excess( - target, - Drain { - weights: change_policy.drain_weights, - value: 0, - }, - ); + pub fn drain_value(&self, change_policy: ChangePolicy) -> Option { + let excess = self.excess(Drain { + weights: change_policy.drain_weights, + value: 0, + }); if excess > change_policy.min_value as i64 { debug_assert_eq!( - self.is_funded(target), - self.is_funded_with_drain( - target, - Drain { - weights: change_policy.drain_weights, - value: excess as u64 - } - ), + self.is_funded(), + self.is_funded_with_drain(Drain { + weights: change_policy.drain_weights, + value: excess as u64 + }), "if the target is met without a drain it must be met after adding the drain" ); Some(excess as u64) @@ -546,8 +1039,8 @@ impl<'a> CoinSelector<'a> { /// [`is_funded_with_drain`]: Self::is_funded_with_drain /// [`is_funded`]: Self::is_funded #[must_use] - pub fn drain(&self, target: Target, change_policy: ChangePolicy) -> Drain { - match self.drain_value(target, change_policy) { + pub fn drain(&self, change_policy: ChangePolicy) -> Drain { + match self.drain_value(change_policy) { Some(value) => Drain { weights: change_policy.drain_weights, value, @@ -556,23 +1049,24 @@ impl<'a> CoinSelector<'a> { } } - /// Select all candidates with an *effective value* greater than 0 at the provided `feerate`. + /// Select all candidates with an *effective value* greater than 0 at the target's feerate. /// - /// A candidate if effective if it provides more value than it takes to pay for at `feerate`. - pub fn select_all_effective(&mut self, feerate: FeeRate) { + /// A candidate is effective if it provides more value than it takes to pay for at that feerate. + pub fn select_all_effective(&mut self) { for i in 0..self.candidate_order.len() { let cand_index = self.candidate_order[i]; if self.selected.contains(cand_index) || self.banned.contains(cand_index) - || self.candidates[cand_index].effective_value(feerate) <= 0.0 + || self.problem.candidates()[cand_index].effective_value(self.target().fee.rate) + <= 0.0 { continue; } - self.selected.insert(cand_index); + self.select(cand_index); } } - /// Select candidates until `target` has been met. + /// Select candidates until the [`target`](Self::target) has been met. /// /// # Errors /// @@ -580,14 +1074,13 @@ impl<'a> CoinSelector<'a> { /// - [`SelectError::MaxWeightExceeded`] if the value is met but the resulting selection exceeds /// [`Target::max_weight`]. Note this only reflects *this* in-order greedy selection; a /// different selection might still fit the cap (use branch and bound to search for one). - pub fn select_until_target_met(&mut self, target: Target) -> Result<(), SelectError> { - self.select_until(|cs| cs.is_funded(target)) - .ok_or_else(|| { - SelectError::InsufficientFunds(InsufficientFunds { - missing: self.excess(target, Drain::NONE).unsigned_abs(), - }) - })?; - if !self.is_within_max_weight(target, DrainWeights::NONE) { + pub fn select_until_target_met(&mut self) -> Result<(), SelectError> { + self.select_until(|cs| cs.is_funded()).ok_or_else(|| { + SelectError::InsufficientFunds(InsufficientFunds { + missing: self.excess(Drain::NONE).unsigned_abs(), + }) + })?; + if !self.is_within_max_weight(DrainWeights::NONE) { return Err(SelectError::MaxWeightExceeded); } Ok(()) @@ -637,7 +1130,6 @@ impl<'a> CoinSelector<'a> { // the max-weight PR lands. pub fn select_srd( &mut self, - target: Target, drain_weights: DrainWeights, change_lower: u64, rng: impl FnMut() -> u64, @@ -648,14 +1140,11 @@ impl<'a> CoinSelector<'a> { let mut excess = 0_i64; self.select_until(|cs| { - is_within_max_weight = cs.is_within_max_weight(target, drain_weights); - excess = cs.excess( - target, - Drain { - weights: drain_weights, - value: 0, - }, - ); + is_within_max_weight = cs.is_within_max_weight(drain_weights); + excess = cs.excess(Drain { + weights: drain_weights, + value: 0, + }); excess >= change_lower as i64 || !is_within_max_weight }) .ok_or_else(|| { @@ -688,10 +1177,9 @@ impl<'a> CoinSelector<'a> { /// Most of the time, you would want to use [`CoinSelector::run_bnb`] instead. pub fn bnb_solutions( &self, - target: Target, metric: M, ) -> impl Iterator, Ordf32)>> { - crate::bnb::BnbIter::new(self.clone(), target, metric) + crate::bnb::BnbIter::new(self.clone(), metric) } /// Run branch and bound to minimize the score of the provided [`BnbMetric`]. @@ -703,11 +1191,10 @@ impl<'a> CoinSelector<'a> { /// Use [`CoinSelector::bnb_solutions`] to access the branch and bound iterator directly. pub fn run_bnb( &mut self, - target: Target, metric: M, max_rounds: usize, ) -> Result<(Ordf32, Drain), NoBnbSolution> { - let mut iter = crate::bnb::BnbIter::new(self.clone(), target, metric); + let mut iter = crate::bnb::BnbIter::new(self.clone(), metric); let mut rounds = 0_usize; let best = iter .by_ref() @@ -716,7 +1203,7 @@ impl<'a> CoinSelector<'a> { .flatten() .last(); if let Some((selector, score)) = best { - let drain = iter.metric.drain(&selector, target); + let drain = iter.metric.drain(&selector); *self = selector; return Ok((score, drain)); } @@ -724,12 +1211,13 @@ impl<'a> CoinSelector<'a> { // No solution. If the iterator still has an item we stopped at the round limit and a // solution may still exist with a larger `max_rounds`. Otherwise the tree was fully // explored, so no selection satisfies the target — a genuine infeasibility, split into - // value vs weight. + // value vs weight. (With unconfirmed ancestors `is_fundable` is only a heuristic, so the + // split between the two can be wrong — the infeasibility itself is not.) if iter.next().is_some() { assert_eq!(rounds, max_rounds); // still-yielding ⟹ we truncated at the cap return Err(NoBnbSolution::RoundLimit { max_rounds, rounds }); } - if !self.is_fundable(target) { + if !self.is_fundable() { return Err(NoBnbSolution::InsufficientFunds); } Err(NoBnbSolution::MaxWeightExceeded) @@ -839,6 +1327,10 @@ impl std::error::Error for SelectError {} #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum NoBnbSolution { /// The candidates can't cover the target value, so no selection is possible. + /// + /// With unconfirmed ancestors this is decided by the heuristic [`CoinSelector::is_fundable`], so + /// it may be reported where [`MaxWeightExceeded`](Self::MaxWeightExceeded) fits better, and vice + /// versa. Either way the search was exhaustive: there is no solution. InsufficientFunds, /// Some selection covers the target value, but every one of them exceeds /// [`Target::max_weight`]. @@ -889,7 +1381,11 @@ impl std::error::Error for NoBnbSolution {} /// A `Candidate` represents an input candidate for [`CoinSelector`]. /// -/// This can either be a single UTXO, or a group of UTXOs that should be spent together. +/// This can either be a single UTXO, or a group of UTXOs that should be spent together. A group +/// may mix legacy and segwit inputs; set [`legacy_count`] and [`segwit_count`] accordingly. +/// +/// [`legacy_count`]: Candidate::legacy_count +/// [`segwit_count`]: Candidate::segwit_count #[derive(Debug, Clone, Copy)] pub struct Candidate { /// Total value of the UTXO(s) that this [`Candidate`] represents. @@ -897,31 +1393,66 @@ pub struct Candidate { /// Total weight of including this/these UTXO(s). /// `txin` fields: `prevout`, `nSequence`, `scriptSigLen`, `scriptSig`, `scriptWitnessLen`, /// `scriptWitness` should all be included. + /// + /// For legacy inputs, do *not* include the `scriptWitnessLen` byte: a legacy input only + /// serializes an (empty) witness when the transaction has a witness section, and + /// [`CoinSelector::input_weight`] adds that 1 WU per legacy input once any segwit input is + /// selected. pub weight: u64, - /// Total number of inputs; so we can calculate extra `varint` weight due to `vin` len changes. - pub input_count: usize, - /// Whether this [`Candidate`] contains at least one segwit spend. - pub is_segwit: bool, + /// Total number of segwit inputs. + /// + /// If any selected candidate has a non-zero `segwit_count`, the transaction serializes a + /// witness section (marker + flag, 2 WU) and every input — including legacy ones — pays for + /// a witness. + pub segwit_count: usize, + /// Total number of legacy (non-segwit) inputs. + /// + /// Each legacy input serializes an empty witness (1 WU) when the transaction has a witness + /// section; [`CoinSelector::input_weight`] prices this per legacy input, so grouped legacy + /// inputs are counted exactly. + pub legacy_count: usize, } impl Candidate { /// Create a [`Candidate`] input that spends a single taproot keyspend output. pub fn new_tr_keyspend(value: u64) -> Self { let weight = TR_KEYSPEND_SATISFACTION_WEIGHT; - Self::new(value, weight, true) + Self::new_segwit(value, weight) } - /// Create a new [`Candidate`] that represents a single input. + /// Create a new [`Candidate`] that represents a single segwit input. /// - /// `satisfaction_weight` is the weight of `scriptSigLen + scriptSig + scriptWitnessLen + - /// scriptWitness`. - pub fn new(value: u64, satisfaction_weight: u64, is_segwit: bool) -> Candidate { + /// `satisfaction_weight` is the additional weight (in weight units) required to satisfy the input + /// beyond [`TXIN_BASE_WEIGHT`] (e.g. `scriptWitnessLen + scriptWitness` in WU at 1 WU/byte, plus + /// any `scriptSig` data and extra `scriptSigLen` varint bytes if nested/wrapped segwit). + /// + /// Note that [`TXIN_BASE_WEIGHT`] already accounts for the outpoint, `nSequence`, and 1 byte for + /// `scriptSigLen`. + pub fn new_segwit(value: u64, satisfaction_weight: u64) -> Candidate { let weight = TXIN_BASE_WEIGHT + satisfaction_weight; Candidate { value, weight, - input_count: 1, - is_segwit, + segwit_count: 1, + legacy_count: 0, + } + } + + /// Create a new [`Candidate`] that represents a single legacy (non-segwit) input. + /// + /// `satisfaction_weight` is the additional weight (in weight units) required to satisfy the input + /// beyond [`TXIN_BASE_WEIGHT`] (e.g. `scriptSig` at 4 WU/byte, plus 4 WU per extra `scriptSigLen` + /// varint byte if `scriptSig` exceeds 252 bytes). + /// + /// Note that [`TXIN_BASE_WEIGHT`] already accounts for the outpoint, `nSequence`, and 1 byte for + /// `scriptSigLen`. + pub fn new_legacy(value: u64, satisfaction_weight: u64) -> Candidate { + let weight = TXIN_BASE_WEIGHT + satisfaction_weight; + Candidate { + value, + weight, + segwit_count: 0, + legacy_count: 1, } } @@ -955,3 +1486,101 @@ impl Candidate { self.implied_fee(feerate) / self.value as f32 } } + +#[cfg(test)] +mod tests { + use super::*; + use crate::{AncestorToBump, Input, TargetFee, TargetOutputs}; + + /// The running totals must depend only on which candidates are selected and banned, never on + /// the order of the operations that got there — branch and bound selects, deselects, bans and + /// unbans in place millions of times, so any drift would silently corrupt its bounds. + #[test] + fn running_totals_match_a_selector_rebuilt_from_its_sets() { + let target = Target { + fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(3.7)), + outputs: TargetOutputs::fund_outputs([(172, 40_000)]), + max_weight: None, + }; + // Two chains reachable from several candidates (0-1 and 2-3) and one only candidate 6 can + // reach (4-5), each mixing an ancestor that pays above the target rate with one below. + let ancestors = [ + AncestorToBump { + txid: 0, + weight: 800, + fee: 100, + parents: vec![], + }, + AncestorToBump { + txid: 1, + weight: 400, + fee: 9_000, + parents: vec![0], + }, + AncestorToBump { + txid: 2, + weight: 1_200, + fee: 0, + parents: vec![], + }, + AncestorToBump { + txid: 3, + weight: 300, + fee: 4_000, + parents: vec![2], + }, + AncestorToBump { + txid: 4, + weight: 500, + fee: 50_000, + parents: vec![5], + }, + AncestorToBump { + txid: 5, + weight: 900, + fee: 0, + parents: vec![], + }, + ]; + let inputs = (0..9_u64).map(|i| Input { + value: 10_000 + i * 3_001, + weight: 272, + is_segwit: i % 3 != 0, + // 9 is not an ancestor, so a coin on it is confirmed. + residing_txid: [1, 3, 3, 2, 9, 1, 4, 3, 1][i as usize], + }); + let problem = SelectionProblem::new(target, inputs, ancestors); + assert!(problem.has_private_ancestors() && problem.has_shared_ancestors()); + + let n = problem.len(); + let mut cs = problem.selector(); + let mut rng = 0x2545_f491_4f6c_dd1d_u64; + for _ in 0..20_000 { + rng ^= rng << 13; + rng ^= rng >> 7; + rng ^= rng << 17; + let index = (rng >> 8) as usize % n; + match rng % 4 { + 0 => { + cs.select(index); + } + 1 => { + cs.deselect(index); + } + 2 => cs.ban(index), + _ => cs.unban(index), + } + + let mut rebuilt = problem.selector(); + for i in cs.selected_indices().iter() { + rebuilt.select(i); + } + for i in cs.banned().iter() { + rebuilt.ban(i); + } + assert_eq!(cs.ancestors, rebuilt.ancestors); + assert_eq!(cs.selected_value(), rebuilt.selected_value()); + assert_eq!(cs.input_weight(), rebuilt.input_weight()); + } + } +} diff --git a/src/lib.rs b/src/lib.rs index 34c86ad..77bb5dd 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -28,6 +28,8 @@ mod target; pub use target::*; mod drain; pub use drain::*; +mod selection_problem; +pub use selection_problem::*; /// Txin "base" fields include `outpoint` (32+4) and `nSequence` (4) and 1 byte for the scriptSig /// length. diff --git a/src/metrics.rs b/src/metrics.rs index 1da1163..2d841d0 100644 --- a/src/metrics.rs +++ b/src/metrics.rs @@ -5,5 +5,3 @@ //! [`CoinSelector::run_bnb`]: crate::CoinSelector::run_bnb mod lowest_fee; pub use lowest_fee::*; -mod changeless; -pub use changeless::*; diff --git a/src/metrics/changeless.rs b/src/metrics/changeless.rs deleted file mode 100644 index a9c9e32..0000000 --- a/src/metrics/changeless.rs +++ /dev/null @@ -1,79 +0,0 @@ -use crate::{bnb::BnbMetric, float::Ordf32, CoinSelector, Drain, Target}; - -/// Constrains an `inner` metric to only changeless solutions. -/// -/// A selection is scored by `inner` only if the inner metric decides it should *not* have a change -/// output (see [`BnbMetric::drain`]); otherwise it is treated as invalid. This lets you find, for -/// example, the lowest-fee changeless solution via `Changeless`. -#[derive(Clone, Copy, Debug)] -pub struct Changeless( - /// The inner metric that scores changeless solutions and owns the change decision. - pub M, -); - -impl Changeless { - /// Whether every selection reachable down this branch (the current one and any superset of it) - /// would have a change output according to the inner metric — so no changeless solution exists - /// here and the branch can be pruned. - /// - /// The inner metric only adds change once the excess is large enough (we assume its change - /// decision is monotone in the excess). So the reachable selection least likely to have change - /// is the one with the smallest excess — the current selection plus every remaining - /// negative-effective-value candidate, since each of those lowers the excess. If even that - /// selection still has change, then so does every reachable selection. - /// - /// NOTE: this relies on candidates being sorted so that all negative effective value candidates - /// are next to each other, which [`requires_ordering_by_descending_value_pwu`] guarantees. - /// - /// [`requires_ordering_by_descending_value_pwu`]: BnbMetric::requires_ordering_by_descending_value_pwu - fn change_unavoidable(&mut self, cs: &CoinSelector<'_>, target: Target) -> bool { - if self.0.drain(cs, target).is_none() { - return false; - } - - let mut least_excess = cs.clone(); - cs.unselected() - .rev() - .take_while(|(_, wv)| wv.effective_value(target.fee.rate) < 0.0) - .for_each(|(index, _)| { - least_excess.select(index); - }); - - self.0.drain(&least_excess, target).is_some() - } -} - -impl BnbMetric for Changeless { - fn drain(&mut self, _cs: &CoinSelector<'_>, _target: Target) -> Drain { - // by definition a changeless selection never has a change output - Drain::NONE - } - - fn score(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option { - // Reject selections that have change. We don't need an explicit target-met check: `inner` - // returns `None` for invalid (e.g. not-target-met) selections. - // - // NOTE: for metrics whose `score` recomputes the drain (e.g. `LowestFee`), this evaluates - // the drain decision twice per node. Sharing it would mean threading the drain into - // `score`, which we avoid to keep metrics composable. - if self.0.drain(cs, target).is_some() { - return None; - } - self.0.score(cs, target) - } - - fn bound(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option { - if self.change_unavoidable(cs, target) { - // every descendant has change, so no changeless solution is reachable - None - } else { - // the changeless-constrained optimum is no better than the inner metric's unconstrained - // optimum, so the inner bound is a valid lower bound - self.0.bound(cs, target) - } - } - - fn requires_ordering_by_descending_value_pwu(&self) -> bool { - true - } -} diff --git a/src/metrics/lowest_fee.rs b/src/metrics/lowest_fee.rs index 5499777..cf8a051 100644 --- a/src/metrics/lowest_fee.rs +++ b/src/metrics/lowest_fee.rs @@ -1,4 +1,4 @@ -use crate::{float::Ordf32, BnbMetric, CoinSelector, Drain, DrainWeights, FeeRate, Target}; +use crate::{float::Ordf32, BnbMetric, CoinSelector, Drain, DrainWeights, FeeRate}; /// Metric that aims to minimize transaction fees. The future fee for spending the change output is /// included in this calculation. @@ -13,8 +13,24 @@ use crate::{float::Ordf32, BnbMetric, CoinSelector, Drain, DrainWeights, FeeRate /// /// Unlike other metrics, `LowestFee` decides for itself whether a selection should have a change /// output: change is added whenever doing so lowers the long-term fee (i.e. the recovered excess -/// outweighs the future cost of spending the change) and the resulting change value is above the -/// dust threshold implied by `dust_relay_feerate`. +/// outweighs the future cost of spending the change), the resulting value is at least the dust +/// threshold implied by `dust_relay_feerate`, and the transaction with change fits +/// [`Target::max_weight`](crate::Target::max_weight). +/// +/// # Unconfirmed ancestors +/// +/// When the [`SelectionProblem`] has unconfirmed ancestors, the fee a selection must pay includes +/// the [`CoinSelector::ancestor_bump`] of the ancestors it drags in, so the search naturally prefers +/// coins that drag in nothing (or that share an already-paid-for ancestor). The score itself is +/// still the child transaction's fee — the bump is inside it, not added on top. +/// +/// The bound uses a child-weight relaxation when ancestors are present (see +/// [`bound`](BnbMetric::bound)): a funded node credits reachable ancestor surplus and possible future +/// change, clamped to the monotone fee floor, while an unfunded one estimates the least child weight +/// needed to meet each fee constraint. The `None` prunes stay off: funding is not monotone, so +/// "select everything and it is still unfunded" does not mean the subtree is empty. +/// +/// [`SelectionProblem`]: crate::SelectionProblem #[derive(Clone, Copy)] pub struct LowestFee { /// The estimated feerate needed to spend our change output later. @@ -27,16 +43,13 @@ pub struct LowestFee { impl LowestFee { /// The value the change output should have, or `None` if this selection should be changeless. - fn drain_value(&self, cs: &CoinSelector<'_>, target: Target) -> Option { + fn drain_value(&self, cs: &CoinSelector<'_>) -> Option { // The change output pays for its own weight, so the value we'd actually recover is the // excess remaining after accounting for that weight. - let excess_with_drain_weight = cs.excess( - target, - Drain { - weights: self.drain_weights, - value: 0, - }, - ); + let excess_with_drain_weight = cs.excess(Drain { + weights: self.drain_weights, + value: 0, + }); // Adding change is only worth it if the value we'd recover exceeds the future cost of // spending it (i.e. it lowers the long-term fee). @@ -56,7 +69,7 @@ impl LowestFee { // ...and only if the change output would not push the tx over `max_weight`. If it would, // we refuse the drain and the excess goes to fee instead (a slightly conservative choice: // it can refuse change even when a no-change tx of this selection would fit). - if !cs.is_within_max_weight(target, self.drain_weights) { + if !cs.is_within_max_weight(self.drain_weights) { return None; } @@ -71,17 +84,20 @@ impl LowestFee { /// inside [`bound`](BnbMetric::bound): deferring the changeless rejection only loosens the lower /// bound and never makes it inadmissible, and `score` reuses the returned drain for its cap /// check so the drain is decided once. - fn fee_score(&self, cs: &CoinSelector<'_>, target: Target) -> Option<(Ordf32, Drain)> { - if !cs.is_funded(target) { + /// + /// The score is the *child* transaction's fee (plus the future cost of spending its change). + /// Any [`CoinSelector::ancestor_bump`] is not added on top: it is already inside the child's fee, + /// because covering it is what [`CoinSelector::is_funded`] demands and what the change + /// calculation gives up. + fn fee_score(&self, cs: &CoinSelector<'_>) -> Option<(Ordf32, Drain)> { + if !cs.is_funded() { return None; } - let drain = self - .drain_value(cs, target) - .map_or(Drain::NONE, |value| Drain { - weights: self.drain_weights, - value, - }); - let fee_for_the_tx = cs.fee(target.value(), drain.value); + let drain = self.drain_value(cs).map_or(Drain::NONE, |value| Drain { + weights: self.drain_weights, + value, + }); + let fee_for_the_tx = cs.fee(drain.value); assert!( fee_for_the_tx >= 0, "must not be called unless selection has met target: fee={}", @@ -95,37 +111,195 @@ impl LowestFee { } } +impl LowestFee { + /// Whether a descendant of `cs` could still add both a change output and at least one more + /// input under `max_weight`. + fn change_is_reachable(&self, cs: &CoinSelector<'_>) -> bool { + match cs.target().max_weight { + None => true, + Some(max_weight) => cs.min_input_weight().map_or(false, |min_input_weight| { + cs.weight(self.drain_weights) + min_input_weight <= max_weight + }), + } + } + + /// The best exact value per weight among the undecided candidates, and whether any undecided + /// candidate carries value at zero weight. + /// + /// Relies on the descending value-per-weight order this metric requires. That order is keyed on + /// `f32`, so two exact ratios can tie there and sit either way round: the exact maximum can only + /// lie in the run sharing the first undecided candidate's `f32` key, so only that run is scanned + /// instead of every undecided candidate. Weightless candidates sort to the front (a value over + /// zero weight is infinite), so they are met before the run. + fn best_undecided_value_pwu(cs: &CoinSelector<'_>) -> (f64, bool) { + let mut best = 0.0_f64; + let mut weightless_value = false; + let mut key = None; + for (_, candidate) in cs.unselected() { + if candidate.weight == 0 { + if candidate.value > 0 { + weightless_value = true; + break; + } + continue; + } + let candidate_key = Ordf32(candidate.value_pwu()); + match key { + None => key = Some(candidate_key), + Some(first) if candidate_key != first => break, + _ => {} + } + best = best.max(candidate.value as f64 / candidate.weight as f64); + } + debug_assert!( + weightless_value + || best + == cs + .unselected() + .filter(|(_, c)| c.weight > 0) + .map(|(_, c)| c.value as f64 / c.weight as f64) + .fold(0.0_f64, f64::max), + "candidates are not in descending value-per-weight order, so the tie-run scan is wrong" + ); + (best, weightless_value) + } + + /// Tighter than [`CoinSelector::fee_floor`] once the value shortfall proves that every funded + /// descendant must add some child input weight. + /// + /// Never returns `None`: a fat private deficit can un-fund a prefix that a subset would have + /// funded, so infeasibility is not something this path is allowed to claim. (The caller has + /// already hard-pruned on child `max_weight`, which is monotone.) + /// + /// The three fee constraints get independent fractional relaxations. Their maximum is still a + /// lower bound on the real added child weight. Candidate ancestry is ignored and the global bump + /// floor is used instead, avoiding package-surplus double counting. Flooring the fractional + /// weight keeps floating-point error in the safe direction. + fn bound_with_ancestors(&self, cs: &CoinSelector<'_>) -> Ordf32 { + if cs.is_funded() { + let (_, drain) = self.fee_score(cs).unwrap(); + let current_score = + cs.fee(drain.value) as u64 + drain.weights.spend_fee(self.long_term_feerate); + let surplus = cs + .ancestor_bump() + .saturating_sub(cs.ancestor_bump_lower_bound()); + let mut bound = current_score.saturating_sub(surplus); + if drain.is_none() { + let cost_of_adding_change = self.drain_weights.waste( + cs.target().fee.rate, + self.long_term_feerate, + cs.target().outputs.n_outputs, + ); + // Subtract the large integer terms before converting anything to float. Casting the + // non-negative waste to u64 floors it, keeping the bound conservative. + let with_change = current_score + .saturating_sub(surplus) + .saturating_sub(cs.excess(Drain::NONE) as u64) + .saturating_add(cost_of_adding_change as u64); + if self.change_is_reachable(cs) { + bound = bound.min(with_change); + } + } + return Ordf32(bound.max(cs.fee_floor()) as f32); + } + + let target = cs.target(); + let bump = cs.ancestor_bump_lower_bound(); + let current_weight = cs.weight(DrainWeights::NONE); + let selected_value = cs.selected_value() as f64; + let value_target = target.value() as f64; + let target_rate = target.fee.rate.spwu() as f64; + let rate_deficit = (value_target + target_rate * current_weight as f64 + bump as f64 + - selected_value) + .max(0.0); + let absolute_deficit = + (value_target + target.fee.absolute as f64 - selected_value).max(0.0); + let (replace_deficit, replace_rate) = target.fee.replace.map_or((0.0, 0.0), |replace| { + let rate = replace.incremental_relay_feerate.spwu() as f64; + ( + (value_target + replace.fee as f64 + rate * current_weight as f64 - selected_value) + .max(0.0), + rate, + ) + }); + + let (best_value, weightless_value) = Self::best_undecided_value_pwu(cs); + let best_rate_gain = (best_value - target_rate).max(0.0); + let best_replace_gain = (best_value - replace_rate).max(0.0); + + // Treat the best candidate as unlimited fractional input. If no positive gain is available, + // or a positive-value zero-weight candidate exists, fall back to zero added weight rather + // than claiming infeasibility. + let weight_for = |deficit: f64, gain_pwu: f64| match (deficit, gain_pwu) { + (deficit, gain) if !weightless_value && deficit > 0.0 && gain > 0.0 => deficit / gain, + _ => 0.0, + }; + let added_weight = weight_for(rate_deficit, best_rate_gain) + .max(weight_for(absolute_deficit, best_value)) + .max(weight_for(replace_deficit, best_replace_gain)); + + // `added_weight` is non-negative, so conversion to u64 truncates (floors) it. + let added_weight = added_weight as u64; + let weight = match current_weight.checked_add(added_weight) { + Some(weight) if added_weight != u64::MAX => weight, + _ => return Ordf32(cs.fee_floor() as f32), + }; + let mut bound = (target.fee.rate.implied_fee_wu(weight) + bump).max(target.fee.absolute); + if let Some(replace) = target.fee.replace { + bound = bound.max(replace.min_fee_to_do_replacement_wu(weight)); + } + Ordf32(bound as f32) + } +} + impl BnbMetric for LowestFee { - fn drain(&mut self, cs: &CoinSelector<'_>, target: Target) -> Drain { - self.drain_value(cs, target) - .map_or(Drain::NONE, |value| Drain { - weights: self.drain_weights, - value, - }) + fn drain(&mut self, cs: &CoinSelector<'_>) -> Drain { + self.drain_value(cs).map_or(Drain::NONE, |value| Drain { + weights: self.drain_weights, + value, + }) } - fn score(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option { - let (score, drain) = self.fee_score(cs, target)?; + fn score(&mut self, cs: &CoinSelector<'_>) -> Option { + let (score, drain) = self.fee_score(cs)?; // A final selection must fit the weight cap. `drain_value` already refuses an over-cap // change, but a changeless selection can still be too heavy on its own. Reuse the drain // `fee_score` already decided rather than recomputing it here. - if !cs.is_within_max_weight(target, drain.weights) { + if !cs.is_within_max_weight(drain.weights) { return None; } Some(score) } - fn bound(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option { + fn bound(&mut self, cs: &CoinSelector<'_>) -> Option { // Weight hard-prune: input weight only grows as this branch is extended, so the lightest // solution in the subtree is this selection with no drain. If even that busts `max_weight`, // the whole subtree is infeasible -> prune. (Also keeps `fee_score(cs).unwrap()` below // sound: a value-met but over-cap node would otherwise score `None`.) - if !cs.is_within_max_weight(target, DrainWeights::NONE) { + // + // Ancestor weight is *not* part of this: `max_weight` caps the child transaction only. + if !cs.is_within_max_weight(DrainWeights::NONE) { return None; } - if cs.is_funded(target) { - let current_score = self.fee_score(cs, target).unwrap().0; + // Lookahead hard-prune (Bitcoin Core's `curr_available_value` test): if everything still + // undecided cannot close the feerate gap, no descendant is funded, so the subtree is empty. + // Funding needs every fee constraint met, so failing this one alone is enough to prune. + // Constant-time, and it fires before either relaxation below does any work. + if cs.best_reachable_rate_excess_wu() < 0 { + return None; + } + + // With unconfirmed ancestors, funding is not monotone, so neither this path nor the one + // below may reason from "select everything and it is still unfunded". Emptiness is claimed + // only where it is provable: by the lookahead above, which relaxes the bump to its + // branch-wide floor. + if cs.problem().has_ancestors() { + return Some(self.bound_with_ancestors(cs)); + } + + if cs.is_funded() { + let current_score = self.fee_score(cs).unwrap().0; // `current_score` is already a valid lower bound for a selection that has change: a // descendant can never lower the fee by removing an existing (worthwhile) change @@ -146,17 +320,17 @@ impl BnbMetric for LowestFee { // `drain_value`, where `change_value` is `excess_with_drain_weight` and `spend_fee` is // `drain_spend_cost`). With `v >= 0` the difference is strictly positive: B always // costs more. - if self.drain_value(cs, target).is_none() { + if self.drain_value(cs).is_none() { // But a descendant might *add* a change output that improves the metric. This // happens when the current selection is changeless only because the change would be // dust: a descendant with more excess could clear the dust threshold and recover // value that is currently burned to fees. let cost_of_adding_change = self.drain_weights.waste( - target.fee.rate, + cs.target().fee.rate, self.long_term_feerate, - target.outputs.n_outputs, + cs.target().outputs.n_outputs, ); - let cost_of_no_change = cs.excess(target, Drain::NONE); + let cost_of_no_change = cs.excess(Drain::NONE); let best_score_with_change = Ordf32(current_score.0 - cost_of_no_change as f32 + cost_of_adding_change); @@ -165,14 +339,7 @@ impl BnbMetric for LowestFee { // of which only make the tx heavier. If there's no room for both under the cap the // improvement is unreachable down this branch, so don't credit it — keep // `current_score` (a tighter, still-admissible bound). - let change_is_reachable = match target.max_weight { - None => true, - Some(max_weight) => cs.min_input_weight().map_or(false, |min_input_weight| { - cs.weight(target.outputs, self.drain_weights) + min_input_weight - <= max_weight - }), - }; - if change_is_reachable && best_score_with_change < current_score { + if self.change_is_reachable(cs) && best_score_with_change < current_score { return Some(best_score_with_change); } } @@ -180,14 +347,12 @@ impl BnbMetric for LowestFee { Some(current_score) } else { // Step 1: select everything up until the input that hits the target. - let (mut cs, resize_index, to_resize) = cs - .clone() - .select_iter() - .find(|(cs, _, _)| cs.is_funded(target))?; + let (mut cs, resize_index, to_resize) = + cs.clone().select_iter().find(|(cs, _, _)| cs.is_funded())?; // If this selection is already perfect, return its score directly. - if cs.excess(target, Drain::NONE) == 0 { - return Some(self.fee_score(&cs, target).unwrap().0); + if cs.excess(Drain::NONE) == 0 { + return Some(self.fee_score(&cs).unwrap().0); }; cs.deselect(resize_index); @@ -208,12 +373,13 @@ impl BnbMetric for LowestFee { // // In the perfect scenario, no additional fee would be required to pay for rounding up when converting from weight units to // vbytes and so all fee calculations below are performed on weight units directly. - let rate_excess = cs.rate_excess_wu(target, Drain::NONE) as f32; + let rate_excess = cs.rate_excess_wu(Drain::NONE) as f32; let mut scale = Ordf32(0.0); if rate_excess < 0.0 { let remaining_value_to_reach_feerate = rate_excess.abs(); - let effective_value_of_resized_input = to_resize.effective_value(target.fee.rate); + let effective_value_of_resized_input = + to_resize.effective_value(cs.target().fee.rate); if effective_value_of_resized_input > 0.0 { let feerate_scale = remaining_value_to_reach_feerate / effective_value_of_resized_input; @@ -225,8 +391,8 @@ impl BnbMetric for LowestFee { // We can use the same approach for replacement we just have to use the // incremental_relay_feerate. - if let Some(replace) = target.fee.replace { - let replace_excess = cs.replacement_excess_wu(target, Drain::NONE) as f32; + if let Some(replace) = cs.target().fee.replace { + let replace_excess = cs.replacement_excess_wu(Drain::NONE) as f32; if replace_excess < 0.0 { let remaining_value_to_reach_feerate = replace_excess.abs(); let effective_value_of_resized_input = @@ -243,7 +409,7 @@ impl BnbMetric for LowestFee { // Handle absolute fee constraint. Unlike feerate and replacement, the // absolute fee is a fixed amount (not weight-proportional), so we just // need enough raw value to cover the gap. - let absolute_excess = cs.absolute_excess(target, Drain::NONE) as f32; + let absolute_excess = cs.absolute_excess(Drain::NONE) as f32; if absolute_excess < 0.0 { let remaining = absolute_excess.abs(); if to_resize.value > 0 { @@ -260,9 +426,8 @@ impl BnbMetric for LowestFee { // no within-cap selection down this branch reaches the target -> prune. This is the // fractional relaxation, so it never prunes a branch with an (integer) within-cap // solution. - if let Some(max_weight) = target.max_weight { - if cs.weight(target.outputs, DrainWeights::NONE) as f32 - + scale.0 * to_resize.weight as f32 + if let Some(max_weight) = cs.target().max_weight { + if cs.weight(DrainWeights::NONE) as f32 + scale.0 * to_resize.weight as f32 > max_weight as f32 { return None; @@ -272,7 +437,7 @@ impl BnbMetric for LowestFee { // `scale` could be 0 even if `is_funded` is `false` due to the latter being based on // rounded-up vbytes. let ideal_fee = scale.0 * to_resize.value as f32 + cs.selected_value() as f32 - - target.value() as f32; + - cs.target().value() as f32; assert!(ideal_fee >= 0.0); Some(Ordf32(ideal_fee)) diff --git a/src/selection_problem.rs b/src/selection_problem.rs new file mode 100644 index 0000000..7b9eb53 --- /dev/null +++ b/src/selection_problem.rs @@ -0,0 +1,575 @@ +use alloc::collections::BTreeMap; +use alloc::vec::Vec; + +use crate::bitset::Bitset; +use crate::{Candidate, CoinSelector, FeeRate, Target}; + +/// An unconfirmed ancestor that may need bumping to the target feerate (CPFP). +/// +/// `Txid` is whatever the caller keys transactions by. This crate has no `bitcoin` dependency. +#[derive(Debug, Clone)] +pub struct AncestorToBump { + /// Caller-chosen id for this transaction. + pub txid: Txid, + /// Weight of this transaction in weight units. + pub weight: u64, + /// Fee this transaction already pays, in satoshis. + pub fee: u64, + /// Direct parents only; transitive ancestors are derived when building a [`SelectionProblem`]. + pub parents: Vec, +} + +/// One or more UTXOs that must be spent together, described on their own terms. +/// +/// Everything here is intrinsic to the coins. Hand these to [`SelectionProblem::new`], which pairs +/// each group with the ancestors it drags in. +pub type InputGroup = Vec>; + +/// A single UTXO, before it is folded into a [`Candidate`]. +#[derive(Debug, Clone, Copy)] +pub struct Input { + /// Value of the UTXO in satoshis. + pub value: u64, + /// Input weight as for [`Candidate::weight`] (legacy inputs omit the empty-witness byte). + pub weight: u64, + /// Whether this input is segwit. + pub is_segwit: bool, + /// Transaction that created this UTXO (may be unconfirmed). + pub residing_txid: Txid, +} + +impl From> for InputGroup { + fn from(input: Input) -> Self { + alloc::vec![input] + } +} + +/// Target, candidates, and (optional) ancestor-bump data for one coin-selection run. +/// +/// Build with [`SelectionProblem::new_no_ancestors`] when nothing is unconfirmed, or +/// [`SelectionProblem::new`] when spending unconfirmed UTXOs. Pass a reference to +/// [`CoinSelector::new`]. +/// +/// Ancestor bump figures are stored here (not on [`Candidate`]) so candidates stay a plain +/// description of inputs. Every unconfirmed transaction that created an input, and all of its +/// transitive unconfirmed ancestors, must be supplied for accurate CPFP pricing. Any absent id, +/// including an [`Input::residing_txid`] or parent id, is treated as confirmed and ignored, which +/// can underestimate the required fee. Deficits are computed against the full supplied ancestor +/// union; unlike Bitcoin Core, this does not remove transactions that could already be mined at an +/// intermediate feerate, so it may also conservatively overestimate a bump. +/// +/// What a selection actually owes is +/// [`CoinSelector::ancestor_bump`](crate::CoinSelector::ancestor_bump): the shortfall of the +/// ancestors its selected candidates drag in, each charged once, weight and fee netted over the +/// union. +#[derive(Debug, Clone)] +pub struct SelectionProblem { + target: Target, + candidates: Vec, + /// Weight and fee of each ancestor, after txids are dropped. + ancestors: Vec<(u64, u64)>, + /// Per-candidate set of ancestor indices dragged in by selecting that candidate. + /// + /// Empty when the problem has no ancestors (see [`has_ancestors`](Self::has_ancestors)); use + /// [`drags_in`](Self::drags_in) rather than indexing this directly. + drags_in: AncestorSets, + /// Summed weight and fee of the ancestors *only* this candidate can drag in. + /// + /// No other candidate reaches them, so they arrive exactly when this candidate is selected. + /// Summed rather than reduced to a bump because the target rate must be applied to the total + /// weight of the whole selection once, and because an ancestor paying above the rate has to be + /// able to subsidize one paying below it. + private: Vec<(u64, u64)>, + /// [`drags_in`](Self::drags_in) restricted to ancestors reachable via several candidates, which + /// are the only ones that still need de-duplicating at selection time. + shared_drags_in: AncestorSets, + /// Whether any ancestor is reachable via exactly one candidate. + has_private_ancestors: bool, + /// Whether any ancestor is reachable via more than one candidate. + has_shared_ancestors: bool, +} + +/// Per-candidate sets of ancestor indices, stored flat. +/// +/// A candidate drags in its residing transactions and their unconfirmed parents — a handful of +/// entries however large the pool is. Storing that as one dense bitset per candidate costs +/// candidates x ancestors bits, which on a 200,000-candidate pool with 26,666 ancestors is 667 MB +/// of very nearly nothing: the sets measure 0.002% full. Flat storage costs the entries themselves. +/// +/// Every read is a full walk of one candidate's set, and the sets never change after construction, +/// so a slice is all this has to be. +#[derive(Debug, Clone)] +struct AncestorSets { + /// Every candidate's ancestor indices, concatenated, each candidate's run sorted. + indices: Vec, + /// `offsets[i]..offsets[i + 1]` bounds candidate `i`. Length is candidate count plus one. + offsets: Vec, +} + +impl AncestorSets { + fn empty(candidates: usize) -> Self { + Self { + indices: Vec::new(), + offsets: alloc::vec![0; candidates + 1], + } + } + + fn with_capacity(candidates: usize) -> Self { + let mut offsets = Vec::with_capacity(candidates + 1); + offsets.push(0); + Self { + indices: Vec::new(), + offsets, + } + } + + /// Append one candidate's set. Callers push in candidate order. + fn push(&mut self, set: impl IntoIterator) { + self.indices.extend(set); + self.offsets.push(self.indices.len() as u32); + } + + fn get(&self, index: usize) -> &[u32] { + let start = self.offsets[index] as usize; + let end = self.offsets[index + 1] as usize; + &self.indices[start..end] + } +} + +/// The fee still owed so the ancestors in `set` meet `rate`, over the whole set at once. +/// +/// Weights and fees are netted across the set, so an overpaying ancestor subsidizes an underpaying +/// one and the result saturates at 0 (the child is never credited). +fn bump_of(ancestors: &[(u64, u64)], rate: FeeRate, set: &[u32]) -> u64 { + let (weight, fee) = set.iter().fold((0_u64, 0_u64), |(w, f), &anc_i| { + let (anc_w, anc_f) = ancestors[anc_i as usize]; + (w + anc_w, f + anc_f) + }); + ancestor_shortfall(rate, (weight, fee)) +} + +/// The fee still owed so ancestors with summed `(weight, fee)` meet `rate`: `ceil(weight · rate) − +/// fee`, saturating at 0. +/// +/// Computed in `f64`, not with [`FeeRate::implied_fee_wu`]'s `f32`. The rate is an `f32`, so it has +/// a 24-bit significand and converts to `f64` exactly, and `weight · rate` is then exact for any +/// weight below 2^29 WU (about 537 million) — orders of magnitude beyond any real ancestor package. +/// Being exact is what lets the ancestor lower bounds in [`CoinSelector`] be compared against this +/// without a rounding allowance. +pub(crate) fn ancestor_shortfall(rate: FeeRate, (weight, fee): (u64, u64)) -> u64 { + let owed = weight as f64 * rate.spwu() as f64 - fee as f64; + if owed <= 0.0 { + return 0; + } + // Round up without `f64::ceil`, which `no_std` lacks. Truncating a positive float rounds down, + // so add one whenever that dropped a fraction. + let truncated = owed as u64; + if (truncated as f64) < owed { + truncated.saturating_add(1) + } else { + truncated + } +} + +impl SelectionProblem { + /// A problem with no unconfirmed ancestors. + /// + /// `candidates` are taken as-is. + pub fn new_no_ancestors( + target: Target, + candidates: impl IntoIterator, + ) -> Self { + let candidates: Vec = candidates.into_iter().collect(); + let n = candidates.len(); + Self { + target, + candidates, + ancestors: Vec::new(), + drags_in: AncestorSets::empty(n), + private: alloc::vec![(0, 0); n], + shared_drags_in: AncestorSets::empty(n), + has_private_ancestors: false, + has_shared_ancestors: false, + } + } + + /// Build candidates from input groups and the unconfirmed ancestors they may drag in. + /// + /// Each input group must be non-empty, and every `AncestorToBump::txid` must be unique. Supply + /// every unconfirmed residing transaction and transitive unconfirmed ancestor needed for + /// accurate pricing; absent ids are assumed confirmed. + /// + /// For each input group, the residing txids and their transitive parents (restricted to + /// `ancestors_to_bump`) form that candidate's `drags_in` set. Ancestors only one candidate can + /// reach are folded into [`private_ancestors`](Self::private_ancestors); the rest stay in + /// [`shared_drags_in`](Self::shared_drags_in) to be de-duplicated per selection. + pub fn new(target: Target, input_groups: G, ancestors_to_bump: A) -> Self + where + Txid: Copy + Ord + Eq, + G: IntoIterator, + G::Item: Into>, + A: IntoIterator, + A::Item: Into>, + { + let ancestors: Vec> = + ancestors_to_bump.into_iter().map(Into::into).collect(); + + let txid_to_anc: BTreeMap = ancestors + .iter() + .enumerate() + .map(|(i, a)| (a.txid, i)) + .collect(); + + let n_anc = ancestors.len(); + let anc_weight_fee: Vec<(u64, u64)> = ancestors.iter().map(|a| (a.weight, a.fee)).collect(); + let mut candidates = Vec::new(); + let mut drags_in = AncestorSets::with_capacity(0); + // One scratch set, reused and emptied per candidate: allocating a dense one per candidate + // is the cost this layout exists to avoid, so it must not reappear during construction. + let mut scratch = Bitset::with_capacity(n_anc); + let mut dragged: Vec = Vec::new(); + + for input_group in input_groups { + let mut cand = Candidate { + value: 0, + weight: 0, + segwit_count: 0, + legacy_count: 0, + }; + dragged.clear(); + + for input in input_group.into() { + cand.value += input.value; + cand.weight += input.weight; + match input.is_segwit { + true => cand.segwit_count += 1, + false => cand.legacy_count += 1, + } + + let mut txid_stack = alloc::vec![input.residing_txid]; + while let Some(txid) = txid_stack.pop() { + if let Some(&anc_i) = txid_to_anc.get(&txid) { + if scratch.insert(anc_i) { + dragged.push(anc_i as u32); + txid_stack.extend(ancestors[anc_i].parents.iter().copied()); + } + } + } + } + + for &anc_i in &dragged { + scratch.remove(anc_i as usize); + } + dragged.sort_unstable(); + candidates.push(cand); + drags_in.push(dragged.iter().copied()); + } + + // An ancestor no other candidate can reach arrives exactly when this one is selected, so its + // weight and fee can be folded into the candidate now. The rest still have to be + // de-duplicated at selection time. + let mut reachable_by = alloc::vec![0_u32; n_anc]; + for &anc_i in &drags_in.indices { + reachable_by[anc_i as usize] += 1; + } + let n_cand = candidates.len(); + let mut private = Vec::with_capacity(n_cand); + let mut shared_drags_in = AncestorSets::with_capacity(n_cand); + let mut has_private_ancestors = false; + let mut has_shared_ancestors = false; + let mut shared: Vec = Vec::new(); + for index in 0..n_cand { + let mut private_weight_fee = (0_u64, 0_u64); + shared.clear(); + for &anc_i in drags_in.get(index) { + if reachable_by[anc_i as usize] == 1 { + let (weight, fee) = anc_weight_fee[anc_i as usize]; + private_weight_fee.0 += weight; + private_weight_fee.1 += fee; + has_private_ancestors = true; + } else { + shared.push(anc_i); + has_shared_ancestors = true; + } + } + private.push(private_weight_fee); + shared_drags_in.push(shared.iter().copied()); + } + + Self { + target, + candidates, + ancestors: anc_weight_fee, + drags_in, + private, + shared_drags_in, + has_private_ancestors, + has_shared_ancestors, + } + } + + /// What this problem is funding. + pub fn target(&self) -> Target { + self.target + } + + /// A copy of this problem — same candidates and ancestry — funding `target` instead. + /// + /// Pair it with [`CoinSelector::with_problem`] to re-measure an existing selection against + /// another target. + pub fn with_target(&self, target: Target) -> Self { + Self { + target, + ..self.clone() + } + } + + /// All candidates, in construction order. + pub fn candidates(&self) -> &[Candidate] { + &self.candidates + } + + /// Candidate at `index`. + pub fn candidate(&self, index: usize) -> Candidate { + self.candidates[index] + } + + /// Number of candidates. + pub fn len(&self) -> usize { + self.candidates.len() + } + + /// Whether there are no candidates. + pub fn is_empty(&self) -> bool { + self.candidates.is_empty() + } + + /// Ancestors as `(weight, fee)` pairs, in the order supplied to [`SelectionProblem::new`]. + /// + /// Supplied ancestors that no candidate reaches remain in this slice but are not charged. + pub fn ancestors(&self) -> &[(u64, u64)] { + &self.ancestors + } + + /// Whether any candidate drags in an unconfirmed ancestor. + /// + /// `false` means every fee calculation reduces to the plain (child-only) case, allowing branch + /// and bound to use its tighter no-ancestor bounds. + pub fn has_ancestors(&self) -> bool { + self.has_private_ancestors || self.has_shared_ancestors + } + + /// Ancestor indices dragged in by selecting candidate `index`. + pub fn drags_in(&self, index: usize) -> &[u32] { + self.drags_in.get(index) + } + + /// Summed `(weight, fee)` of the ancestors only candidate `index` can drag in. + /// + /// Deliberately not reduced to a bump: the target rate applies to the total ancestor weight of + /// the whole selection at once, and an ancestor paying above the rate must be able to subsidize + /// one paying below it. See [`CoinSelector::ancestor_bump`](crate::CoinSelector::ancestor_bump). + pub fn private_ancestors(&self, index: usize) -> (u64, u64) { + self.private[index] + } + + /// [`drags_in`](Self::drags_in) restricted to the ancestors that several candidates can reach. + /// + /// Those are the only ones that can be dragged in twice over, so they are the only ones a + /// selection has to de-duplicate; the rest are folded into + /// [`private_ancestors`](Self::private_ancestors). + pub fn shared_drags_in(&self, index: usize) -> &[u32] { + self.shared_drags_in.get(index) + } + + /// Whether any ancestor is reachable via exactly one candidate. + /// + /// When `false`, every ancestor is shared and [`private_ancestors`](Self::private_ancestors) is + /// `(0, 0)` throughout, so summing it can be skipped. + pub fn has_private_ancestors(&self) -> bool { + self.has_private_ancestors + } + + /// Whether any ancestor is reachable via more than one candidate. + /// + /// When `false`, what a selection owes is a plain sum over its selected candidates — nothing has + /// to be de-duplicated. + pub fn has_shared_ancestors(&self) -> bool { + self.has_shared_ancestors + } + + /// How much a group of ancestors with summed `(weight, fee)` pays above the target rate, in + /// whole satoshis rounded up, or 0 if it pays at or below it. + /// + /// Rounded up so that crediting it can only lower a lower bound. Computed the same way every + /// time, so a running total that adds and later subtracts it returns exactly to where it was. + pub(crate) fn ancestor_surplus(&self, (weight, fee): (u64, u64)) -> u64 { + let surplus = fee as f64 - weight as f64 * self.target.fee.rate.spwu() as f64; + if surplus <= 0.0 { + 0 + } else { + // Round up without `f64::ceil`, which `no_std` lacks. Truncating a positive float + // rounds down, so add one whenever that dropped a fraction. + let truncated = surplus as u64; + if (truncated as f64) < surplus { + truncated.saturating_add(1) + } else { + truncated + } + } + } + + /// The fee still owed so the ancestors only this candidate would drag in meet + /// [`Target::fee`](crate::TargetFee)'s rate, as if it were the only selected candidate. + /// + /// Informational: must never be summed over a selection (shared ancestors would be charged + /// twice). What a selection owes is [`CoinSelector::ancestor_bump`](crate::CoinSelector::ancestor_bump). + pub fn local_bump(&self, index: usize) -> u64 { + bump_of(&self.ancestors, self.target.fee.rate, self.drags_in(index)) + } + + /// A [`CoinSelector`] over this problem. + pub fn selector(&self) -> CoinSelector<'_> { + CoinSelector::new(self) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::{FeeRate, TargetFee, TargetOutputs}; + + fn target(feerate_sat_vb: f32) -> Target { + Target { + fee: TargetFee { + rate: FeeRate::from_sat_per_vb(feerate_sat_vb), + absolute: 0, + replace: None, + }, + outputs: TargetOutputs { + value_sum: 0, + weight_sum: 0, + n_outputs: 0, + }, + max_weight: None, + } + } + + #[test] + fn no_ancestors_round_trip() { + let cands = [ + Candidate::new_segwit(100_000, 100), + Candidate::new_legacy(50_000, 200), + ]; + let p = SelectionProblem::new_no_ancestors(target(10.0), cands); + assert_eq!(p.len(), 2); + assert_eq!(p.candidate(0).value, 100_000); + assert_eq!(p.candidate(1).legacy_count, 1); + assert!(p.ancestors().is_empty()); + assert_eq!(p.local_bump(0), 0); + assert_eq!(p.local_bump(1), 0); + assert!(p.drags_in(0).is_empty()); + } + + #[test] + fn transitive_parents() { + // UTXO on C; C parents B; B parents A. All unconfirmed. + let ancestors = [ + AncestorToBump { + txid: "A", + weight: 400, + fee: 0, + parents: vec![], + }, + AncestorToBump { + txid: "B", + weight: 400, + fee: 0, + parents: vec!["A"], + }, + AncestorToBump { + txid: "C", + weight: 400, + fee: 0, + parents: vec!["B"], + }, + ]; + let inputs = [Input { + value: 10_000, + weight: 272, + is_segwit: true, + residing_txid: "C", + }]; + let p = SelectionProblem::new(target(10.0), inputs, ancestors); + assert_eq!(p.len(), 1); + let dragged: Vec = p.drags_in(0).to_vec(); + assert_eq!(dragged, vec![0_u32, 1, 2]); // A, B, C + } + + #[test] + fn shared_ancestor_in_both_drags_in() { + let ancestors = [AncestorToBump { + txid: "P", + weight: 1_000, + fee: 0, + parents: vec![], + }]; + let inputs = [ + Input { + value: 10_000, + weight: 272, + is_segwit: true, + residing_txid: "P", + }, + Input { + value: 20_000, + weight: 272, + is_segwit: true, + residing_txid: "P", + }, + ]; + let p = SelectionProblem::new(target(10.0), inputs, ancestors); + assert!(p.drags_in(0).contains(&0)); + assert!(p.drags_in(1).contains(&0)); + assert_eq!(p.local_bump(0), p.local_bump(1)); + assert!(p.local_bump(0) > 0); + } + + #[test] + fn overpaying_ancestor_zero_bump() { + // weight 400 wu at 1 sat/vb => ~100 sats implied; fee already 10_000 + let ancestors = [AncestorToBump { + txid: "P", + weight: 400, + fee: 10_000, + parents: vec![], + }]; + let inputs = [Input { + value: 10_000, + weight: 272, + is_segwit: true, + residing_txid: "P", + }]; + let p = SelectionProblem::new(target(1.0), inputs, ancestors); + assert_eq!(p.local_bump(0), 0); + } + + #[test] + fn unknown_parent_ignored() { + let ancestors = [AncestorToBump { + txid: "child", + weight: 400, + fee: 0, + parents: vec!["confirmed_parent"], + }]; + let inputs = [Input { + value: 10_000, + weight: 272, + is_segwit: true, + residing_txid: "child", + }]; + let p = SelectionProblem::new(target(10.0), inputs, ancestors); + let dragged: Vec = p.drags_in(0).to_vec(); + assert_eq!(dragged, vec![0_u32]); // only child + } +} diff --git a/tests/ancestor.proptest-regressions b/tests/ancestor.proptest-regressions new file mode 100644 index 0000000..0bfbfd6 --- /dev/null +++ b/tests/ancestor.proptest-regressions @@ -0,0 +1,10 @@ +# Seeds for failure cases proptest has generated in the past. It is +# automatically read and these particular cases re-run before any +# novel cases are generated. +# +# It is recommended to check this file in to source control so that +# everyone who runs the test benefits from these saved cases. +cc 89e69058dd34f3215542f3ec44fbccc338577a6bfab488505a5e83dc5c1e88f6 # shrinks to spec = AncestorProblemSpec { candidates: [(1000, 200, 0), (1000, 200, 4)], ancestors: [(200, 0, 0)], target_value: 10000, feerate: 1.0, max_weight: None } +cc a13749585a44ffd51e890b50e8c58d6fc505895069ea32e5f211cc18fb96eacc # shrinks to spec = AncestorProblemSpec { candidates: [(24030, 664, 0), (114687, 721, 1), (58432, 646, 3), (66845, 200, 1)], ancestors: [(200, 1117, 0), (1977, 0, 1)], target_value: 172127, feerate: 2.2867246, max_weight: None } +cc 0028100a2247a35fd2ec3aa9bace2d5c5062812f0378553aac4da2e7016bf5c1 # shrinks to spec = AncestorProblemSpec { candidates: [(104036, 200, 0), (1000, 200, 1), (132930, 200, 2)], ancestors: [(600, 0, 0), (200, 233, 0)], target_value: 237606, feerate: 1.2605457, max_weight: None } +cc 134a9ae0f44a12d83ed1569a1f0f890bc71e93ad4ec699196412b559e12cb8a3 # shrinks to spec = AncestorProblemSpec { candidates: [(1000, 200, 6)], ancestors: [(200, 0, 0), (200, 0, 0), (3295, 0, 2)], target_value: 10000, feerate: 16.07041, max_weight: None } diff --git a/tests/ancestor.rs b/tests/ancestor.rs new file mode 100644 index 0000000..272b8dd --- /dev/null +++ b/tests/ancestor.rs @@ -0,0 +1,1077 @@ +#![allow(unused_imports)] +//! Coin selection over candidates that drag in unconfirmed ancestors (CPFP). +//! +//! The invariant under test is that a selection's fee obligation includes the bump owed by the +//! **union** of the ancestors its selected candidates drag in — each ancestor charged exactly once, +//! weights and fees netted over the union — and that `LowestFee` branch and bound stays correct +//! under the resulting non-monotone funding. + +mod common; + +use bdk_coin_select::{ + float::Ordf32, metrics::LowestFee, AncestorToBump, BnbMetric, Candidate, CoinSelector, Drain, + DrainWeights, FeeRate, Input, Replace, SelectionProblem, Target, TargetFee, TargetOutputs, + TX_FIXED_FIELD_WEIGHT, +}; +use proptest::prelude::*; + +/// Not a txid of any ancestor we pass in, so inputs residing on it are treated as confirmed. +const CONFIRMED: &str = "confirmed"; + +const P2WPKH_INPUT_WEIGHT: u64 = 272; + +fn target(feerate_sat_per_vb: f32, value: u64) -> Target { + Target { + fee: TargetFee { + rate: FeeRate::from_sat_per_vb(feerate_sat_per_vb), + absolute: 0, + replace: None, + }, + outputs: TargetOutputs { + value_sum: value, + weight_sum: 100, + n_outputs: 1, + }, + max_weight: None, + } +} + +fn input(value: u64, residing_txid: &'static str) -> Input<&'static str> { + Input { + value, + weight: P2WPKH_INPUT_WEIGHT, + is_segwit: true, + residing_txid, + } +} + +fn ancestor( + txid: &'static str, + weight: u64, + fee: u64, + parents: Vec<&'static str>, +) -> AncestorToBump<&'static str> { + AncestorToBump { + txid, + weight, + fee, + parents, + } +} + +fn metric() -> LowestFee { + LowestFee { + long_term_feerate: FeeRate::from_sat_per_vb(1.0), + dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), + drain_weights: DrainWeights::TR_KEYSPEND, + } +} + +/// The bump is charged on top of the child's own feerate obligation, so it eats exactly that much +/// excess relative to the same selection with nothing unconfirmed behind it. +#[test] +fn bump_is_charged_on_top_of_the_childs_own_fee() { + let t = target(10.0, 90_000); + // 1000 wu at 10 sat/vb (2.5 sat/wu) => 2500 sats owed, and the ancestor pays nothing. + let problem = + SelectionProblem::new(t, [input(100_000, "P")], [ancestor("P", 1_000, 0, vec![])]); + let mut cs = problem.selector(); + cs.select(0); + + assert_eq!(cs.ancestor_bump(), 2_500); + + let no_ancestors = SelectionProblem::new_no_ancestors( + t, + [Candidate { + value: 100_000, + weight: P2WPKH_INPUT_WEIGHT, + segwit_count: 1, + legacy_count: 0, + }], + ); + let mut clean_cs = no_ancestors.selector(); + clean_cs.select(0); + + assert_eq!(clean_cs.ancestor_bump(), 0); + assert_eq!( + cs.weight(DrainWeights::NONE), + clean_cs.weight(DrainWeights::NONE) + ); + assert_eq!( + cs.excess(Drain::NONE), + clean_cs.excess(Drain::NONE) - 2_500, + "the bump is the only difference between the two selections" + ); + assert_eq!( + cs.implied_fee(DrainWeights::NONE), + clean_cs.implied_fee(DrainWeights::NONE) + 2_500 + ); +} + +/// An unconfirmed ancestor can cost more than the coin sitting on it is worth: funding is no longer +/// monotone in the selection. +#[test] +fn dragged_in_ancestor_can_unfund_a_selection() { + let t = target(10.0, 90_000); + let problem = SelectionProblem::new( + t, + [input(100_000, CONFIRMED), input(100_000, "P")], + // 100_000 wu at 2.5 sat/wu => 250_000 sats owed: far more than the coin is worth. + [ancestor("P", 100_000, 0, vec![])], + ); + + let mut clean_only = problem.selector(); + clean_only.select(0); + assert!(clean_only.is_funded()); + + let mut both = problem.selector(); + both.select(0); + both.select(1); + assert!( + !both.is_funded(), + "adding a coin with an expensive ancestor un-funds a funded selection" + ); +} + +/// A shared ancestor is paid for once, no matter how many selected candidates drag it in — summing +/// the per-candidate `local_bump` figures would pay for it twice. +#[test] +fn shared_ancestor_is_charged_once() { + let t = target(10.0, 10_000); + let problem = SelectionProblem::new( + t, + [input(50_000, "P"), input(60_000, "P")], + [ancestor("P", 1_000, 0, vec![])], + ); + + assert_eq!(problem.local_bump(0), 2_500); + assert_eq!(problem.local_bump(1), 2_500); + + let mut cs = problem.selector(); + cs.select(0); + cs.select(1); + + assert_eq!(cs.selected_ancestors().len(), 1); + assert_eq!(cs.ancestor_bump(), 2_500); + assert_ne!( + cs.ancestor_bump(), + problem.local_bump(0) + problem.local_bump(1) + ); +} + +/// Deselecting one of two candidates that share an ancestor keeps the ancestor: it is still dragged +/// in by the other one. +#[test] +fn deselecting_keeps_an_ancestor_another_candidate_still_drags_in() { + let t = target(10.0, 10_000); + let problem = SelectionProblem::new( + t, + [ + input(50_000, "P"), + input(60_000, "P"), + input(70_000, CONFIRMED), + ], + [ancestor("P", 1_000, 0, vec![])], + ); + let mut cs = problem.selector(); + + cs.select(0); + cs.select(1); + assert_eq!(cs.ancestor_bump(), 2_500); + + cs.deselect(0); + assert_eq!(cs.ancestor_bump(), 2_500, "candidate 1 still drags in P"); + + cs.select(2); + assert_eq!( + cs.ancestor_bump(), + 2_500, + "a confirmed coin drags in nothing" + ); + + cs.deselect(1); + assert_eq!(cs.ancestor_bump(), 0, "nothing selected drags in P anymore"); +} + +/// The whole transitive chain is charged, and fees are netted across it (not per ancestor). +#[test] +fn transitive_ancestors_are_netted_as_one_package() { + let t = target(10.0, 10_000); + let problem = SelectionProblem::new( + t, + [input(50_000, "B")], + [ + ancestor("A", 400, 0, vec![]), + ancestor("B", 400, 1_000, vec!["A"]), + ], + ); + let mut cs = problem.selector(); + cs.select(0); + + // Union: weight 800 => 2000 sats owed at 2.5 sat/wu, of which B already paid 1000. + assert_eq!(cs.selected_ancestors().len(), 2); + assert_eq!(cs.ancestor_bump(), 1_000); +} + +/// Dragging in an ancestor that overpays *lowers* what the selection owes, because the deficit is +/// netted over the union. This is what makes a funded selection's fee a bad lower bound for its +/// descendants. +#[test] +fn overpaying_ancestor_offsets_an_underpaying_one() { + let t = target(1.0, 10_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [input(50_000, "RICH"), input(50_000, "POOR")], + [ + ancestor("RICH", 400, 10_000, vec![]), + ancestor("POOR", 400, 0, vec![]), + ], + ); + + let mut poor_only = problem.selector(); + poor_only.select(1); + assert_eq!(poor_only.ancestor_bump(), 100); + + let mut rich_only = problem.selector(); + rich_only.select(0); + assert_eq!(rich_only.ancestor_bump(), 0, "never credits the child"); + + let mut both = problem.selector(); + both.select(0); + both.select(1); + assert_eq!( + both.ancestor_bump(), + 0, + "RICH's surplus covers POOR's deficit, so the superset owes less" + ); +} + +/// Ancestor weight is not part of the child transaction, so it must not count against +/// [`Target::max_weight`]. +#[test] +fn ancestor_weight_does_not_count_against_max_weight() { + let mut t = target(10.0, 10_000); + let heavy = 100_000; + let problem = SelectionProblem::new( + t, + [input(50_000, "P")], + [ancestor("P", heavy, heavy, vec![])], + ); + let mut cs = problem.selector(); + cs.select(0); + + let child_weight = cs.weight(DrainWeights::NONE); + assert!(child_weight < heavy); + + t.max_weight = Some(child_weight); + let capped = SelectionProblem::new( + t, + [input(50_000, "P")], + [ancestor("P", heavy, heavy, vec![])], + ); + let mut capped_cs = capped.selector(); + capped_cs.select(0); + assert!(capped_cs.is_within_max_weight(DrainWeights::NONE)); +} + +/// Two coins of equal value and weight are *not* interchangeable when only one of them drags in an +/// ancestor, so branch and bound must not ban them as a group. +/// +/// Here the only fundable selection is the clean coin alone, and it sits *after* the coin with the +/// expensive ancestor in the search order (equal value-per-weight, so the sort is stable). If the +/// exclusion branch banned it along with its look-alike, the search would report no solution. +#[test] +fn look_alikes_with_different_ancestors_are_not_banned_together() { + let t = target(10.0, 90_000); + let problem = SelectionProblem::new( + t, + [input(100_000, "P"), input(100_000, CONFIRMED)], + [ancestor("P", 100_000, 0, vec![])], + ); + + assert_eq!(problem.candidate(0).value, problem.candidate(1).value); + assert_eq!(problem.candidate(0).weight, problem.candidate(1).weight); + + let mut cs = problem.selector(); + let (_score, _drain) = cs + .run_bnb(metric(), 100_000) + .expect("the clean coin funds the target on its own"); + + assert!(cs.is_selected(1)); + assert!(!cs.is_selected(0)); +} + +/// The bump has to be inside the fee the metric reports, not added on top of it. +#[test] +fn score_is_the_childs_fee_which_already_covers_the_bump() { + let t = target(10.0, 90_000); + let problem = + SelectionProblem::new(t, [input(100_000, "P")], [ancestor("P", 1_000, 0, vec![])]); + let mut cs = problem.selector(); + cs.select(0); + + let mut m = metric(); + let score = m.score(&cs).expect("funded"); + let drain = m.drain(&cs); + assert_eq!( + score, + Ordf32((cs.fee(drain.value) as u64 + drain.weights.spend_fee(m.long_term_feerate)) as f32) + ); + assert!( + cs.fee(drain.value) as u64 >= cs.ancestor_bump(), + "a funded selection's child fee covers the bump" + ); +} + +/// An ancestor only one candidate can reach is folded into that candidate up front; the rest are +/// left to be de-duplicated per selection. +#[test] +fn ancestors_are_split_into_private_and_shared() { + let t = target(10.0, 10_000); + let problem = SelectionProblem::new( + t, + [ + vec![input(50_000, "MINE")], + vec![input(50_000, "OURS")], + vec![input(50_000, "OURS")], + ], + [ + ancestor("MINE", 1_000, 7, vec![]), + ancestor("OURS", 2_000, 9, vec![]), + ], + ); + + assert!(problem.has_shared_ancestors()); + + // Candidate 0 is the only one that can reach MINE, so it is charged for it directly. + assert_eq!(problem.private_ancestors(0), (1_000, 7)); + assert!(problem.shared_drags_in(0).is_empty()); + + // OURS is reachable two ways, so it stays in the shared set for both. + assert_eq!(problem.private_ancestors(1), (0, 0)); + assert_eq!(problem.private_ancestors(2), (0, 0)); + assert_eq!(problem.shared_drags_in(1), &[1_u32]); + assert_eq!(problem.shared_drags_in(2), &[1_u32]); + + // Either way `drags_in` still describes the full truth. + assert_eq!(problem.drags_in(0), &[0_u32]); + assert_eq!(problem.drags_in(1), &[1_u32]); + + // And a problem where nothing is shared says so, which is what lets the bump skip + // de-duplication entirely. + let unshared = SelectionProblem::new( + t, + [input(50_000, "MINE")], + [ancestor("MINE", 1_000, 0, vec![])], + ); + assert!(!unshared.has_shared_ancestors()); + assert!(unshared.has_ancestors()); +} + +// --- the bump lower bound used by `LowestFee`'s bound --- + +/// With nothing overpaying within reach, no descendant can owe less than this selection does, so the +/// lower bound is the full bump — the figure branch and bound gets to keep. +#[test] +fn bump_lower_bound_is_the_full_bump_when_nothing_overpays() { + let t = target(10.0, 10_000); + let problem = SelectionProblem::new( + t, + [ + input(50_000, "P"), + input(60_000, "Q"), + input(70_000, CONFIRMED), + ], + [ + ancestor("P", 1_000, 0, vec![]), + ancestor("Q", 2_000, 0, vec![]), + ], + ); + + let mut cs = problem.selector(); + cs.select(0); + assert_eq!(cs.ancestor_bump(), 2_500); + assert_eq!( + cs.ancestor_bump_lower_bound(), + 2_500, + "Q only ever adds to what is owed, so it cannot lower the floor" + ); + + // The reachable-but-unselected ancestors are exactly Q's. + let addable: Vec<_> = cs.addable_ancestors().iter().collect(); + assert_eq!(addable, vec![1]); +} + +/// A reachable ancestor that overpays is exactly what a descendant could use to owe less, so the +/// bound gives up precisely that surplus and no more. +#[test] +fn bump_lower_bound_gives_up_the_reachable_surplus() { + let t = target(1.0, 10_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [input(50_000, "POOR"), input(50_000, "RICH")], + [ + ancestor("POOR", 4_000, 0, vec![]), // owes 1_000 + ancestor("RICH", 400, 10_000, vec![]), // overpays by 9_900 + ], + ); + + let mut cs = problem.selector(); + cs.select(0); + assert_eq!(cs.ancestor_bump(), 1_000); + assert_eq!( + cs.ancestor_bump_lower_bound(), + 0, + "RICH's 9_900 surplus swamps the 1_000 owed" + ); + + // Which is not pessimism: that descendant really does owe nothing. + let mut both = cs.clone(); + both.select(1); + assert_eq!(both.ancestor_bump(), 0); +} + +/// Only the surplus actually within reach is given up. +#[test] +fn bump_lower_bound_only_credits_reachable_surplus() { + let t = target(1.0, 10_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [input(50_000, "POOR"), input(50_000, "RICH")], + [ + ancestor("POOR", 4_000, 0, vec![]), // owes 1_000 + ancestor("RICH", 400, 1_100, vec![]), // overpays by 1_000 + ], + ); + + let mut cs = problem.selector(); + cs.select(0); + assert_eq!(cs.ancestor_bump(), 1_000); + assert_eq!(cs.ancestor_bump_lower_bound(), 0); + + // Ban the coin that would bring RICH in and the surplus is out of reach again. + let mut banned = cs.clone(); + banned.ban(1); + assert!(banned.addable_ancestors().is_empty()); + assert_eq!(banned.ancestor_bump_lower_bound(), 1_000); + + // Likewise once there is nothing left to add. + let mut exhausted = cs.clone(); + exhausted.select(1); + assert!(exhausted.is_exhausted()); + assert_eq!( + exhausted.ancestor_bump_lower_bound(), + exhausted.ancestor_bump() + ); +} + +/// The whole point: the fee floor `LowestFee` bounds with actually charges for the ancestors. +#[test] +fn bound_credits_the_bump_when_nothing_overpays() { + let t = target(10.0, 90_000); + let problem = SelectionProblem::new( + t, + [input(100_000, "P"), input(100_000, CONFIRMED)], + [ancestor("P", 1_000, 0, vec![])], + ); + + let mut cs = problem.selector(); + cs.select(0); + + let child_fee = t.fee.rate.implied_fee_wu(cs.weight(DrainWeights::NONE)); + let bound = metric().bound(&cs).expect("within max_weight"); + assert!( + bound >= Ordf32((child_fee + 2_500) as f32), + "bound {} must charge the child's own fee ({}) plus the 2_500 bump", + bound, + child_fee + ); +} + +#[test] +fn bump_lower_bound_accounts_for_large_f32_fee_rounding() { + let t = target(172.0, 1_000); // exactly 43 sat/wu + let problem = SelectionProblem::new( + t, + [input(20_000_000, "P")], + [ancestor("P", 399_999, 0, vec![])], + ); + let mut cs = problem.selector(); + cs.select(0); + + assert!( + cs.ancestor_bump_lower_bound() <= cs.ancestor_bump(), + "the lower bound must not exceed the bump, even where f32 fee arithmetic would round" + ); +} + +/// A funded node's bound must give up the surplus a descendant could still pick up — otherwise it +/// sits above that descendant's score. +#[test] +fn funded_bound_gives_up_reachable_surplus() { + let t = target(1.0, 10_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [input(50_000, "POOR"), input(50_000, "RICH")], + [ + ancestor("POOR", 4_000, 0, vec![]), // owes 1_000 + ancestor("RICH", 400, 10_000, vec![]), // overpays by 9_900 + ], + ); + + let mut cs = problem.selector(); + cs.select(0); + assert!(cs.is_funded()); + assert_eq!(cs.ancestor_bump(), 1_000); + assert_eq!(cs.ancestor_bump_lower_bound(), 0); + + let score = metric().score(&cs).unwrap(); + let bound = metric().bound(&cs).unwrap(); + assert!( + bound <= Ordf32(score.0 - 1_000.0), + "bound {} must sit at least the 1_000 surplus below score {}", + bound, + score + ); + + let mut both = cs.clone(); + both.select(1); + let both_score = metric().score(&both).unwrap(); + assert!( + bound <= both_score, + "bound {} above descendant score {}", + bound, + both_score + ); +} + +/// Subtracting two large `f32`s can round the bound upward. Surplus is therefore subtracted in +/// integer space before the result is converted to the metric's `f32` score. +#[test] +fn funded_bound_subtracts_surplus_before_float_conversion() { + let t = Target { + fee: TargetFee { + rate: FeeRate::from_sat_per_vb(20_000.0), // 5_000 sat/wu + absolute: 0, + replace: None, + }, + outputs: TargetOutputs { + value_sum: 0, + weight_sum: 100, + n_outputs: 1, + }, + max_weight: None, + }; + let problem = SelectionProblem::new( + t, + [ + Input { + value: 1_998_700_000, + weight: 0, + is_segwit: false, + residing_txid: "POOR", + }, + Input { + value: 0, + weight: 0, + is_segwit: false, + residing_txid: "RICH", + }, + ], + [ + ancestor("POOR", 400_000, 2_000_000, vec![]), // owes 1_998_000_000 + ancestor("RICH", 0, 1_998_000_000, vec![]), // cancels POOR exactly + ], + ); + let mut metric = LowestFee { + long_term_feerate: FeeRate::from_sat_per_vb(1.0), + dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), + drain_weights: DrainWeights::NONE, + }; + + let mut node = problem.selector(); + node.select(0); + assert_eq!(node.ancestor_bump(), 1_998_000_000); + assert_eq!(node.ancestor_bump_lower_bound(), 0); + let bound = metric.bound(&node).unwrap(); + + let mut descendant = node.clone(); + descendant.select(1); + let score = metric.score(&descendant).unwrap(); + assert_eq!(score, Ordf32(700_000.0)); + assert!(bound <= score, "bound {} above descendant {}", bound, score); +} + +/// Selecting everything can un-fund, but that must not make the bound claim the subtree is empty. +#[test] +fn unfunded_bound_does_not_claim_infeasibility() { + let t = target(10.0, 90_000); + let problem = SelectionProblem::new( + t, + [input(100_000, CONFIRMED), input(100_000, "P")], + [ancestor("P", 100_000, 0, vec![])], + ); + + let cs = problem.selector(); + assert!(!cs.is_funded()); + assert!( + metric().bound(&cs).is_some(), + "an unfunded root with a live funded subset must not be pruned" + ); +} + +/// Existing package surplus can pay a later candidate's private deficit. Pricing that deficit as +/// the candidate's marginal cost would put the bound above the descendant's score. +#[test] +fn unfunded_bound_credits_selected_package_surplus() { + let t = target(1.0, 100_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [input(60_000, "RICH"), input(50_000, "POOR")], + [ + ancestor("RICH", 400, 10_000, vec![]), // surplus 9_900 + ancestor("POOR", 4_000, 0, vec![]), // deficit 1_000 + ], + ); + + let mut node = problem.selector(); + node.select(0); + assert!(!node.is_funded()); + + let bound = metric().bound(&node).unwrap(); + let mut descendant = node.clone(); + descendant.select(1); + let score = metric().score(&descendant).unwrap(); + assert!( + bound <= score, + "bound {} above package-subsidized descendant {}", + bound, + score + ); +} + +/// The absolute fee is already the final child fee floor; the resize must not add target-rate +/// marginal cost on top of it. +#[test] +fn unfunded_bound_does_not_double_count_absolute_fee() { + let mut t = target(1.0, 100_000); + t.fee.absolute = 5_000; + let problem = + SelectionProblem::new(t, [input(105_000, "P")], [ancestor("P", 4_000, 0, vec![])]); + + let root = problem.selector(); + let bound = metric().bound(&root).unwrap(); + let mut descendant = root.clone(); + descendant.select(0); + let score = metric().score(&descendant).unwrap(); + assert_eq!(score, Ordf32(5_000.0)); + assert!(bound <= score, "bound {} above descendant {}", bound, score); +} + +/// RBF rule 4 prices only child weight. Ancestor weight must not enter its effective value, and the +/// replacement floor must not be charged twice. +#[test] +fn unfunded_bound_does_not_double_count_rbf_fee() { + let mut t = target(1.0, 100_000); + t.fee.replace = Some(Replace { + fee: 5_000, + incremental_relay_feerate: FeeRate::from_sat_per_vb(1.0), + }); + let problem = + SelectionProblem::new(t, [input(105_104, "P")], [ancestor("P", 4_000, 0, vec![])]); + + let root = problem.selector(); + let bound = metric().bound(&root).unwrap(); + let mut descendant = root.clone(); + descendant.select(0); + let score = metric().score(&descendant).unwrap(); + assert_eq!(score, Ordf32(5_104.0)); + assert!(bound <= score, "bound {} above descendant {}", bound, score); +} + +/// Surplus cannot be cherry-picked: an ancestor arrives only by selecting a candidate, which drags +/// in that candidate's whole chain. So a coin whose parent overpays but whose grandparent does not +/// offers no way to owe less, and the bound must not pretend otherwise. +#[test] +fn bump_lower_bound_nets_ancestors_that_must_arrive_together() { + let t = target(1.0, 10_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [input(50_000, "POOR"), input(50_000, "RICH")], + [ + ancestor("POOR", 4_000, 0, vec![]), // owes 1_000 + // Selecting the second coin brings RICH *and* its unpaid parent GRAN. + ancestor("GRAN", 8_000, 0, vec![]), // owes 2_000 + ancestor("RICH", 400, 10_000, vec!["GRAN"]), // overpays by 9_900 + ], + ); + + let mut cs = problem.selector(); + cs.select(0); + assert_eq!(cs.ancestor_bump(), 1_000); + + // RICH's 9_900 surplus is real, but only comes with GRAN's 2_000 deficit: still a net surplus. + assert_eq!(cs.ancestor_bump_lower_bound(), 0); + let mut both = cs.clone(); + both.select(1); + assert_eq!( + both.ancestor_bump(), + 0, + "that descendant really owes nothing" + ); + + // Now make the chain's deficit outweigh the surplus. Crediting RICH alone would wrongly drop the + // bound to 0; netting the chain keeps the full bump. + let deep = SelectionProblem::new( + t, + [input(50_000, "POOR"), input(50_000, "RICH")], + [ + ancestor("POOR", 4_000, 0, vec![]), + ancestor("GRAN", 80_000, 0, vec![]), // owes 20_000 + ancestor("RICH", 400, 10_000, vec!["GRAN"]), + ], + ); + let mut cs = deep.selector(); + cs.select(0); + assert_eq!(cs.ancestor_bump(), 1_000); + assert_eq!( + cs.ancestor_bump_lower_bound(), + 1_000, + "taking RICH means taking GRAN, which costs far more than RICH's surplus is worth" + ); + + let mut both = cs.clone(); + both.select(1); + assert!( + both.ancestor_bump() > 1_000, + "confirmed by the descendant, which owes more, not less" + ); +} + +/// An ancestor several candidates can reach cannot be tied to any one of them, so its surplus is +/// credited on its own rather than netted against a particular candidate's other ancestors. +#[test] +fn bump_lower_bound_credits_shared_surplus_on_its_own() { + let t = target(1.0, 10_000); // 0.25 sat/wu + let problem = SelectionProblem::new( + t, + [ + vec![input(50_000, "POOR")], + // Both of these reach RICH; the second also drags in its own expensive chain. + vec![input(50_000, "RICH")], + vec![input(50_000, "RICH"), input(50_000, "HEAVY")], + ], + [ + ancestor("POOR", 4_000, 0, vec![]), // owes 1_000 + ancestor("RICH", 400, 10_000, vec![]), // overpays by 9_900 + ancestor("HEAVY", 80_000, 0, vec![]), // owes 20_000 + ], + ); + + let mut cs = problem.selector(); + cs.select(0); + assert_eq!(cs.ancestor_bump(), 1_000); + assert_eq!( + cs.ancestor_bump_lower_bound(), + 0, + "RICH is reachable without HEAVY, so its surplus counts" + ); +} + +/// The lookahead prune leaves out candidates whose standalone effective value is negative, on the +/// grounds that they can only lower what is still reachable. With ancestors that is not the whole +/// story: such a candidate can *fund* a selection by dragging in an ancestor that overpays, which +/// lowers the bump the rest of the selection owes. The prune credits that through the bump's +/// branch-wide floor, so it must not cut this branch off. +/// +/// The greedy seed funds the target with the cheap clean coin, so it cannot stand in for the search +/// here: the optimum is reachable only down the branch that excludes that coin, which is exactly +/// the node whose only remaining candidate is one the lookahead does not count. +#[test] +fn lookahead_keeps_a_branch_funded_only_by_a_subsidizing_ancestor() { + let t = target(10.0, 90_000); // 2.5 sat/wu + let problem = SelectionProblem::new( + t, + [ + // Pays for itself, but not enough to cover the target and its own ancestor's bump. + input(100_000, "POOR"), + // Costs far more weight than it is worth, so the lookahead ignores its value — but it + // drags in an ancestor paying 50_000 sats over the rate, which is worth more than the + // 5_000 sats of fee its own weight costs. + Input { + value: 100, + weight: 2_000, + is_segwit: true, + residing_txid: "RICH", + }, + // Enough to fund the target alongside the first coin, but at a worse fee than paying + // the bump off with RICH's surplus. + input(10_000, CONFIRMED), + ], + [ + ancestor("POOR", 4_000, 0, vec![]), // owes 10_000 + ancestor("RICH", 400, 51_000, vec![]), // overpays by 50_000 + ], + ); + + let mut poor_only = problem.selector(); + poor_only.select(0); + assert!( + !poor_only.is_funded(), + "the bump on POOR leaves it short of the target" + ); + assert!( + problem.candidate(1).effective_value(t.fee.rate) < 0.0, + "the subsidizing coin's own value never covers its weight" + ); + + // The search sorts by descending value per weight before seeding, so seed from that order. + let mut greedy = problem.selector(); + greedy.sort_candidates_by_descending_value_pwu(); + greedy + .select_until_target_met() + .expect("the clean coin funds it"); + assert!( + greedy.is_selected(2) && !greedy.is_selected(1), + "the seed takes the clean coin, so the search has to find the rest: {}", + greedy + ); + + let mut exhaustive = problem.selector(); + let (best_score, _) = + common::exhaustive_search(&mut exhaustive, &mut metric()).expect("solvable"); + assert!( + exhaustive.is_selected(0) && exhaustive.is_selected(1) && !exhaustive.is_selected(2), + "the optimum pays the bump off with RICH's surplus: {}", + exhaustive + ); + + let mut cs = problem.selector(); + let (score, _) = cs + .run_bnb(metric(), 100_000) + .expect("the optimum must not be pruned"); + assert_eq!(score, best_score, "bnb settled for {}", cs); +} + +// --- randomized cross-checks --- + +/// Spec for a randomly generated ancestor problem. Indices are taken modulo the relevant length so +/// any combination of generated numbers describes a valid (acyclic) problem. +#[derive(Debug, Clone)] +struct AncestorProblemSpec { + /// `(value, weight, residing_txid_selector)` per candidate. + candidates: Vec<(u64, u64, usize)>, + /// `(weight, fee, parent_selector)` per unconfirmed ancestor. + ancestors: Vec<(u64, u64, usize)>, + target_value: u64, + feerate: f32, + max_weight: Option, +} + +impl AncestorProblemSpec { + fn build(&self) -> SelectionProblem { + let n_anc = self.ancestors.len(); + + let ancestors: Vec> = self + .ancestors + .iter() + .enumerate() + .map(|(i, &(weight, fee, parent_sel))| { + // Parents are strictly earlier ancestors (keeps the graph acyclic); selecting `i` + // itself means "no unconfirmed parent". + let parent = parent_sel % (i + 1); + AncestorToBump { + txid: i, + weight, + fee, + parents: if parent == i { vec![] } else { vec![parent] }, + } + }) + .collect(); + + let inputs: Vec> = self + .candidates + .iter() + .map(|&(value, weight, residing_sel)| Input { + value, + weight, + is_segwit: true, + // `n_anc` means the coin sits on a confirmed tx (no matching txid). + residing_txid: residing_sel % (n_anc + 1), + }) + .collect(); + + let mut t = target(self.feerate, self.target_value); + t.max_weight = self.max_weight; + SelectionProblem::new(t, inputs, ancestors) + } +} + +fn spec_strategy() -> impl Strategy { + ( + prop::collection::vec((1_000u64..200_000, 200u64..1_000, 0usize..8), 1..6), + prop::collection::vec((200u64..4_000, 0u64..3_000, 0usize..8), 0..4), + 10_000u64..400_000, + 1.0f32..30.0, + proptest::option::of(400u64..3_000), + ) + .prop_map( + |(candidates, ancestors, target_value, feerate, max_weight)| AncestorProblemSpec { + candidates, + ancestors, + target_value, + feerate, + max_weight, + }, + ) +} + +/// Independently computed bump for a selection, straight from the definition: union the ancestor +/// sets of the selected candidates, sum weight and fee over that union, and take the exact +/// shortfall. +fn expected_bump(problem: &SelectionProblem, cs: &CoinSelector<'_>, feerate: FeeRate) -> u64 { + let mut union = std::collections::BTreeSet::new(); + for i in cs.selected_indices().iter() { + union.extend(problem.drags_in(i).iter().map(|&a| a as usize)); + } + let (weight, fee) = union + .iter() + .map(|&i| problem.ancestors()[i]) + .fold((0u64, 0u64), |(w, f), (aw, af)| (w + aw, f + af)); + // Exact: an f32 rate converts to f64 exactly, and these weights are far too small for the + // product to round. + (weight as f64 * feerate.spwu() as f64 - fee as f64) + .ceil() + .max(0.0) as u64 +} + +proptest! { + /// Every selection's bump must equal the union-derived figure — in particular it must never be + /// the sum of the per-candidate `local_bump`s when ancestors are shared. + #[test] + fn bump_matches_union_definition(spec in spec_strategy()) { + let problem = spec.build(); + let feerate = problem.target().fee.rate; + let cs = problem.selector(); + + prop_assert_eq!(cs.ancestor_bump(), expected_bump(&problem, &cs, feerate)); + + for (node, _) in common::ExhaustiveIter::new(&cs).into_iter().flatten() { + prop_assert_eq!( + node.ancestor_bump(), + expected_bump(&problem, &node, feerate), + "selection={}", node + ); + } + } + + /// The bump lower bound must hold for the whole subtree, which is what lets the fee floor credit + /// it: no selection reachable from a node may owe less than the node's bound says. + #[test] + fn bump_lower_bound_holds_for_every_descendant(spec in spec_strategy()) { + let problem = spec.build(); + let root = problem.selector(); + + let nodes = std::iter::once(root.clone()).chain( + common::ExhaustiveIter::new(&root) + .into_iter() + .flatten() + .map(|(node, _)| node), + ); + + for node in nodes { + let lower_bound = node.ancestor_bump_lower_bound(); + prop_assert!( + lower_bound <= node.ancestor_bump(), + "node={} lb={} owes={}", node, lower_bound, node.ancestor_bump() + ); + + for (descendant, inclusion) in common::ExhaustiveIter::new(&node).into_iter().flatten() { + if !inclusion { + continue; + } + prop_assert!( + lower_bound <= descendant.ancestor_bump(), + "node={} lb={} descendant={} owes={}", + node, lower_bound, descendant, descendant.ancestor_bump() + ); + } + } + } + + /// The bound must never exceed the score of any selection in its subtree (else branch and bound + /// can prune the optimum), and `None` must really mean "nothing in this subtree is valid". + #[test] + fn bound_is_admissible_with_ancestors(spec in spec_strategy()) { + let problem = spec.build(); + let mut metric = metric(); + + let mut root = problem.selector(); + if metric.requires_ordering_by_descending_value_pwu() { + root.sort_candidates_by_descending_value_pwu(); + } + + let nodes = std::iter::once(root.clone()).chain( + common::ExhaustiveIter::new(&root) + .into_iter() + .flatten() + .map(|(node, _)| node), + ); + + for node in nodes { + let bound = metric.bound(&node); + let subtree = std::iter::once(node.clone()).chain( + common::ExhaustiveIter::new(&node) + .into_iter() + .flatten() + .filter(|(_, inclusion)| *inclusion) + .map(|(descendant, _)| descendant), + ); + + for descendant in subtree { + let score = metric.score(&descendant); + match bound { + Some(lb) => if let Some(score) = score { + prop_assert!( + score >= lb, + "bound too tight: node={} lb={} descendant={} score={}", + node, lb, descendant, score + ); + }, + None => prop_assert!( + score.is_none(), + "pruned a subtree with a solution: node={} descendant={} score={:?}", + node, descendant, score + ), + } + } + } + } + + /// With unlimited rounds, branch and bound must land on the same optimum as brute force — both + /// the score and the feasibility verdict. + #[test] + fn bnb_finds_the_brute_force_optimum(spec in spec_strategy()) { + let problem = spec.build(); + + let mut exhaustive_cs = problem.selector(); + let mut exhaustive_metric = metric(); + let expected = common::exhaustive_search(&mut exhaustive_cs, &mut exhaustive_metric); + + let mut bnb_cs = problem.selector(); + let found = common::bnb_search(&mut bnb_cs, metric(), usize::MAX); + + match (expected, found) { + (Some((expected_score, _)), Ok((score, _))) => { + prop_assert_eq!(score, expected_score, "bnb={} exhaustive={}", bnb_cs, exhaustive_cs); + } + (None, Err(_)) => {} + (expected, found) => prop_assert!( + false, + "disagreement: exhaustive={:?} bnb={:?}", + expected.map(|(score, _)| score), + found.map(|(score, _)| score), + ), + } + } +} diff --git a/tests/bnb.rs b/tests/bnb.rs index 45a22dc..573a0ca 100644 --- a/tests/bnb.rs +++ b/tests/bnb.rs @@ -1,6 +1,7 @@ mod common; use bdk_coin_select::{ - float::Ordf32, BnbMetric, Candidate, CoinSelector, Drain, Target, TargetFee, TargetOutputs, + float::Ordf32, BnbMetric, Candidate, CoinSelector, Drain, SelectionProblem, Target, TargetFee, + TargetOutputs, }; #[macro_use] extern crate alloc; @@ -11,16 +12,16 @@ use proptest::{prelude::*, proptest, test_runner::*}; fn test_wv(mut rng: impl RngCore) -> impl Iterator { core::iter::repeat_with(move || { let value = rng.random_range(0..1_000); - let mut candidate = Candidate { + let candidate = Candidate { value, weight: 100, - input_count: rng.random_range(1..2), - is_segwit: rng.random_bool(0.5), + segwit_count: rng.random_range(1..2), + legacy_count: 0, }; - // HACK: set is_segwit = true for all these tests because you can't actually lower bound - // things easily with how segwit inputs interfere with their weights. We can't modify the - // above since that would change what we pull from rng. - candidate.is_segwit = true; + // Keep drawing the bool these tests always drew so the rng stream (and therefore the + // generated cases) is unchanged. All candidates are segwit: mixing in legacy inputs makes + // their weights context-dependent, which these tests can't lower-bound easily. + let _ = rng.random_bool(0.5); candidate }) } @@ -32,8 +33,8 @@ struct MinExcessThenWeight; const EXCESS_RATIO: f32 = 1_000_000_f32; impl BnbMetric for MinExcessThenWeight { - fn score(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option { - let excess = cs.excess(target, Drain::NONE); + fn score(&mut self, cs: &CoinSelector<'_>) -> Option { + let excess = cs.excess(Drain::NONE); if excess < 0 { None } else { @@ -43,13 +44,13 @@ impl BnbMetric for MinExcessThenWeight { } } - fn bound(&mut self, cs: &CoinSelector<'_>, target: Target) -> Option { + fn bound(&mut self, cs: &CoinSelector<'_>) -> Option { let mut cs = cs.clone(); - cs.select_until_target_met(target).ok()?; + cs.select_until_target_met().ok()?; Some(Ordf32(cs.input_weight() as f32)) } - fn drain(&mut self, _cs: &CoinSelector<'_>, _target: Target) -> Drain { + fn drain(&mut self, _cs: &CoinSelector<'_>) -> Drain { Drain::NONE } } @@ -62,26 +63,15 @@ fn bnb_finds_an_exact_solution_in_n_iter() { let num_additional_canidates = 12; let mut rng = TestRng::deterministic_rng(RngAlgorithm::ChaCha); - let mut wv = test_wv(&mut rng).map(|mut candidate| { - candidate.is_segwit = true; - candidate - }); + let mut wv = test_wv(&mut rng); let solution: Vec = (0..solution_len).map(|_| wv.next().unwrap()).collect(); - let solution_weight = { - let mut cs = CoinSelector::new(&solution); - cs.select_all(); - cs.input_weight() - }; - let target_value = solution.iter().map(|c| c.value).sum(); - let mut candidates = solution; + let mut candidates = solution.clone(); candidates.extend(wv.take(num_additional_canidates)); candidates.sort_unstable_by_key(|wv| core::cmp::Reverse(wv.value)); - let cs = CoinSelector::new(&candidates); - let target = Target { outputs: TargetOutputs { value_sum: target_value, @@ -93,7 +83,18 @@ fn bnb_finds_an_exact_solution_in_n_iter() { max_weight: None, }; - let solutions = cs.bnb_solutions(target, MinExcessThenWeight); + let problem = SelectionProblem::new_no_ancestors(target, solution.iter().copied()); + + let solution_weight = { + let mut cs = CoinSelector::new(&problem); + cs.select_all(); + cs.input_weight() + }; + + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let cs = CoinSelector::new(&problem); + let solutions = cs.bnb_solutions(MinExcessThenWeight); let mut rounds = 0; let (best, score) = solutions @@ -103,7 +104,7 @@ fn bnb_finds_an_exact_solution_in_n_iter() { .last() .expect("it found a solution"); - assert_eq!(rounds, 3194); + assert_eq!(rounds, 62453); assert_eq!(best.input_weight(), solution_weight); assert_eq!(best.selected_value(), target_value, "score={:?}", score); } @@ -116,8 +117,6 @@ fn bnb_finds_solution_if_possible_in_n_iter() { let wv = test_wv(&mut rng); let candidates = wv.take(num_inputs).collect::>(); - let cs = CoinSelector::new(&candidates); - let target = Target { outputs: TargetOutputs { value_sum: target_value, @@ -128,7 +127,10 @@ fn bnb_finds_solution_if_possible_in_n_iter() { max_weight: None, }; - let solutions = cs.bnb_solutions(target, MinExcessThenWeight); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let cs = CoinSelector::new(&problem); + let solutions = cs.bnb_solutions(MinExcessThenWeight); let mut rounds = 0; let (sol, _score) = solutions @@ -138,8 +140,8 @@ fn bnb_finds_solution_if_possible_in_n_iter() { .last() .expect("found a solution"); - assert_eq!(rounds, 164); - let excess = sol.excess(target, Drain::NONE); + assert_eq!(rounds, 95); + let excess = sol.excess(Drain::NONE); assert_eq!(excess, 0); } @@ -150,19 +152,19 @@ proptest! { let mut rng = TestRng::deterministic_rng(RngAlgorithm::ChaCha); let wv = test_wv(&mut rng); let candidates = wv.take(num_inputs).collect::>(); - let cs = CoinSelector::new(&candidates); let target = Target { outputs: TargetOutputs { value_sum: target_value, weight_sum: 0, n_outputs: 1 }, fee: TargetFee::ZERO, max_weight: None, }; - - let solutions = cs.bnb_solutions(target, MinExcessThenWeight); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let cs = CoinSelector::new(&problem); + let solutions = cs.bnb_solutions(MinExcessThenWeight); match solutions.enumerate().filter_map(|(i, sol)| Some((i, sol?))).last() { Some((_i, (sol, _score))) => assert!(sol.selected_value() >= target_value), - _ => prop_assert!(!cs.is_fundable(target)), + _ => prop_assert!(!cs.is_fundable()), } } @@ -177,20 +179,29 @@ proptest! { let mut wv = test_wv(&mut rng); let solution: Vec = (0..solution_len).map(|_| wv.next().unwrap()).collect(); - let solution_weight = { - let mut cs = CoinSelector::new(&solution); - cs.select_all(); - cs.input_weight() - }; - let target_value = solution.iter().map(|c| c.value).sum(); - let mut candidates = solution; + let mut candidates = solution.clone(); candidates.extend(wv.take(num_additional_canidates)); - let mut cs = CoinSelector::new(&candidates); + let target = Target { + outputs: TargetOutputs { value_sum: target_value, weight_sum: 0, n_outputs: 1 }, + // we're trying to find an exact selection value so set fees to 0 + fee: TargetFee::ZERO, + max_weight: None, + }; + + let problem = SelectionProblem::new_no_ancestors(target, solution.iter().copied()); + + let solution_weight = { + let mut cs = CoinSelector::new(&problem); + cs.select_all(); + cs.input_weight() + }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); for i in 0..num_preselected.min(solution_len) { cs.select(i); } @@ -198,14 +209,7 @@ proptest! { // sort in descending value cs.sort_candidates_by_key(|(_, wv)| core::cmp::Reverse(wv.value)); - let target = Target { - outputs: TargetOutputs { value_sum: target_value, weight_sum: 0, n_outputs: 1 }, - // we're trying to find an exact selection value so set fees to 0 - fee: TargetFee::ZERO, - max_weight: None, - }; - - let solutions = cs.bnb_solutions(target, MinExcessThenWeight); + let solutions = cs.bnb_solutions(MinExcessThenWeight); let (_i, (best, _score)) = solutions .enumerate() diff --git a/tests/changeless.proptest-regressions b/tests/changeless.proptest-regressions deleted file mode 100644 index 97089e2..0000000 --- a/tests/changeless.proptest-regressions +++ /dev/null @@ -1,9 +0,0 @@ -# Seeds for failure cases proptest has generated in the past. It is -# automatically read and these particular cases re-run before any -# novel cases are generated. -# -# It is recommended to check this file in to source control so that -# everyone who runs the test benefits from these saved cases. -cc b03fc0267d15cf4455c7f00feed18d1ba82a783a38bf689dacdd572356013877 # shrinks to num_inputs = 7, target = 1277, feerate = 1.0, min_fee = 177, base_weight = 0, long_term_feerate_diff = 0.0, change_weight = 1, change_spend_weight = 1 -cc 2ba2cfa2412c0f3c9de4eb35caeefa1a086797f5c3f5fc0528396cc90a85d993 # shrinks to num_inputs = 5, target = 908, feerate = 7.823237, replace = None, base_weight = 222, long_term_feerate_diff = 2.4063222, change_weight = 14, change_spend_weight = 14 -cc fc2f2211d811690b78ca4206be874e5c3e99727626f79306bbdd25d59df9c27b # shrinks to num_inputs = 10, target = 3821, feerate = 4.104783, replace = None, base_weight = 321, long_term_feerate_diff = 2.7914581, change_weight = 1, change_spend_weight = 1 diff --git a/tests/changeless.rs b/tests/changeless.rs deleted file mode 100644 index aac10a3..0000000 --- a/tests/changeless.rs +++ /dev/null @@ -1,104 +0,0 @@ -#![allow(unused)] -mod common; -use bdk_coin_select::{ - float::Ordf32, - metrics::{Changeless, LowestFee}, - Candidate, CoinSelector, DrainWeights, FeeRate, Target, TargetFee, TargetOutputs, -}; -use proptest::{prelude::*, proptest, test_runner::*}; -use rand::{prelude::IteratorRandom, Rng, RngCore}; - -fn test_wv(mut rng: impl RngCore) -> impl Iterator { - core::iter::repeat_with(move || { - let value = rng.random_range(0..1_000); - Candidate { - value, - weight: rng.random_range(0..100), - input_count: rng.random_range(1..2), - is_segwit: false, - } - }) -} - -proptest! { - #![proptest_config(ProptestConfig { - ..Default::default() - })] - - #[test] - #[cfg(not(debug_assertions))] // too slow if compiling for debug - fn compare_against_benchmarks( - n_candidates in 0..15_usize, // candidates (n) - target_value in 500..1_000_000_u64, // target value (sats) - n_target_outputs in 1..150_usize, // the number of outputs we're funding - target_weight in 0..10_000_u32, // the sum of the weight of the outputs (wu) - replace in common::maybe_replace(0..10_000u64), // The weight of the transaction we're replacing - feerate in 1.0..100.0_f32, // feerate (sats/vb) - feerate_lt_diff in -5.0..50.0_f32, // longterm feerate diff (sats/vb) - drain_weight in 100..=500_u32, // drain weight (wu) - drain_spend_weight in 1..=2000_u32, // drain spend weight (wu) - drain_dust in 100..=1000_u64, // drain dust (sats) - n_drain_outputs in 1..150usize, // the number of drain outputs - ) { - println!("======================================="); - let start = std::time::Instant::now(); - let mut rng = TestRng::deterministic_rng(RngAlgorithm::ChaCha); - let feerate = FeeRate::from_sat_per_vb(feerate); - let drain_weights = DrainWeights { - output_weight: drain_weight as u64, - spend_weight: drain_spend_weight as u64, - n_outputs: n_drain_outputs, - }; - - let wv = test_wv(&mut rng); - let candidates = wv.take(n_candidates).collect::>(); - - let cs = CoinSelector::new(&candidates); - - let target = Target { - outputs: TargetOutputs { - n_outputs: n_target_outputs, - value_sum: target_value, - weight_sum: target_weight as u64, - }, - fee: TargetFee { - rate: feerate, - replace, - ..TargetFee::ZERO - }, - max_weight: None, - }; - - let make_metric = || { - Changeless(LowestFee { - long_term_feerate: feerate, - dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), - drain_weights, - }) - }; - - let solutions = cs.bnb_solutions(target, make_metric()); - - println!("candidates: {:#?}", cs.candidates().collect::>()); - - let best = solutions - .enumerate() - .filter_map(|(i, sol)| Some((i, sol?))) - .last(); - - - match best { - Some((_i, (_sol, _score))) => { - /* there is nothing to check about a changeless solution */ - } - None => { - let mut cs = cs.clone(); - let mut metric = make_metric(); - let has_solution = common::exhaustive_search(&mut cs, target, &mut metric).is_some(); - dbg!(format!("{}", cs)); - assert!(!has_solution); - } - } - dbg!(start.elapsed()); - } -} diff --git a/tests/common.rs b/tests/common.rs index 273b830..7b45f88 100644 --- a/tests/common.rs +++ b/tests/common.rs @@ -2,7 +2,7 @@ use bdk_coin_select::{ float::Ordf32, metrics::LowestFee, BnbMetric, Candidate, CoinSelector, Drain, DrainWeights, - FeeRate, NoBnbSolution, Replace, Target, TargetFee, TargetOutputs, + FeeRate, NoBnbSolution, Replace, SelectionProblem, Target, TargetFee, TargetOutputs, }; use proptest::{ prelude::*, @@ -51,7 +51,9 @@ where let target = params.target(); - let mut selection = CoinSelector::new(&candidates); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let mut selection = CoinSelector::new(&problem); let mut exp_selection = selection.clone(); if metric.requires_ordering_by_descending_value_pwu() { @@ -61,8 +63,8 @@ where println!("\texhaustive search:"); let now = std::time::Instant::now(); - let exp_result = exhaustive_search(&mut exp_selection, target, &mut metric); - let exp_change = metric.drain(&exp_selection, target); + let exp_result = exhaustive_search(&mut exp_selection, &mut metric); + let exp_change = metric.drain(&exp_selection); let exp_result_str = result_string(&exp_result.ok_or("no possible solution"), exp_change); println!( "\t\telapsed={:8}s result={}", @@ -72,14 +74,11 @@ where // bonus check: ensure replacement fee is respected if exp_result.is_some() { let selected_value = exp_selection.selected_value(); - let drain = metric.drain(&exp_selection, target); + let drain = metric.drain(&exp_selection); let target_value = target.value(); let replace_fee = params .replace - .map(|replace| { - replace - .min_fee_to_do_replacement(exp_selection.weight(target.outputs, drain.weights)) - }) + .map(|replace| replace.min_fee_to_do_replacement(exp_selection.weight(drain.weights))) .unwrap_or(0); assert!(selected_value - target_value - drain.value >= replace_fee); } @@ -87,8 +86,8 @@ where println!("\tbranch and bound:"); let now = std::time::Instant::now(); let mut bnb_metric = metric.clone(); - let result = bnb_search(&mut selection, target, metric, usize::MAX); - let change = bnb_metric.drain(&selection, target); + let result = bnb_search(&mut selection, metric, usize::MAX); + let change = bnb_metric.drain(&selection); let result_str = result_string(&result, change); println!( "\t\telapsed={:8}s result={}", @@ -112,14 +111,11 @@ where // bonus check: ensure replacement fee is respected let selected_value = selection.selected_value(); - let drain = bnb_metric.drain(&selection, target); + let drain = bnb_metric.drain(&selection); let target_value = target.value(); let replace_fee = params .replace - .map(|replace| { - replace - .min_fee_to_do_replacement(selection.weight(target.outputs, drain.weights)) - }) + .map(|replace| replace.min_fee_to_do_replacement(selection.weight(drain.weights))) .unwrap_or(0); assert!(selected_value - target_value - drain.value >= replace_fee); } @@ -147,8 +143,10 @@ where let target = params.target(); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let init_cs = { - let mut cs = CoinSelector::new(&candidates); + let mut cs = CoinSelector::new(&problem); if metric.requires_ordering_by_descending_value_pwu() { cs.sort_candidates_by_descending_value_pwu(); } @@ -157,12 +155,12 @@ where print_candidates(¶ms, &init_cs); for (cs, _) in ExhaustiveIter::new(&init_cs).into_iter().flatten() { - if let Some(lb_score) = metric.bound(&cs, target) { + if let Some(lb_score) = metric.bound(&cs) { // This is the branch's lower bound. In other words, this is the BEST selection // possible (can overshoot) traversing down this branch. Let's check that! - if let Some(score) = metric.score(&cs, target) { - let has_change = metric.drain(&cs, target).is_some(); + if let Some(score) = metric.score(&cs) { + let has_change = metric.drain(&cs).is_some(); prop_assert!( score >= lb_score, "checking branch: selection={} score={} change={} lb={}", @@ -178,9 +176,9 @@ where .flatten() .filter(|(_, inc)| *inc) { - if let Some(descendant_score) = metric.score(&descendant_cs, target) { - let parent_has_change = metric.drain(&cs, target).is_some(); - let descendant_has_change = metric.drain(&descendant_cs, target).is_some(); + if let Some(descendant_score) = metric.score(&descendant_cs) { + let parent_has_change = metric.drain(&cs).is_some(); + let descendant_has_change = metric.drain(&descendant_cs).is_some(); prop_assert!( descendant_score >= lb_score, " @@ -190,7 +188,7 @@ where cs, parent_has_change, lb_score, - cs.is_funded(target), + cs.is_funded(), descendant_cs, descendant_has_change, descendant_score, @@ -269,14 +267,21 @@ pub fn gen_candidates(n: usize) -> Vec { core::iter::repeat_with(move || { let value = rng.random_range(1..500_001); let weight = rng.random_range(1..2001); - let input_count = rng.random_range(1..3); - let is_segwit = rng.random_bool(0.01); + + let (mut legacy_count, mut segwit_count); + loop { + legacy_count = rng.random_range(0..3); + segwit_count = if rng.random_bool(0.01) { 1 } else { 0 }; + if legacy_count > 0 || segwit_count > 0 { + break; + } + } Candidate { value, weight, - input_count, - is_segwit, + segwit_count, + legacy_count, } }) .take(n) @@ -340,11 +345,7 @@ impl<'a> Iterator for ExhaustiveIter<'a> { } } -pub fn exhaustive_search( - cs: &mut CoinSelector, - target: Target, - metric: &mut M, -) -> Option<(Ordf32, usize)> +pub fn exhaustive_search(cs: &mut CoinSelector, metric: &mut M) -> Option<(Ordf32, usize)> where M: BnbMetric, { @@ -359,7 +360,7 @@ where .enumerate() .inspect(|(i, _)| rounds = *i) .filter(|(_, (_, inclusion))| *inclusion) - .filter_map(|(_, (cs, _))| metric.score(&cs, target).map(|score| (cs, score))); + .filter_map(|(_, (cs, _))| metric.score(&cs).map(|score| (cs, score))); for (child_cs, score) in iter { match &mut best { @@ -388,10 +389,8 @@ where /// [`CoinSelector::is_funded`] + [`CoinSelector::is_within_max_weight`], so it inherits the /// exact weight model and is independent of the BnB weight prune it audits. Exponential — small `n` /// only. -pub fn exact_selection_possible(cs: &CoinSelector, target: Target) -> bool { - let feasible = |s: &CoinSelector| { - s.is_funded(target) && s.is_within_max_weight(target, DrainWeights::NONE) - }; +pub fn exact_selection_possible(cs: &CoinSelector) -> bool { + let feasible = |s: &CoinSelector| s.is_funded() && s.is_within_max_weight(DrainWeights::NONE); // the current selection itself (no additions) is a valid subset and isn't yielded by the iter feasible(cs) || ExhaustiveIter::new(cs) @@ -401,7 +400,6 @@ pub fn exact_selection_possible(cs: &CoinSelector, target: Target) -> bool { pub fn bnb_search( cs: &mut CoinSelector, - target: Target, metric: M, max_rounds: usize, ) -> Result<(Ordf32, usize), NoBnbSolution> @@ -410,7 +408,7 @@ where { let mut rounds = 0_usize; let (selection, score) = cs - .bnb_solutions(target, metric) + .bnb_solutions(metric) .inspect(|_| rounds += 1) .take(max_rounds) .flatten() @@ -448,8 +446,9 @@ pub fn compare_against_benchmarks( let start = std::time::Instant::now(); let mut rng = TestRng::deterministic_rng(RngAlgorithm::ChaCha); let target = params.target(); - let cs = CoinSelector::new(&candidates); - let solutions = cs.bnb_solutions(target, metric.clone()); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let cs = CoinSelector::new(&problem); + let solutions = cs.bnb_solutions(metric.clone()); let best = solutions .enumerate() @@ -465,7 +464,7 @@ pub fn compare_against_benchmarks( core::cmp::Reverse(Ordf32(wv.effective_value(target.fee.rate))) }); // we filter out failing onces below - let _ = naive_select.select_until_target_met(target); + let _ = naive_select.select_until_target_met(); naive_select }, { @@ -475,7 +474,7 @@ pub fn compare_against_benchmarks( }, { let mut all_effective_selected = cs.clone(); - all_effective_selected.select_all_effective(target.fee.rate); + all_effective_selected.select_all_effective(); all_effective_selected }, { @@ -485,7 +484,7 @@ pub fn compare_against_benchmarks( // exists, so the comparison below isn't vacuous. let mut greedy = cs.clone(); greedy.sort_candidates_by_descending_value_pwu(); - let _ = greedy.select_until_target_met(target); + let _ = greedy.select_until_target_met(); greedy }, ]; @@ -501,11 +500,11 @@ pub fn compare_against_benchmarks( let cmp_benchmarks = cmp_benchmarks .into_iter() .filter_map(|cs| { - let score = metric.clone().score(&cs, target)?; + let score = metric.clone().score(&cs)?; Some((cs, score)) }) .collect::>(); - let sol_score = metric.score(&sol, target); + let sol_score = metric.score(&sol); for (_bench_id, (mut bench, bench_score)) in cmp_benchmarks.into_iter().enumerate() { prop_assert!( @@ -526,7 +525,7 @@ pub fn compare_against_benchmarks( None => { // Full feasibility (value *and* max_weight) is needed here; `is_fundable` // only covers value, so use the exact exhaustive oracle to assert impossibility. - prop_assert!(!exact_selection_possible(&cs, target)); + prop_assert!(!exact_selection_possible(&cs)); } } @@ -546,8 +545,8 @@ fn randomly_satisfy_target<'a, R: rand::Rng>( let mut last_score: Option = None; while let Some(next) = cs.unselected_indices().choose(rng) { cs.select(next); - if cs.is_funded(target) { - let curr_score = metric.score(&cs, target); + if cs.is_funded() { + let curr_score = metric.score(&cs); if let Some(last_score) = last_score { if curr_score.is_none() || curr_score.unwrap() > last_score { break; diff --git a/tests/lowest_fee.rs b/tests/lowest_fee.rs index d6b8cba..91d3a83 100644 --- a/tests/lowest_fee.rs +++ b/tests/lowest_fee.rs @@ -1,10 +1,9 @@ #![allow(unused_imports)] - mod common; -use bdk_coin_select::metrics::{Changeless, LowestFee}; +use bdk_coin_select::metrics::LowestFee; use bdk_coin_select::{ BnbMetric, Candidate, ChangePolicy, CoinSelector, Drain, DrainWeights, FeeRate, NoBnbSolution, - Replace, Target, TargetFee, TargetOutputs, TX_FIXED_FIELD_WEIGHT, + Replace, SelectionProblem, Target, TargetFee, TargetOutputs, TX_FIXED_FIELD_WEIGHT, }; use proptest::prelude::*; @@ -85,17 +84,19 @@ proptest! { Candidate { value: 20_000, weight: (32 + 4 + 4 + 1) * 4 + 64 + 32, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }; params.n_candidates ]; - let mut cs = CoinSelector::new(&candidates); + let problem = SelectionProblem::new_no_ancestors(params.target(), candidates.iter().copied()); + + let mut cs = CoinSelector::new(&problem); let metric = params.lowest_fee_metric(); - let is_impossible = !cs.is_fundable(params.target()); - match common::bnb_search(&mut cs, params.target(), metric, params.n_candidates * 10) { + let is_impossible = !cs.is_fundable(); + match common::bnb_search(&mut cs, metric, params.n_candidates * 10) { Ok((score, rounds)) => { // the +1 is because the iterator will always try selecting nothing as a solution so we have // to do one extra iteration to try that @@ -162,10 +163,11 @@ proptest! { let target = params.target(); let metric = params.lowest_fee_metric(); - let exact_possible = common::exact_selection_possible(&CoinSelector::new(&candidates), target); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let exact_possible = common::exact_selection_possible(&CoinSelector::new(&problem)); - let mut cs = CoinSelector::new(&candidates); - let bnb_found = common::bnb_search(&mut cs, target, metric, usize::MAX).is_ok(); + let mut cs = CoinSelector::new(&problem); + let bnb_found = common::bnb_search(&mut cs, metric, usize::MAX).is_ok(); prop_assert_eq!( bnb_found, exact_possible, "bnb_found={} but exact_possible={} (weight prune may have dropped a feasible subtree)", @@ -174,49 +176,6 @@ proptest! { } } -/// We wrap `LowestFee` in `Changeless` to derive a metric that finds the lowest-fee changeless -/// solution. Constraining to changeless should never take fewer rounds than the unconstrained -/// `LowestFee`. -#[test] -fn combined_changeless_metric() { - let params = common::StrategyParams { - n_candidates: 100, - target_value: 100_000, - target_weight: 1000 - TX_FIXED_FIELD_WEIGHT as u32 - 1, - replace: None, - feerate: 5.0, - feerate_lt_diff: -4.0, - drain_weight: 200, - drain_spend_weight: 600, - drain_dust: 200, - n_target_outputs: 1, - n_drain_outputs: 1, - max_weight: None, - }; - - let candidates = common::gen_candidates(params.n_candidates); - let mut cs_a = CoinSelector::new(&candidates); - let mut cs_b = CoinSelector::new(&candidates); - - let target = params.target(); - let metric_lowest_fee = params.lowest_fee_metric(); - - let metric_changeless = Changeless(params.lowest_fee_metric()); - - // cs_a uses the unconstrained metric - let (score, rounds) = common::bnb_search(&mut cs_a, target, metric_lowest_fee, usize::MAX) - .expect("must find solution"); - println!("score={:?} rounds={}", score, rounds); - - // cs_b uses the changeless-constrained metric - let (combined_score, combined_rounds) = - common::bnb_search(&mut cs_b, target, metric_changeless, usize::MAX) - .expect("must find solution"); - println!("score={:?} rounds={}", combined_score, combined_rounds); - - assert!(combined_rounds >= rounds); -} - /// Because this metric decides change optimally, it never creates a change output whose value /// wouldn't cover the future cost of spending it. Here a single input overshoots the target by only /// ~130 sats — far less than the drain's spend cost — so the fee-optimal choice is to burn the @@ -234,29 +193,31 @@ fn does_not_create_change_below_spend_cost() { max_weight: None, }; - let candidates = vec![ + let candidates = [ Candidate { value: 100_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, Candidate { value: 50_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, // NOTE: this input has negative effective value Candidate { value: 10, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, ]; - let mut cs = CoinSelector::new(&candidates); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let mut cs = CoinSelector::new(&problem); let drain_weights = DrainWeights { output_weight: 100, @@ -270,17 +231,18 @@ fn does_not_create_change_below_spend_cost() { drain_weights, }; - let (score, _) = common::bnb_search(&mut cs, target, metric, 10).expect("finds solution"); + let (score, _) = common::bnb_search(&mut cs, metric, 10).expect("finds solution"); // The optimal selection is candidate 0 alone, and it must be changeless. + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); let expected = { - let mut expected = CoinSelector::new(&candidates); + let mut expected = CoinSelector::new(&problem); expected.select(0); expected }; assert_eq!(cs.selected_indices(), expected.selected_indices()); assert!( - metric.drain(&cs, target).is_none(), + metric.drain(&cs).is_none(), "optimal selection must be changeless" ); @@ -290,12 +252,7 @@ fn does_not_create_change_below_spend_cost() { with_extra_input.select(2); with_extra_input }; - assert!( - score - <= metric - .score(&with_extra_input, target) - .expect("target is met") - ); + assert!(score <= metric.score(&with_extra_input).expect("target is met")); } #[test] @@ -317,18 +274,18 @@ fn zero_fee_tx() { max_weight: None, }; - let candidates = vec![ + let candidates = [ Candidate { value: 100_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, Candidate { value: 50_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, ]; @@ -338,14 +295,15 @@ fn zero_fee_tx() { n_outputs: 1, }; - let mut cs = CoinSelector::new(&candidates); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let mut cs = CoinSelector::new(&problem); let metric = LowestFee { long_term_feerate, dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), drain_weights, }; - let (_score, _rounds) = - common::bnb_search(&mut cs, target, metric, 1000).expect("must find solution"); + let (_score, _rounds) = common::bnb_search(&mut cs, metric, 1000).expect("must find solution"); } // --- `run_bnb` failure classification (`NoBnbSolution` variants) --- @@ -354,8 +312,8 @@ fn err_candidate(value: u64) -> Candidate { Candidate { value, weight: 272, // ~1 P2WPKH input - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, } } @@ -379,14 +337,15 @@ fn err_outputs(value_sum: u64) -> TargetOutputs { fn run_bnb_reports_insufficient_funds() { // Two 100k inputs can't cover a 10M target: the value is simply unreachable. let candidates = [err_candidate(100_000), err_candidate(100_000)]; - let mut cs = CoinSelector::new(&candidates); let target = Target { outputs: err_outputs(10_000_000), fee: TargetFee::ZERO, max_weight: None, }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); assert_eq!( - cs.run_bnb(target, err_metric(), 100_000).unwrap_err(), + cs.run_bnb(err_metric(), 100_000).unwrap_err(), NoBnbSolution::InsufficientFunds, ); } @@ -400,18 +359,38 @@ fn run_bnb_reports_max_weight_exceeded() { err_candidate(100_000), err_candidate(100_000), ]; - let mut cs = CoinSelector::new(&candidates); let target = Target { outputs: err_outputs(250_000), fee: TargetFee::ZERO, max_weight: Some(1), }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); assert_eq!( - cs.run_bnb(target, err_metric(), 100_000).unwrap_err(), + cs.run_bnb(err_metric(), 100_000).unwrap_err(), NoBnbSolution::MaxWeightExceeded, ); } +/// The search is seeded with the greedy selection, so a budget too small to search anything still +/// comes back with a usable answer instead of `RoundLimit`. Without that, a caller on a large pool +/// falls through to whatever fallback it has for something branch and bound could have covered. +#[test] +fn run_bnb_returns_the_greedy_selection_on_a_tight_budget() { + let candidates = core::iter::repeat(err_candidate(100_000)) + .take(500) + .collect::>(); + let target = Target { + outputs: err_outputs(1_000_000), + fee: TargetFee::ZERO, + max_weight: None, + }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); + cs.run_bnb(err_metric(), 1).expect("the seed is a solution"); + assert!(cs.is_funded()); +} + #[test] fn run_bnb_reports_round_limit() { // A solvable target, but zero rounds: we can't conclude infeasibility, only that we gave up. @@ -420,17 +399,70 @@ fn run_bnb_reports_round_limit() { err_candidate(100_000), err_candidate(100_000), ]; - let mut cs = CoinSelector::new(&candidates); let target = Target { outputs: err_outputs(250_000), fee: TargetFee::ZERO, max_weight: None, }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); assert_eq!( - cs.run_bnb(target, err_metric(), 0).unwrap_err(), + cs.run_bnb(err_metric(), 0).unwrap_err(), NoBnbSolution::RoundLimit { max_rounds: 0, rounds: 0, }, ); } + +/// A segwit and a legacy candidate with the same value and weight are *not* interchangeable: the +/// segwit one adds the witness header to the tx, and the legacy one doesn't. So excluding one must +/// not also ban the other, or branch and bound misses the cheaper (legacy-only) selection. +#[test] +fn does_not_ban_candidates_that_differ_only_in_script_type() { + let target = Target { + fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(10.0)), + outputs: TargetOutputs { + value_sum: 50_000, + weight_sum: 200 - TX_FIXED_FIELD_WEIGHT - 1, + n_outputs: 1, + }, + max_weight: None, + }; + let candidates = [ + Candidate { + value: 100_000, + weight: 472, + segwit_count: 1, + legacy_count: 0, + }, + Candidate { + value: 100_000, + weight: 472, + segwit_count: 0, + legacy_count: 1, + }, + ]; + let metric = LowestFee { + long_term_feerate: FeeRate::from_sat_per_vb(10.0), + dust_relay_feerate: FeeRate::from_sat_per_vb(1.0), + drain_weights: DrainWeights::TR_KEYSPEND, + }; + + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let mut exhaustive = CoinSelector::new(&problem); + let (best_score, _) = + common::exhaustive_search(&mut exhaustive, &mut metric.clone()).expect("solvable"); + assert!( + exhaustive.is_selected(1) && !exhaustive.is_selected(0), + "legacy-only is the optimum: {}", + exhaustive + ); + + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + + let mut cs = CoinSelector::new(&problem); + let (score, _) = cs.run_bnb(metric, 100).expect("solvable"); + assert_eq!(score, best_score, "bnb selected {}", cs); +} diff --git a/tests/srd.rs b/tests/srd.rs index b1b3096..df09df6 100644 --- a/tests/srd.rs +++ b/tests/srd.rs @@ -2,8 +2,8 @@ mod common; use bdk_coin_select::{ - Candidate, CoinSelector, Drain, DrainWeights, FeeRate, SelectError, Target, TargetFee, - TargetOutputs, CHANGE_LOWER, TR_SPK_WEIGHT, TXOUT_BASE_WEIGHT, + Candidate, CoinSelector, Drain, DrainWeights, FeeRate, SelectError, SelectionProblem, Target, + TargetFee, TargetOutputs, CHANGE_LOWER, TR_SPK_WEIGHT, TXOUT_BASE_WEIGHT, }; /// Deterministic, dependency-free `u64` source (SplitMix64) so we can drive `select_srd` without a @@ -36,8 +36,9 @@ fn srd_success_yields_healthy_change_that_meets_target() { let mut successes = 0; for seed in 0..300u64 { - let mut cs = CoinSelector::new(&candidates); - let result = cs.select_srd(target, drain_weights, CHANGE_LOWER, splitmix64(seed)); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); + let result = cs.select_srd(drain_weights, CHANGE_LOWER, splitmix64(seed)); if let Ok(drain) = result { successes += 1; @@ -49,18 +50,15 @@ fn srd_success_yields_healthy_change_that_meets_target() { ); assert_eq!(drain.weights, drain_weights); assert!( - cs.is_funded_with_drain(target, drain), + cs.is_funded_with_drain(drain), "seed {}: target not met with the returned drain", seed ); // The reported change equals the actual excess available to the drain. - let excess = cs.excess( - target, - Drain { - weights: drain_weights, - value: 0, - }, - ); + let excess = cs.excess(Drain { + weights: drain_weights, + value: 0, + }); assert_eq!(drain.value as i64, excess); } } @@ -71,32 +69,33 @@ fn srd_success_yields_healthy_change_that_meets_target() { #[test] fn srd_insufficient_funds() { // 3 * 50_000 = 150_000 total, well below target (200_000) + CHANGE_LOWER (50_000) + fees. - let candidates = vec![ + let candidates = [ Candidate { value: 50_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, Candidate { value: 50_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, Candidate { value: 50_000, weight: 100, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }, ]; let target = target(200_000, 5.0); let drain_weights = DrainWeights::TR_KEYSPEND; for seed in 0..50u64 { - let mut cs = CoinSelector::new(&candidates); - let result = cs.select_srd(target, drain_weights, CHANGE_LOWER, splitmix64(seed)); + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); + let result = cs.select_srd(drain_weights, CHANGE_LOWER, splitmix64(seed)); assert!( matches!(result, Err(SelectError::InsufficientFunds(_))), "seed {}: expected InsufficientFunds, got {:?}", @@ -117,8 +116,8 @@ fn srd_max_weight_exceeded() { Candidate { value: 100_000, weight: 1000, - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }; 10 ]; @@ -129,11 +128,13 @@ fn srd_max_weight_exceeded() { }; // Weight of the smallest selection that reaches target + change_lower, with no cap. - let mut probe = CoinSelector::new(&candidates); + let problem = + SelectionProblem::new_no_ancestors(target(200_000, 5.0), candidates.iter().copied()); + let mut probe = CoinSelector::new(&problem); probe - .select_until(|cs| cs.excess(target(200_000, 5.0), drain) >= CHANGE_LOWER as i64) + .select_until(|cs| cs.excess(drain) >= CHANGE_LOWER as i64) .expect("candidates can cover target + change_lower"); - let needed_weight = probe.weight(target(200_000, 5.0).outputs, drain_weights); + let needed_weight = probe.weight(drain_weights); // Cap just below that, so SRD trips the weight limit as it reaches `change_lower`. let capped = Target { @@ -142,8 +143,9 @@ fn srd_max_weight_exceeded() { }; for seed in 0..20u64 { - let mut cs = CoinSelector::new(&candidates); - let result = cs.select_srd(capped, drain_weights, CHANGE_LOWER, splitmix64(seed)); + let problem = SelectionProblem::new_no_ancestors(capped, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); + let result = cs.select_srd(drain_weights, CHANGE_LOWER, splitmix64(seed)); assert!( matches!(result, Err(SelectError::MaxWeightExceeded)), "seed {}: expected MaxWeightExceeded, got {:?}", @@ -166,13 +168,14 @@ fn srd_adds_nothing_when_already_sufficient() { }; // Preselect enough that the change already exceeds `change_lower`. - let mut cs = CoinSelector::new(&candidates); - cs.select_until(|cs| cs.excess(target, drain) >= CHANGE_LOWER as i64) + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut cs = CoinSelector::new(&problem); + cs.select_until(|cs| cs.excess(drain) >= CHANGE_LOWER as i64) .expect("candidates can cover target + change_lower"); let before: Vec = cs.selected_indices().iter().collect(); let out = cs - .select_srd(target, drain_weights, CHANGE_LOWER, splitmix64(3)) + .select_srd(drain_weights, CHANGE_LOWER, splitmix64(3)) .expect("already sufficient"); let after: Vec = cs.selected_indices().iter().collect(); diff --git a/tests/weight.rs b/tests/weight.rs index 6a8dbb5..8da613e 100644 --- a/tests/weight.rs +++ b/tests/weight.rs @@ -1,7 +1,11 @@ #![allow(clippy::zero_prefixed_literal)] -use bdk_coin_select::{Candidate, CoinSelector, Drain, DrainWeights, TargetOutputs}; +use bdk_coin_select::{ + Candidate, CoinSelector, Drain, DrainWeights, FeeRate, SelectionProblem, Target, TargetFee, + TargetOutputs, +}; use bitcoin::{consensus::Decodable, ScriptBuf, Transaction}; +use proptest::prelude::*; fn hex_val(c: u8) -> u8 { match c { @@ -21,6 +25,26 @@ pub fn hex_decode(hex: &str) -> Vec { bytes } +// https://mempool.space/tx/5f231df4f73694b3cca9211e336451c20dab136e0a843c2e3166cdcb093e91f4 +const THREE_INPUT_LEGACY_TX_HEX: &str = "0100000003fe785783e14669f638ba902c26e8e3d7036fb183237bc00f8a10542191c7171300000000fdfd00004730440220418996f20477d143d02ad47e74e5949641b6c2904159ab7c592d2cfc659f9bd802205b18f18ac86b714971f84a8b74a4cb14ad5c1a5b9d0d939bb32c6ae4032f4ea10148304502210091296ff8dd87b5ebfc3d47cb82cfe4750d52c544a2b88a85970354a4d0d4b1db022069632067ee6f30f06145f649bc76d5e5d5e6404dbe985e006fcde938f778c297014c695221030502b8ade694d57a6e86998180a64f4ce993372830dc796c3d561ad8b2a504de210272b68e1c037c4630eff7ea5858640cc0748e36f5de82fb38529ef1fd0a89670d2103ba0544a3a2aa9f2314022760b78b5c833aebf6f88468a089550f93834a2886ed53aeffffffff7e048a7c53a8af656e24442c65fe4c4299b1494f6c7579fe0fd9fa741ce83e3279000000fc004730440220018fa343acccd048ed8f8f179e1b6ae27435a41b5fb2c1d96a5a772777acc6dc022074783814f2100c6fc4d4c976f941212be50825814502ca0cbe3f929db789979e0147304402206373f01b73fb09876d0f5ee3087e0614cab3be249934bc2b7eb64ee67f53dc8302200b50f8a327020172b82aaba7480c77ecf07bb32322a05f4afbc543aa97d2fde8014c69522103039d906b2494e310f6c7774c98618be552720d04781e073dd3ff25d5906f22662103d82026baa529619b103ec6341d548a7eb6d924061a8469a7416155513a3071c12102e452bc4aa726d44646ba80db70465683b30efde282a19aa35c6029ae8925df5e53aeffffffffef80f0b1cc543de4f73d59c02a3c575ae5d0af17c1e11e6be7abe3325c777507ad000000fdfd00004730440220220fee11bf836621a11a8ea9100a4600c109c13895f11468d3e2062210c5481902201c5c8a462175538e87b8248e1ed3927c3a461c66d1b46215641c875e86eb22c4014830450221008d2de8c2f20a720129c372791e595b9602b1a9bce99618497aec5266148ffc1302203a493359d700ed96323f8805ed03e909959ff0f22eff359028db6861486b1555014c6952210374a4add33567f09967592c5bcdc3db421fdbba67bac4636328f96d941da31bd221039636c2ffac90afb7499b16e265078113dfb2d77b54270e37353217c9eaeaf3052103d0bcea6d10cdd2f16018ea71572631708e26f457f67cda36a7f816a87f7791d253aeffffffff04977261000000000016001470385d054721987f41521648d7b2f5c77f735d6bee92030000000000225120d0cda1b675a0b369964cbfa381721aae3549dd2c9c6f2cf71ff67d5bc277afd3f2aaf30000000000160014ed2d41ba08313dbb2630a7106b2fedafc14aa121d4f0c70000000000220020e5c7c00d174631d2d1e365d6347b016fb87b6a0c08902d8e443989cb771fa7ec00000000"; + +/// The 3-legacy-input mainnet tx above. +fn legacy_three_input_tx() -> Transaction { + Transaction::consensus_decode(&mut hex_decode(THREE_INPUT_LEGACY_TX_HEX).as_slice()).unwrap() +} + +/// The same tx with the middle input turned into a (semi-realistic) P2WPKH segwit spend. +fn legacy_three_input_tx_mixed() -> Transaction { + let mut tx = legacy_three_input_tx(); + tx.input[1].script_sig = ScriptBuf::default(); + tx.input[1].witness = vec![ + // semi-realistic p2wpkh spend + hex_decode("3045022100bdc115b86e9c863279132b4808459cf9b266c8f6a9c14a3dfd956986b807e3320220265833b85197679687c5d5eed1b2637489b34249d44cf5d2d40bc7b514181a5101"), + hex_decode("02077741a668889ce15d59365886375aea47a7691941d7a0d301697edbc773b45b"), + ].into(); + tx +} + #[test] fn segwit_one_input_one_output() { // FROM https://mempool.space/tx/e627fbb7f775a57fd398bf9b150655d4ac3e1f8afed4255e74ee10d7a345a9cc @@ -35,8 +59,8 @@ fn segwit_one_input_one_output() { .map(|(txin, value)| Candidate { value, weight: txin.segwit_weight().to_wu(), - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }) .collect::>(); @@ -46,16 +70,22 @@ fn segwit_one_input_one_output() { n_outputs: tx.output.len(), }; - let mut coin_selector = CoinSelector::new(&candidates); + let target = Target { + outputs: target_ouputs, + fee: TargetFee::ZERO, + max_weight: None, + }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut coin_selector = CoinSelector::new(&problem); coin_selector.select_all(); assert_eq!( - coin_selector.weight(target_ouputs, DrainWeights::NONE), + coin_selector.weight(DrainWeights::NONE), tx.weight().to_wu() ); assert_eq!( (coin_selector - .implied_feerate(target_ouputs, Drain::NONE) + .implied_feerate(Drain::NONE) .unwrap() .as_sat_vb() * 10.0) @@ -78,28 +108,33 @@ fn segwit_two_inputs_one_output() { .map(|(txin, value)| Candidate { value, weight: txin.segwit_weight().to_wu(), - input_count: 1, - is_segwit: true, + segwit_count: 1, + legacy_count: 0, }) .collect::>(); - let mut coin_selector = CoinSelector::new(&candidates); - let target_ouputs = TargetOutputs { value_sum: tx.output.iter().map(|output| output.value.to_sat()).sum(), weight_sum: tx.output.iter().map(|output| output.weight().to_wu()).sum(), n_outputs: tx.output.len(), }; + let target = Target { + outputs: target_ouputs, + fee: TargetFee::ZERO, + max_weight: None, + }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut coin_selector = CoinSelector::new(&problem); coin_selector.select_all(); assert_eq!( - coin_selector.weight(target_ouputs, DrainWeights::NONE), + coin_selector.weight(DrainWeights::NONE), tx.weight().to_wu() ); assert_eq!( (coin_selector - .implied_feerate(target_ouputs, Drain::NONE) + .implied_feerate(Drain::NONE) .unwrap() .as_sat_vb() * 10.0) @@ -122,8 +157,8 @@ fn legacy_three_inputs() { .map(|(txin, value)| Candidate { value, weight: txin.legacy_weight().to_wu(), - input_count: 1, - is_segwit: false, + segwit_count: 0, + legacy_count: 1, }) .collect::>(); @@ -133,16 +168,22 @@ fn legacy_three_inputs() { n_outputs: tx.output.len(), }; - let mut coin_selector = CoinSelector::new(&candidates); + let target = Target { + outputs: target_ouputs, + fee: TargetFee::ZERO, + max_weight: None, + }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut coin_selector = CoinSelector::new(&problem); coin_selector.select_all(); assert_eq!( - coin_selector.weight(target_ouputs, DrainWeights::NONE), + coin_selector.weight(DrainWeights::NONE), orig_weight.to_wu() ); assert_eq!( (coin_selector - .implied_feerate(target_ouputs, Drain::NONE) + .implied_feerate(Drain::NONE) .unwrap() .as_sat_vb() * 10.0) @@ -179,8 +220,8 @@ fn legacy_three_inputs_one_segwit() { txin.legacy_weight() } .to_wu(), - input_count: 1, - is_segwit, + segwit_count: is_segwit as usize, + legacy_count: !is_segwit as usize, } }) .collect::>(); @@ -191,11 +232,145 @@ fn legacy_three_inputs_one_segwit() { n_outputs: tx.output.len(), }; - let mut coin_selector = CoinSelector::new(&candidates); + let target = Target { + outputs: target_ouputs, + fee: TargetFee::ZERO, + max_weight: None, + }; + let problem = SelectionProblem::new_no_ancestors(target, candidates.iter().copied()); + let mut coin_selector = CoinSelector::new(&problem); coin_selector.select_all(); assert_eq!( - coin_selector.weight(target_ouputs, DrainWeights::NONE), + coin_selector.weight(DrainWeights::NONE), + tx.weight().to_wu() + ); +} + +#[test] +fn legacy_three_inputs_grouped() { + // Same tx as `legacy_three_inputs`, but all three legacy inputs carried by a single candidate. + // No witness section is serialized, so nothing is added per legacy input — this guards the + // all-legacy path (it also passed under the old per-candidate accounting). + let tx = legacy_three_input_tx(); + let input_values = [022_680_000, 006_558_175, 006_558_200]; + + let candidates = [Candidate { + value: input_values.iter().sum(), + weight: tx + .input + .iter() + .map(|txin| txin.legacy_weight().to_wu()) + .sum(), + segwit_count: 0, + legacy_count: tx.input.len(), + }]; + + let target_ouputs = TargetOutputs { + value_sum: tx.output.iter().map(|output| output.value.to_sat()).sum(), + weight_sum: tx.output.iter().map(|output| output.weight().to_wu()).sum(), + n_outputs: tx.output.len(), + }; + + let problem = SelectionProblem::new_no_ancestors( + Target { + fee: TargetFee::ZERO, + outputs: target_ouputs, + max_weight: None, + }, + candidates.iter().copied(), + ); + let mut coin_selector = CoinSelector::new(&problem); + coin_selector.select_all(); + + assert_eq!( + coin_selector.weight(DrainWeights::NONE), + tx.weight().to_wu() + ); +} + +#[test] +fn legacy_pair_grouped_with_segwit_input() { + // Same tx as `legacy_three_inputs_one_segwit`, but the two legacy inputs are grouped into a + // single candidate. In a segwit tx each legacy input still serializes an (empty) witness + // costing 1 WU, so the grouped candidate must pay 2 WU — not 1 — for its two empty witnesses. + let tx = legacy_three_input_tx_mixed(); + let input_values = [022_680_000, 006_558_175, 006_558_200]; + + let candidates = [ + Candidate { + value: input_values[0] + input_values[2], + weight: tx.input[0].legacy_weight().to_wu() + tx.input[2].legacy_weight().to_wu(), + segwit_count: 0, + legacy_count: 2, + }, + Candidate { + value: input_values[1], + weight: tx.input[1].segwit_weight().to_wu(), + segwit_count: 1, + legacy_count: 0, + }, + ]; + + let target_ouputs = TargetOutputs { + value_sum: tx.output.iter().map(|output| output.value.to_sat()).sum(), + weight_sum: tx.output.iter().map(|output| output.weight().to_wu()).sum(), + n_outputs: tx.output.len(), + }; + + let problem = SelectionProblem::new_no_ancestors( + Target { + fee: TargetFee::ZERO, + outputs: target_ouputs, + max_weight: None, + }, + candidates.iter().copied(), + ); + let mut coin_selector = CoinSelector::new(&problem); + coin_selector.select_all(); + + assert_eq!( + coin_selector.weight(DrainWeights::NONE), + tx.weight().to_wu() + ); +} + +#[test] +fn mixed_group_all_inputs_one_candidate() { + // Same tx as `legacy_three_inputs_one_segwit`, with all three inputs — legacy *and* segwit — + // in a single mixed candidate. `legacy_count`/`segwit_count` price this exactly: 2 WU for the + // two empty legacy witnesses, the segwit header once, and a 3-input varint. + let tx = legacy_three_input_tx_mixed(); + let input_values = [022_680_000, 006_558_175, 006_558_200]; + + let candidates = [Candidate { + value: input_values.iter().sum(), + weight: tx.input[0].legacy_weight().to_wu() + + tx.input[1].segwit_weight().to_wu() + + tx.input[2].legacy_weight().to_wu(), + segwit_count: 1, + legacy_count: 2, + }]; + + let target_outputs = TargetOutputs { + value_sum: tx.output.iter().map(|output| output.value.to_sat()).sum(), + weight_sum: tx.output.iter().map(|output| output.weight().to_wu()).sum(), + n_outputs: tx.output.len(), + }; + + let problem = SelectionProblem::new_no_ancestors( + Target { + fee: TargetFee::ZERO, + outputs: target_outputs, + max_weight: None, + }, + candidates.iter().copied(), + ); + let mut coin_selector = CoinSelector::new(&problem); + coin_selector.select_all(); + + assert_eq!( + coin_selector.weight(DrainWeights::NONE), tx.weight().to_wu() ); } @@ -211,3 +386,99 @@ fn new_tr_keyspend_correct_weight() { Candidate::new_tr_keyspend(420).weight ); } + +proptest! { + /// `CoinSelector` keeps running sums of the selected candidates. After any sequence of + /// selects and deselects they must match a recompute from the selected set. + #[test] + fn running_sums_match_recompute( + candidates in proptest::collection::vec( + (0u64..1_000_000, 0u64..2_000, 0usize..4, 0usize..4).prop_map( + |(value, weight, segwit_count, legacy_count)| Candidate { + value, + weight, + segwit_count, + legacy_count, + }, + ), + 1..300, + ), + ops in proptest::collection::vec((any::(), any::()), 0..600), + ) { + let problem = SelectionProblem::new_no_ancestors( + Target { + fee: TargetFee::ZERO, + outputs: TargetOutputs::fund_outputs([]), + max_weight: None, + }, + candidates.iter().copied(), + ); + let mut cs = CoinSelector::new(&problem); + for (index, select) in ops { + let index = index.index(candidates.len()); + if select { + cs.select(index); + } else { + cs.deselect(index); + } + + let selected = cs.selected().map(|(_, c)| c).collect::>(); + let is_segwit_tx = selected.iter().any(|c| c.segwit_count > 0); + let input_count = selected.iter().map(|c| c.segwit_count + c.legacy_count).sum::(); + let varint_size = match input_count { + 0..=0xfc => 1, + 0xfd..=0xffff => 3, + _ => 5, + }; + let expected_weight = varint_size * 4 + + selected + .iter() + .map(|c| c.weight + if is_segwit_tx { c.legacy_count as u64 } else { 0 }) + .sum::() + + if is_segwit_tx { 2 } else { 0 }; + + prop_assert_eq!(cs.input_weight(), expected_weight); + prop_assert_eq!(cs.selected_value(), selected.iter().map(|c| c.value).sum::()); + } + } +} + +/// Adding the first segwit input adds the witness header, so a candidate worth more than its own +/// weight can still lower the excess. `is_fundable` must not reject a selection that is already +/// funded just because adding every such candidate un-funds it. +#[test] +fn is_fundable_never_rejects_an_already_funded_mixed_selection() { + let target = Target { + fee: TargetFee::from_feerate(FeeRate::from_sat_per_vb(4.0)), + outputs: TargetOutputs { + value_sum: 1_000, + weight_sum: 0, + n_outputs: 0, + }, + max_weight: None, + }; + let candidates = [ + Candidate { + value: 1_201, + weight: 158, + segwit_count: 0, + legacy_count: 1, + }, + Candidate { + value: 21, + weight: 20, + segwit_count: 1, + legacy_count: 0, + }, + ]; + let problem = SelectionProblem::new_no_ancestors(target, candidates); + let mut selector = problem.selector(); + selector.select(0); + assert!(selector.is_funded()); + assert!(problem.candidate(1).effective_value(target.fee.rate) > 0.0); + + let mut all = selector.clone(); + all.select(1); + assert!(!all.is_funded(), "the witness header un-funds it"); + assert!(selector.is_fundable()); +}