Add shared Bitcoin transaction vectors

[benma-agent]

Add a readable source of Bitcoin transaction signing vectors, authored
as PSBTs with the metadata needed by signing APIs. Derive firmware
requests in memory, exercise them in signtx.rs, and remove overlapping
bespoke tests.

For successful vectors, populate the PSBT with the firmware responses
and check that the completed transaction and its signatures are valid.

The generated fixtures are meant to be consumed by client libraries so
firmware and clients can cover the same transactions and versioned
expectations without repo-specific vector sources. They will also be
useful when adding the Bitcoin API to the new bitbox-api-ts library.

Keep the generated JSON synchronized with its Rust source through a
drift test.

[cedwies]

Concept ACK, looks good. Here are some points I'd like to raise for continuing:

1. Manual copying is reasonable for proving the concept. Do you want an automated update/hash check for production?
2. For some test vectors you check the shape (correct signature kind, valid looking encoding, ...) and not whether the signature is cryptographically correct. Will the finished vector-based firmware tests independently verify all returned signatures, and will client tests keep equivalent validation for every case they support? Or how do we prevent all clients from agreeing with a wrong answer?
3. I think skipping cases should be made explicit and visible in order to avoid silent skipping while CI remains green. Like: These exact eight vectors are intentionally unsupported. If that list changes, fail (and require a human decision?).

[benma]

Concept ACK, looks good. Here are some points I'd like to raise for continuing:

Thanks!

1. Manual copying is reasonable for proving the concept. Do you want an automated update/hash check for production?

I don't think it's needed, at least not for the time being. Similar to protobuf messages, they are also copied to the client libs on demand. Maybe a small utility tool or Makefile entry to make it easier, but it's only copying one file 🤷

2. For some test vectors you check the shape (correct signature kind, valid looking encoding, ...) and not whether the signature is cryptographically correct. Will the finished vector-based firmware tests independently verify all returned signatures, and will client tests keep equivalent validation for every case they support? Or how do we prevent all clients from agreeing with a wrong answer?

The signatures are created inside the table driven unit tests, and their validity (full tx verification including sigs) is checked in the client libs. I thought this was also done in the vector tests in this repo, but I was wrong, so it slipped through - will add it here too.

3. I think skipping cases should be made explicit and visible in order to avoid silent skipping while CI remains green. Like: `These exact eight vectors are intentionally unsupported. If that list changes, fail (and require a human decision?).`

Skipping happens when the vector is not supported, and it is already explicit. Afaik in the firmware, all vectors are used. In the Rust lib, the vectors with silent_payments / payment_requests are skipped as not supported yet:

fn skip_reason(vector: &TestVector) -> Option<&'static str> {
    if !vector.psbt.options.outputs.is_empty() || !vector.psbt.options.payment_requests.is_empty() {
        return Some("btc_sign_psbt does not expose per-output options or payment requests");
    }
    None
}

The Go lib also supports all.

Did I miss something?

[benma]

The signatures are created inside the table driven unit tests, and their validity (full tx verification including sigs) is checked in the client libs. I thought this was also done in the vector tests in this repo, but I was wrong, so it slipped through - will add it here too.

@cedwies added the validity/sig-check now in a 2nd commit

[cedwies]

Nice, regarding 1:

ACK, makes sense.

regarding 3:

The rule is explicit, but my concern was slightly different: The exact amount of skipped coverage is not fixed or enforced. If today 8 vectors match the skip condition, and another vector is added and also skipped, this can be overlooked. But this would be a small follow-up at most, and not relevant to the concept ACK.

[benma]

regarding 3:

The rule is explicit, but my concern was slightly different: The exact amount of skipped coverage is not fixed or enforced. If today 8 vectors match the skip condition, and another vector is added and also skipped, this can be overlooked. But this would be a small follow-up at most, and not relevant to the concept ACK.

Makes sense to help reviewers see which test inclusions/exclusions happened, but I agree it's not urgent.

I lifted this PR out of draft, ready for actual review (along with the PRs in the other repos)

[cedwies]

utACK, please see my comment

[cedwies]

Nonblocking: For InvalidInput vectors that are intended to fail during pre-sign validation, would it make sense to also assert that observations.signatures is empty? Before, this was not checked either, so this would not be a regression. It would protect against a regression where the firmware returns a signature for Invalid Input.