Skip to content

add BIP141 conventions for serialized coinbase #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
GitGab19 merged 3 commits into from
Sep 9, 2025
Merged

add BIP141 conventions for serialized coinbase #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4ddd08e
add BIP141 conventions
plebhash Aug 26, 2025
1f3cc6f
partially add suggestions from Sjors review
plebhash Sep 1, 2025
7251e64
add section for BIP141 on NewTemplate
plebhash Sep 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions 03-Protocol-Overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -286,3 +286,65 @@ Upon receiving the message, the client re-initiates the Noise handshake and uses
For security reasons, it is not possible to reconnect to a server with a certificate signed by a different pool authority key.
The message intentionally does not contain a **pool public key** and thus cannot be used to reconnect to a different pool.
This ensures that an attacker will not be able to redirect hashrate to an arbitrary server should the pool server get compromised and instructed to send reconnects to a new location.

## 3.7 BIP141

[BIP141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki) introduced the notion of Segregated Witness (SegWit) into the Bitcoin protocol.

This deserves some consideration in the Stratum V2 protocol design, mainly because this affects how the Coinbase transaction is serialized across different messages.
Copy link
Contributor

@Sjors Sjors Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So maybe add:

For a block that contains any SegWit transactions (in practice almost any non-empty block), the coinbase transaction MUST have a witness as well as an OP_RETURN output.

The OP_RETURN output is provided by Template Provider in the coinbase_tx_outputs field.

The coinbase witness is not provided by the Template Provider. Although today it's typically 0x00..00, no assumption should be made about this value. The only consensus requirement is that it is a 32 byte push, but future consensus rules may require a different value.

For a block that does not contain any SegWit transactions, the coinbase transaction generated by current versions of Bitcoin Core won't have a witness. But that's not a consensus rule. Future versions and other node implements MAY set a witness anyway.

In other words, if the Template Provider provides an empty block, it is not safe to assume its coinbase transaction doesn't have a witness.

Copy link
Member Author

@plebhash plebhash Aug 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OP_RETURN output is provided by Template Provider in the coinbase_tx_outputs field.

Although today it's typically 0x00..00, no assumption should be made about this value.

These two sentences are conflicting.

TP provides an OP_RETURN with a commitment hash that's calculated with a Coinbase witness as part of its pre-image. So whoever is downstream to TP is pretty much forced to abide by the Coinbase witness that TP chose, despite the fact that this is not really communicated anywhere on Sv2 TDP.

Unfortunately, there is a NewTemplate.coinbase_tx_outputs (where OP_RETURN with witness commitments is sent), but there's no NewTemplate.coinbase_witness or alike.

As expected, our current TP implementation does it with a 0x00..00 witness.

I validated this empirically, by trying to propagate a block with a witness commitment sent by our current TP and a Coinbase witness different than 0x00..00. The block was rejected by its peers.

So either:

  • A: TP should stop providing the OP_RETURN with the witness commitment, leaving it for the receiving end to decide which Coinbase witness they want to use and calculate a witness commitment accordingly
    OR
  • B: we establish some convention that says it's always 0x00..00 until some soft-fork changes it into consensus-critical

Option A would pretty much render NewTemplate.coinbase_tx_outputs useless. Or at least, AFAIK, sending Witness commitment is the only meaningful purpose for this field.

Option B would force a spec upgrade if-and-when a soft-fork changes things with regards to Coinbase's witness reserved value.

Copy link
Contributor

@Sjors Sjors Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Option B would force a spec upgrade if-and-when a soft-fork changes things with regards to Coinbase's witness reserved value.

This would be a disaster, because it's hard enough to get miners to upgrade their node software, let alone their entire sv2 stack.

It sounds like closing #15 was a mistake?

Copy link
Contributor

@Sjors Sjors Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As expected, our current TP implementation does it with a 0x00..00 witness.

I'm a bit confused about what you mean here. You're referring to the TP I implemented (sv2-tp and its bitcoind -sv2 predecessor), right? This lets Bitcoin Core generate a block template. The coinbase it uses for that is never communicated, but it indeed uses 0x00..00 and the OP_RETURN value reflects that.

I validated this empirically, by trying to propagate a block with a witness commitment sent by our current TP and a Coinbase witness different than 0x00..00. The block was rejected by its peers.

For SubmitSolution the coinbase has to be reconstructed by other SRI roles. The Template Provider just passes it along. Are you saying that if these other roles pick a different value than 0x00..00 then the result is broken? That would make sense.

Copy link
Member Author

@plebhash plebhash Aug 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds like closing #15 was a mistake?

According to my understanding, yes. Or at least as far as we expect:

  • the Template Provider to be the one who calculates the Witness Commitment
  • Sv2 to be reasonably future-proof with regards to soft-forks

When it comes to who's responsible for calculating the Witness Commitment, an alternative option would be to shift this responsibility to the Sv2 role who receives NewTemplate. The tradeoffs would be:

  • this would delay creation of jobs, since:
    • a RequestTransactionData/.Success round-trip would be necessary
    • we'd have to wait for the calculation of the Witness Root
  • we don't know what kind of consensus-critical data would have to be committed into witness reserved value on this hypothetical soft fork... does the Sv2 role have all the necessary data to make that decision, or is that something that only a Bitcoin Full Node could do?

So having TP be responsible for calculating the Witness Commitment feels like a sensible design decision.

But as long as NewTemplate carries no information about Coinbase witness (which was proposed and rejected on #15), the Sv2 role that receives this message and wants to propagate a block later has no option but to ASSUME a value for Coinbase witness. And today we live with this "implied convention" of using 0x000...000, which is an intuitively sensible choice (but really could have been anything, given that this field is not consensus-critical).

It's kinda sad to realize that the protocol is not future-proof on this regard, especially given that this could have been solved with just one extra field on NewTemplate, as #15 proposed.

However, I must also point out that the purpose of this current PR is not to advocate for spec changes, but rather to simply establish conventions that avoids confusion and enables smooth interoperability with what the Bitcoin protocol consensus looks like today. So I'm pretty ok with saying "just use 0x000...000 for witness reserved value" as an official convention and leaving future/hypothetical problems for the future.

In case protocol stakeholders (@jakubtrnka @Fi3 @TheBlueMatt) feel this deserves a discussion for spec change, we can do it on a separate issue/PR.

Copy link
Member Author

@plebhash plebhash Sep 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a few words suggested by @Sjors #153 (comment)

but not all

that's because I feel this topic about future implications of Coinbase witness is derailing into a discussion on its own, and as stated above I'd like to keep the scope of this PR constrained to conventions around serialization without diving into potential spec changes

so I crated #156 as a place where we can discuss about it more freely

Copy link
Contributor

@Sjors Sjors Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think it's worth mentioning that the witness commitment is (currently) assumed to be zero.

Copy link
Member Author

@plebhash plebhash Sep 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean the Coinbase witness (currently aka witness reserved value) is assumed to be zero, right?

the witness commitment is 0x6a24aa21a9ed followed by the hash of the concatenation of witness root and witness reserved value, so we don't really expect it to be zero.

anyways, just added a commit for this:

7251e64

Copy link
Contributor

@Sjors Sjors Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean the Coinbase witness (currently aka witness reserved value)

Yes


For a block that contains any SegWit transactions (in practice almost any non-empty block), the Coinbase transaction MUST have a witness as well as an `OP_RETURN` output carrying the witness commitment. For an empty block, the Coinbase transaction MAY have a witness and the `OP_RETURN` output with the witness commitment anyway.

The `OP_RETURN` output with the witness commitment is provided by Sv2 Template Providers in the `coinbase_tx_outputs` field of `NewTemplate` message.

On a serialized SegWit transaction, the BIP141 fields are:
- marker
- flag
- witness count
- witness length
- witness

The presence or absence of these fields on a serialized Coinbase has important implications on the Stratum V2 protocol. More specifically, the following messages are affected:
- `NewExtendedMiningJob` of Mining Protocol
- `DeclareMiningJob` of Job Declaration Protocol
- `SubmitSolution` of Template Distribution Protocol
- `NewTemplate` of Template Distribution Protocol

### 3.7.1 BIP141 on `NewExtendedMiningJob`

On the Mining Protocol's `NewExtendedMiningJob` there are two fields affected by BIP141:
- `coinbase_tx_prefix`
- `coinbase_tx_suffix`

When concatenated with `extranonce_prefix` + `extranonce`, these fields form a serialized Coinbase transaction that is then hashed to produce a `txid`. Together with `merkle_path`, this `txid`
will then form the `merkle_root` of the Block Header.

In case the Template's Coinbase is a SegWit transaction, BIP141 fields MUST be stripped away from `coinbase_tx_prefix` and `coinbase_tx_suffix`, otherwise clients would be calculating a `merkle_root` with the Coinbase's `wtxid`, which goes against Bitcoin Consensus.
Copy link
Contributor

@Sjors Sjors Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So IIUC NewExtendedMiningJob is the only place that needs to strip witness data. Maybe that suggests a defect in the spec? I'm not familiar with how extended mining jobs work, but isn't there a way to avoid stripping witness data still give clients the data they need to get the correct merkle_root?

Copy link
Member Author

@plebhash plebhash Aug 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, first some recap on Sv2 Standard vs Extended Jobs:

Standard Jobs have a NewMiningJob.merkle_root field. That's meant to be an innovation that makes messages smaller and puts the burden of Merkle Root calculation on the Server. There's no extranonce rolling.

Extended Jobs are closer to Sv1, in the sense that there's a coinbase_tx_prefix (akin to Sv1 Coinbase1) and coinbase_tx_suffix (akin to Sv1 Coinbase2). That's meant to allow either:

  • broadcasting a single job to multiple channels
  • allowing extranonce rolling

AFAIK most Sv1 servers send their mining.notify with Coinbase1 + Coinbase2 that's already stripped of BIP141 fields. Maybe some Sv1 firmware is able to detect whether that's not the case and strip it afterwards, but according to @Fi3 that's not ubiquous, so not safe to assume.

Until recently, SRI was sending and expecting NewExtendedMiningJob messages that contained BIP141 fields serialized into coinbase_tx_prefix and coinbase_tx_suffix. Until stratum-mining/stratum#1399 led us to realize that Braiins was doing Extended Jobs that follow the Sv1 pattern of stripping those.

isn't there a way to avoid stripping witness data still give clients the data they need to get the correct merkle_root

We could leave it up for the client to figure out whether NewExtendedMiningJob already stripped BIP141 or not, and stripping it before hashing. But @TheBlueMatt opposed this idea here and here (with some mixup in terminology).

I agree with you that it's a bit confusing to follow one pattern for one message and a different pattern in other messages. @Fi3 also pointed this out here, which motivated the addition of the new BIP141 section into 03-Protocol-Overview.md on this PR.

Overall, when it comes to NewExtendedMiningJob I'm not really opinionated towards doing one way or the other.

But I'm strongly opinionated that it's of upmost importance to have clear (even if somewhat counter-intuitive) conventions on the spec, which is why I'm pushing for these discussions. Not having anything is a recipe for interoperability hell, as we already started to see with stratum-mining/stratum#1399

Copy link
Contributor

@Sjors Sjors Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have a clear distinction between Stratum v2 and Stratum v1, with the Translator role taking care of serialisation changes.

Copy link
Member Author

@plebhash plebhash Aug 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fairness, the receiving end of NewExtendedMiningJob (be it Sv2 firmware or tProxy) has no purpose for BIP141 fields, given that coinbase_tx_prefix + coinbase_tx_suffix are only meant to enable calculation of the Coinbase's txid then merkle_root

so one could argue that forcing NewExtendedMiningJob to carry BIP141 would be bloating the message size unnecessarily, and we'd be sacrificing bandwidth efficiency with no real purpose other than making the spec slightly less confusing for the reader

as much as I value clarity and cleanness in the spec, this feels like a reasonable tradeoff?

Copy link
Contributor

@Sjors Sjors Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally think that a tiny increase in message size is worth the reduction in complexity, unless it you're at the edge of some network packet size and want to prevent roundtrips.

But for the purpose of this PR it may be better to just document things as they are now.


The server MUST retain the original value of those fields so that it can reconstruct the Coinbase as a SegWit transaction whenever it needs to propagate a block.

### 3.7.2 BIP141 on `DeclareMiningJob`

On the Job Declaration Protocol's `DeclareMiningJob` there are two fields affected by BIP141:
- `coinbase_tx_prefix`
- `coinbase_tx_suffix`

Differently from `NewExtendedMiningJob`, in case the Template's Coinbase is a SegWit transaction, BIP141 fields MUST NOT be stripped from `DeclareMiningJob`'s `coinbase_tx_prefix` and `coinbase_tx_suffix`.

That's because JDS needs to be able to reconstruct the Coinbase as a SegWit transaction whenever it needs to propagate a block.

### 3.7.3 BIP141 on `SubmitSolution`

On the Template Distribution Protocol's `SubmitSolution` there is one field affected by BIP141:
- `coinbase_tx`

Differently from `NewExtendedMiningJob`, in case the Template's Coinbase is a SegWit transaction, BIP141 fields MUST NOT be stripped from `SubmitSolution`'s `coinbase_tx`.

That's because the Template Distribution Server would not be able to propagate a block without that data.

### 3.7.4. BIP141 on `NewTemplate`

On the Template Distribution Protocol's `NewTemplate` there is one field affected by BIP141:
- `coinbase_tx_outputs`

In case of blocks containing SegWit transactions (and optionally blocks that don't as well), this field carries the `OP_RETURN` output with the witness commitment. The `witness reserved value` (Coinbase witness) used for calculating this witness commitment is assumed to be 32 bytes of `0x00`, as it currently holds no consensus-critical meaning. This [may change in future soft-forks](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#extensible-commitment-structure).
2 changes: 2 additions & 0 deletions 05-Mining-Protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,6 +427,8 @@ A proxy MUST translate the message for all downstream channels belonging to the
- For a **standard channel**: `extranonce_prefix`
- For an **extended channel**: `extranonce_prefix + extranonce (=N bytes)`, where `N` is the negotiated extranonce space for the channel (`OpenMiningChannel.Success.extranonce_size`)

\*If the original coinbase is a SegWit transaction, `coinbase_tx_prefix` and `coinbase_tx_suffix` MUST be stripped of BIP141 fields (marker, flag, witness count, witness length and witness reserved value).

### 5.3.17 `SetNewPrevHash` (Server -> Client, broadcast)

Prevhash is distributed whenever a new block is detected in the network by an upstream node or when a new downstream opens a channel.
Expand Down
2 changes: 2 additions & 0 deletions 06-Job-Declaration-Protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,8 @@ A request sent by JDC that proposes a selected set of transactions to JDS.
| coinbase_tx_suffix | B0_64K | Serialized bytes representing the final part of the coinbase transaction (after extranonce) |
| tx_ids_list | SEQ0_64K[U256] | List of hashes of the transaction set contained in the template. JDS checks the list against its mempool and requests missing txs via `ProvideMissingTransactions`. Does not include the coinbase transaction (as there is no corresponding full data for it yet). |
| excess_data | B0_64K | Extra data which the Pool may require to validate the work (as defined in the Template Distribution Protocol) |
\*Differently from `NewExtendedMiningJob`, if the original coinbase is a SegWit transaction, `coinbase_tx_prefix` and `coinbase_tx_suffix` MUST NOT be stripped of BIP141 fields (marker, flag, witness count, witness length and witness reserved value).


### 6.4.5 `DeclareMiningJob.Success` (Server -> Client)

Expand Down
4 changes: 4 additions & 0 deletions 07-Template-Distribution-Protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ The primary template-providing function. Note that the `coinbase_tx_outputs` byt

Please note that differently from `SetCustomMiningJob.coinbase_tx_outputs` and `AllocateMiningJobToken.Success.coinbase_tx_outputs`, `NewTemplate.coinbase_tx_outputs` MUST NOT be serialized as a CompactSize-prefixed array. This field must simply carry the ordered sequence of consensus‑serialized outputs, but the number of outputs MUST be inferred from `NewTemplate.coinbase_tx_outputs_count`. This is the equivalent of taking a CompactSize-prefixed array and dropping its (outer) prefix.

Please also note that in case the block contains SegWit transactions (and optionally blocks that don't as well), `NewTemplate.coinbase_tx_outputs` MUST carry the witness commitment. The `witness reserved value` (Coinbase witness) used for calculating this witness commitment is assumed to be 32 bytes of `0x00`, as it currently holds no consensus-critical meaning. This [may change in future soft-forks](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#extensible-commitment-structure).

## 7.3 `SetNewPrevHash` (Server -> Client)

Upon successful validation of a new best block, the server MUST immediately provide a `SetNewPrevHash` message.
Expand Down Expand Up @@ -118,3 +120,5 @@ Upon finding a coinbase transaction/nonce pair which double-SHA256 hashes at or
| header_timestamp | U32 | The nTime field in the block header. This MUST be greater than or equal to the header_timestamp field in the latest SetNewPrevHash message and lower than or equal to that value plus the number of seconds since the receipt of that message. |
| header_nonce | U32 | The nonce field in the header |
| coinbase_tx | B0_64K | The full serialized coinbase transaction, meeting all the requirements of the NewTemplate message, above |

\*Differently from `NewExtendedMiningJob`, if the original coinbase is a SegWit transaction, `coinbase_tx` MUST NOT be stripped of BIP141 fields (marker, flag, witness count, witness length and witness reserved value).
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ This repository contains the Stratum V2 protocol specification.
- [3.6.3. `SetupConnection.Error` (Server -> Client)](./03-Protocol-Overview.md#363-setupconnectionerror-server---client)
- [3.6.4. `ChannelEndpointChanged` (Server -> Client)](./03-Protocol-Overview.md#364-channelendpointchanged-server---client)
- [3.6.5. `Reconnect` (Server -> Client)](./03-Protocol-Overview.md#365-reconnect-server---client)
- [3.7. BIP141](./03-Protocol-Overview.md#37-bip141)
- [3.7.1. BIP141 on `NewExtendedMiningJob`](./03-Protocol-Overview.md#371-bip141-on-newextendedminingjob)
- [3.7.2. BIP141 on `DeclareMiningJob`](./03-Protocol-Overview.md#372-bip141-on-declareminingjob)
- [3.7.3. BIP141 on `SubmitSolution`](./03-Protocol-Overview.md#373-bip141-on-submitsolution)
- [3.7.4. BIP141 on `NewTemplate`](./03-Protocol-Overview.md#374-bip141-on-newtemplate)
- [4. Protocol Security](./04-Protocol-Security.md#4-protocol-security)
- [4.1. Motivation for Authenticated Encryption with Associated Data](./04-Protocol-Security.md#41-motivation-for-authenticated-encryption-with-associated-data)
- [4.2. Motivation for Using the Noise Protocol Framework](./04-Protocol-Security.md#42-motivation-for-using-the-noise-protocol-framework)
Expand Down