BUIP023: Miner block creation latency optimization Proposer: Andrew Stone Submitted: 2016-08-1 Status: draft
The mining of empty or pre-validation blocks are a mechanism that can naturally or artifically reduce the transaction capacity of the bitcoin network (see www.bitcoinunlimited.info/1txn). These blocks are mined when miners have received the headers of the next likely block but have not yet produced the next block template (a block template is a block that has not yet been mined -- its SHA256 does not meet the current difficulty).
During this interval, the miner must download and validate the full block and generate and communicate the next block template to the mining pool software. By optimizing any of these 4 steps, the average transaction committment capacity of the network is increased since the proportion of empty blocks is decreased.
This BUIP addresses the second two steps (generation and communication of the next block template) which will collectively be called "miner block creation". Two areas will be examined.
First the CreateNewBlock() function will be optimized -- in particular, it should be possible to skip validation of the transactions in the new block since they consist of transactions from the mempool. Transactions are validated when admitted to the mempool. But there may be more optimizations that become apparent as performance measurement tools are applied to the code.
Second, the getblocktemplate RPC call will be deprecated (it will still be available, but the new interface is preferred) in favor of a binary protocol that organizes data in a type,length,value (TLV) format. Endian will be that of the sending node, and specified in the binary protocol header. This will avoid endian swaps for the common case where the bitcoin daemon and pool server share the same CPU architecture. This protocol will be capable of "pushing" new blocks to the pool as soon as a block is found, rather then relying on the RPC (remote procedure call) polling techniques, and separate event channels currently used.
Extensions to this binary protocol that further optimise latency can be considered and implemented if they will be effective. For example, the pool software could provide the bitcoin daemon with the coinbase output, allowing the bitcoin daemon to create the full and exact binary block data required to pass to miners. This eliminates the step where the bitcoin pool parses the block, modifies it, and generates a new block.
The bitcoin daemon could also provide a miner interface directly, rather then passing the block to the pool and having the pool pass the block to miners. This would save a network "hop" and data translation time.
The "client side" (pool side) code will be provided as a shared library, and the open source pool software "ckpool" will be modified to show how a pool would take advantage of this new library/protocol.