Misc Versions in Node Codebase

suem · February 3, 2021, 3:39am

(This post is just for QoL of developers, to memorize those all kinds of versions which are quite important sometimes, especially for upgrading and backward compatibility related topics.)

ProtocolVersion

In core/src/global.rs, there’s a PROTOCOL_VERSION const which is the default “local” protocol version for this node.

So far 1,2,3 have been used in mwc.

1 for [ < 3.0.0] releases. with the 1st version of TxKernel structure (see below link).
2 for [3.0.0 ~ 4.1.0] releases. transaction kernels changed in v2 due to “variable size kernels”.
3 for [4.2.0 ~ current] releases. transaction input changed in v3 due to “commit only” inputs upgrading.

mwc will bump the protocol version to 1000 in next HF (v6.0.0) for NIT.

So far, peers with all protocol versions can talk to each other, including 1000.

In the future, for example, when we bump the protocol version to 2000, the peers with “1,2,3” versions will be kicked out. This will be helpful to throw the burden of those very old versions, to avoid those boring backward compatibility on quite old versions.

1st version of TxKernel structure: {features: 1byte, fee: 8bytes, lock_height: 8bytes, excess: 33, sig: 64}

github.com

mwcproject/mwc-node/blob/2.4.3/core/src/core/transaction.rs#L158-L180


pub struct TxKernel {
	/// Options for a kernel's structure or use
	pub features: KernelFeatures,
	/// Fee originally included in the transaction this proof is for.
	#[serde(with = "secp_ser::string_or_u64")]
	pub fee: u64,
	/// This kernel is not valid earlier than lock_height blocks
	/// The max lock_height of all *inputs* to this transaction
	#[serde(with = "secp_ser::string_or_u64")]
	pub lock_height: u64,
	/// Remainder of the sum of all transaction commitments. If the transaction
	/// is well formed, amounts components should sum to zero and the excess
	/// is hence a valid public key.
	#[serde(
		serialize_with = "secp_ser::as_hex",
		deserialize_with = "secp_ser::commitment_from_hex"
	)]
	pub excess: Commitment,
	/// The signature proving the excess is a valid public key, which signs
	/// the transaction fee.

This file has been truncated. show original

v2 TxKernel structure for “variable size kernels”:

github.com

mwcproject/mwc-node/blob/3.0.0/core/src/core/transaction.rs#L40-L55


pub enum KernelFeatures {
	/// Plain kernel (the default for Grin txs).
	Plain {
		/// Plain kernels have fees.
		fee: u64,
	},
	/// A coinbase kernel.
	Coinbase,
	/// A kernel with an explicit lock height (and fee).
	HeightLocked {
		/// Height locked kernels have fees.
		fee: u64,
		/// Height locked kernels have lock heights.
		lock_height: u64,
	},
}

Kernel note: (1) kernel hash always in serialization with v1 format! (2) kernel serialization for p2p broadcasting will handle the protocol versions.

github.com

mwcproject/mwc-node/blob/3.0.0/core/src/core/transaction.rs#L186-L200


impl Writeable for KernelFeatures {
	/// Protocol version may increment rapidly for other unrelated changes.
	/// So we match on ranges here and not specific version values.
	fn write<W: Writer>(&self, writer: &mut W) -> Result<(), ser::Error> {
		// Care must be exercised when writing for hashing purposes.
		// All kernels are hashed using original v1 serialization strategy.
		if writer.serialization_mode() == ser::SerializationMode::Hash {
			return self.write_v1(writer);
		}

		match writer.protocol_version().value() {
			0..=1 => self.write_v1(writer),
			2..=ProtocolVersion::MAX => self.write_v2(writer),
		}
	}

Input note: currently, all transactions in v3 will be converted into v2 when adding to pool (add_to_pool/convert_tx_v2), refer to https://github.com/mimblewimble/grin/pull/3419 for detail.

HeaderVersion

This is the block header version, which is normally used for hardfork only. So far, mwc has used 1 and 2, because we only had one hardfork till now.

github.com

mwcproject/mwc-node/blob/3.1.0/core/src/consensus.rs#L145


/// `40_000 / 47 = 851` (txs per block)
///
pub const MAX_BLOCK_WEIGHT: usize = 40_000;

/// Check whether the block version is valid at a given height
/// MWC doesn't want like grin change the algorithms for mining. So version is constant
pub fn header_version(height: u64) -> HeaderVersion {
	if height < get_c31_hard_fork_block_height() {
		HeaderVersion(1)
	} else {
		HeaderVersion(2)
	}
}

/// Check whether the block version is valid at a given height.
/// Currently we only use the default version. No hard forks planned.
pub fn valid_header_version(height: u64, version: HeaderVersion) -> bool {
	if height < get_c31_hard_fork_block_height() {
		version == HeaderVersion(1)
	} else {
		version == HeaderVersion(2)

mwc will bump this header version to 3 in next hardfork (v6.0.0) for [NIT].

Backend database version.

This is the database version which is useful to migrate the database on upgrading. Database version reused the same data as ProtocolVersion but obviously in different context.

So far, version 1,2,3 have been used.

2 is used for [3.0.0 ~ 4.1.0] release, transaction kernels changed in v2 due to “variable size kernels”.
3 is used for [4.2.0 ~ current], transaction input changed in v3 due to “commit only” inputs upgrading.

Note: the database migration can not jump versions. For example, migration from v2 to v3 is ok, but it’s not ok from v1 to v3. In another word, release 2.x.x can not upgrade to 4.x.x directly.

github.com

mwcproject/mwc-node/blob/3.0.0/chain/src/chain.rs#L1296-L1304


	fn migrate_db_v1_v2(&self) -> Result<(), Error> {
		let store_v1 = self.store.with_version(ProtocolVersion(1));
		let batch = store_v1.batch()?;
		for (_, block) in batch.blocks_iter()? {
			batch.migrate_block(&block, ProtocolVersion(2))?;
		}
		batch.commit()?;
		Ok(())
	}

github.com

mwcproject/mwc-node/blob/4.2.0/chain/src/chain.rs#L1420-L1428


	fn migrate_db_v2_v3(store: &ChainStore) -> Result<(), Error> {
		let store_v2 = store.with_version(ProtocolVersion(2));
		let batch = store_v2.batch()?;
		for (_, block) in batch.blocks_iter()? {
			batch.migrate_block(&block, ProtocolVersion(3))?;
		}
		batch.commit()?;
		Ok(())
	}

Backend MMR data file version.

This is the MMR data file version which is useful to migrate the MMR data on upgrading. MMR data file version also reused the same data as ProtocolVersion but obviously in different context.

Different MMR data file can use different version.

So far, only version 1 has been used for output/rangeproof/header/sync MMRs. For kernel MMR, version 1 and 2 have been used.

1 is used for [ … ~ 2.x.x] release.
2 is used for [3.0.0 ~ current] release, transaction kernels changed in v2 due to “variable size kernels”.

github.com

mwcproject/mwc-node/blob/3.0.0/chain/src/txhashset/txhashset.rs#L140


		root_dir: String,
		commit_index: Arc<ChainStore>,
		header: Option<&BlockHeader>,
	) -> Result<TxHashSet, Error> {
		let output_pmmr_h = PMMRHandle::new(
			&root_dir,
			TXHASHSET_SUBDIR,
			OUTPUT_SUBDIR,
			true,
			true,
			ProtocolVersion(1),
			header,
		)?;

		let rproof_pmmr_h = PMMRHandle::new(
			&root_dir,
			TXHASHSET_SUBDIR,
			RANGE_PROOF_SUBDIR,
			true,
			true,
			ProtocolVersion(1),

github.com

mwcproject/mwc-node/blob/3.0.0/chain/src/txhashset/txhashset.rs#L158


			true,
			true,
			ProtocolVersion(1),
			header,
		)?;

		// Initialize the bitmap accumulator from the current output PMMR.
		let bitmap_accumulator = TxHashSet::bitmap_accumulator(&output_pmmr_h)?;

		let mut maybe_kernel_handle: Option<PMMRHandle<TxKernel>> = None;
		let versions = vec![ProtocolVersion(2), ProtocolVersion(1)];
		for version in versions {
			let handle = PMMRHandle::new(
				&root_dir,
				TXHASHSET_SUBDIR,
				KERNEL_SUBDIR,
				false, // not prunable
				false, // variable size kernel data file
				version,
				None,
			)?;