Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: adapt StatsSigningTestingTool to work with Bytes wrapper for transactions #17144

Merged
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
63edb17
refactor: consumer for system transactions
mustafauzunn Dec 19, 2024
50669f7
refactor: adapt StatsSigningTestingTool to work with Bytes wrapper fo…
IvanKavaldzhiev Dec 20, 2024
fd7222c
Merge remote-tracking branch 'origin/develop' into 16871-adapt-StatsS…
IvanKavaldzhiev Dec 20, 2024
a7d6e22
style: restore headers
IvanKavaldzhiev Dec 20, 2024
b4dcec7
Merge remote-tracking branch 'origin/16703-refactor-consumer-system-t…
IvanKavaldzhiev Dec 20, 2024
8478fd9
Merge branch 'develop' into 16703-refactor-consumer-system-transactions
mustafauzunn Dec 20, 2024
2c6d151
Merge remote-tracking branch 'origin/16703-refactor-consumer-system-t…
IvanKavaldzhiev Dec 20, 2024
b67ca58
Merge remote-tracking branch 'origin/develop' into 16871-adapt-StatsS…
IvanKavaldzhiev Dec 20, 2024
2145caa
nit: resolve PR comments
IvanKavaldzhiev Dec 20, 2024
9315b1c
nit: resolve PR comments
IvanKavaldzhiev Dec 20, 2024
c26e80d
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Dec 26, 2024
a3d2c8b
nit: resolve PR comment and change system transaction differentiation
IvanKavaldzhiev Dec 27, 2024
7520540
nit: fix build
IvanKavaldzhiev Dec 27, 2024
c82d849
nit: resolve PR comment
IvanKavaldzhiev Dec 27, 2024
dcb186e
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Dec 27, 2024
3d56d76
Merge branch 'main' into 16871-adapt-StatsSigningTestingTool-to-work-…
rbarkerSL Jan 2, 2025
4e9ef8b
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 6, 2025
0cc6405
Merge remote-tracking branch 'origin/16871-adapt-StatsSigningTestingT…
IvanKavaldzhiev Jan 6, 2025
f3fd9b8
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 7, 2025
09bba4d
refactor: consume converted system transaction in callback instead of…
IvanKavaldzhiev Jan 7, 2025
734ed9a
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 7, 2025
90f12bc
style: spotless apply and remove unnecessary check
IvanKavaldzhiev Jan 7, 2025
c91aa49
style: spotless apply
IvanKavaldzhiev Jan 7, 2025
9877ca8
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 8, 2025
e8e5eae
nit: add final declarations and comments for elaborating business logic
IvanKavaldzhiev Jan 8, 2025
24c549a
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 9, 2025
8a809cf
refactor: simplify logic for distinguishing between system and applic…
IvanKavaldzhiev Jan 10, 2025
fd85b62
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 10, 2025
2ab240e
nit: resolve PR comment
IvanKavaldzhiev Jan 10, 2025
289fe7c
nit: fix PR comment
IvanKavaldzhiev Jan 10, 2025
32c9803
nit: resolve PR comment
IvanKavaldzhiev Jan 10, 2025
4e0a898
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 13, 2025
ebc6f9a
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 13, 2025
dfa1545
nit: resolve PR comments
IvanKavaldzhiev Jan 13, 2025
e80a544
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 15, 2025
3a1cead
style: spotless apply
IvanKavaldzhiev Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,8 +1,30 @@
// SPDX-License-Identifier: Apache-2.0
/*
* Copyright (C) 2024-2025 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

plugins { id("org.hiero.gradle.module.application") }

// Remove the following line to enable all 'javac' lint checks that we have turned on by default
// and then fix the reported issues.
tasks.withType<JavaCompile>().configureEach { options.compilerArgs.add("-Xlint:-cast") }

application.mainClass = "com.swirlds.demo.stats.signing.StatsSigningTestingToolMain"

testModuleInfo {
requires("org.assertj.core")
requires("org.junit.jupiter.api")
requires("org.mockito")
requires("org.junit.jupiter.params")
}
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@
import static com.swirlds.platform.test.fixtures.state.FakeStateLifecycles.FAKE_MERKLE_STATE_LIFECYCLES;
import static com.swirlds.platform.test.fixtures.state.FakeStateLifecycles.registerMerkleStateRootClassIds;

import com.hedera.hapi.platform.event.StateSignatureTransaction;
import com.hedera.pbj.runtime.io.buffer.Bytes;
import com.hedera.pbj.runtime.io.stream.WritableStreamingData;
import com.swirlds.common.constructable.ClassConstructorPair;
import com.swirlds.common.constructable.ConstructableRegistry;
import com.swirlds.common.constructable.ConstructableRegistryException;
Expand All @@ -52,6 +55,8 @@
import com.swirlds.platform.system.Platform;
import com.swirlds.platform.system.SwirldMain;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

Expand All @@ -63,23 +68,25 @@
public class StatsSigningTestingToolMain implements SwirldMain {
// the first four come from the parameters in the config.txt file

public static final byte SYSTEM_TRANSACTION_MARKER = 0;
private static final Logger logger = LogManager.getLogger(StatsSigningTestingToolMain.class);

static {
try {
logger.info(STARTUP.getMarker(), "Registering StatsSigningTestingToolState with ConstructableRegistry");
ConstructableRegistry constructableRegistry = ConstructableRegistry.getInstance();
final ConstructableRegistry constructableRegistry = ConstructableRegistry.getInstance();
constructableRegistry.registerConstructable(
new ClassConstructorPair(StatsSigningTestingToolState.class, () -> {
StatsSigningTestingToolState statsSigningTestingToolState = new StatsSigningTestingToolState(
FAKE_MERKLE_STATE_LIFECYCLES,
version -> new BasicSoftwareVersion(version.major()),
() -> null);
final StatsSigningTestingToolState statsSigningTestingToolState =
new StatsSigningTestingToolState(
FAKE_MERKLE_STATE_LIFECYCLES,
version -> new BasicSoftwareVersion(version.major()),
() -> null);

Check warning on line 84 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L83-L84

Added lines #L83 - L84 were not covered by tests
return statsSigningTestingToolState;
}));
registerMerkleStateRootClassIds();
logger.info(STARTUP.getMarker(), "StatsSigningTestingToolState is registered with ConstructableRegistry");
} catch (ConstructableRegistryException e) {
} catch (final ConstructableRegistryException e) {

Check warning on line 89 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L89

Added line #L89 was not covered by tests
logger.error(STARTUP.getMarker(), "Failed to register StatsSigningTestingToolState", e);
throw new RuntimeException(e);
}
Expand Down Expand Up @@ -257,7 +264,7 @@

if (transPerSecToCreate > -1) { // if not unlimited (-1 means unlimited)
// ramp up the TPS to the expected value
long elapsedTime = now / MILLISECONDS_TO_NANOSECONDS - rampUpStartTimeMilliSeconds;
final long elapsedTime = now / MILLISECONDS_TO_NANOSECONDS - rampUpStartTimeMilliSeconds;

Check warning on line 267 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L267

Added line #L267 was not covered by tests
double rampUpTPS = 0;
if (elapsedTime < TPS_RAMP_UP_WINDOW_MILLISECONDS) {
rampUpTPS = expectedTPS * elapsedTime / ((double) (TPS_RAMP_UP_WINDOW_MILLISECONDS));
Expand Down Expand Up @@ -313,4 +320,21 @@
public BasicSoftwareVersion getSoftwareVersion() {
return softwareVersion;
}

@Override
@NonNull
public Bytes encodeSystemTransaction(@NonNull final StateSignatureTransaction transaction) {
final var bytes = new ByteArrayOutputStream();
final var out = new WritableStreamingData(bytes);

// Add a marker to indicate the start of a system transaction. This is used
// to later differentiate between application transactions and system transactions.
out.writeByte(SYSTEM_TRANSACTION_MARKER);
try {
StateSignatureTransaction.PROTOBUF.write(transaction, out);
return Bytes.wrap(bytes.toByteArray());
} catch (final IOException e) {
throw new IllegalStateException("Failed to encode a system transaction.", e);

Check warning on line 337 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L336-L337

Added lines #L336 - L337 were not covered by tests
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,14 @@
*/

import static com.swirlds.common.utility.CommonUtils.hex;
import static com.swirlds.demo.stats.signing.StatsSigningTestingToolMain.SYSTEM_TRANSACTION_MARKER;
import static com.swirlds.logging.legacy.LogMarker.EXCEPTION;
import static com.swirlds.logging.legacy.LogMarker.TESTING_EXCEPTIONS_ACCEPTABLE_RECONNECT;

import com.hedera.hapi.node.base.SemanticVersion;
import com.hedera.hapi.platform.event.StateSignatureTransaction;
import com.hedera.pbj.runtime.ParseException;
import com.hedera.pbj.runtime.io.buffer.Bytes;
import com.swirlds.common.constructable.ConstructableIgnored;
import com.swirlds.common.crypto.CryptographyHolder;
import com.swirlds.common.crypto.TransactionSignature;
Expand Down Expand Up @@ -115,13 +118,29 @@
@Override
public void preHandle(
@NonNull final Event event,
@NonNull final Consumer<ScopedSystemTransaction<StateSignatureTransaction>> stateSignatureTransaction) {
@NonNull
final Consumer<ScopedSystemTransaction<StateSignatureTransaction>>
stateSignatureTransactionCallback) {
final SttTransactionPool sttTransactionPool = transactionPoolSupplier.get();
if (sttTransactionPool != null) {
event.forEachTransaction(transaction -> {
// We are not interested in pre-handling any system transactions, as they are
// specific for the platform only.We also don't want to consume deprecated
// EventTransaction.STATE_SIGNATURE_TRANSACTION system transactions in the
// callback,since it's intended to be used only for the new form of encoded system
// transactions in Bytes.Thus, we can directly skip the current
// iteration, if it processes a deprecated system transaction with the
// EventTransaction.STATE_SIGNATURE_TRANSACTION type.
if (transaction.isSystem()) {
return;
}

// We should consume in the callback the new form of system transactions in Bytes
if (areTransactionBytesSystemOnes(transaction)) {
consumeSystemTransaction(transaction, event, stateSignatureTransactionCallback);
return;
}

final TransactionSignature transactionSignature =
sttTransactionPool.expandSignatures(transaction.getApplicationTransaction());
if (transactionSignature != null) {
Expand All @@ -139,15 +158,33 @@
public void handleConsensusRound(
@NonNull final Round round,
@NonNull final PlatformStateModifier platformState,
@NonNull final Consumer<ScopedSystemTransaction<StateSignatureTransaction>> stateSignatureTransaction) {
@NonNull
final Consumer<ScopedSystemTransaction<StateSignatureTransaction>>
stateSignatureTransactionCallback) {
throwIfImmutable();
round.forEachTransaction(this::handleTransaction);

round.forEachEventTransaction((event, transaction) -> {
// We are not interested in handling any system transactions, as they are
// specific for the platform only.We also don't want to consume deprecated
// EventTransaction.STATE_SIGNATURE_TRANSACTION system transactions in the
// callback,since it's intended to be used only for the new form of encoded system
// transactions in Bytes.Thus, we can directly skip the current
// iteration, if it processes a deprecated system transaction with the
// EventTransaction.STATE_SIGNATURE_TRANSACTION type.
if (transaction.isSystem()) {
return;
}

// We should consume in the callback the new form of system transactions in Bytes
if (areTransactionBytesSystemOnes(transaction)) {
consumeSystemTransaction(transaction, event, stateSignatureTransactionCallback);
} else {
handleTransaction(transaction);
}
});
}

private void handleTransaction(final ConsensusTransaction trans) {
if (trans.isSystem()) {
return;
}
final TransactionSignature s = trans.getMetadata();

if (s != null && validateSignature(s, trans) && s.getSignatureStatus() != VerificationStatus.VALID) {
Expand Down Expand Up @@ -175,6 +212,40 @@
maybeDelay();
}

/**
* Checks if the transaction bytes are system ones.
*
* @param transaction the transaction to check
* @return true if the transaction bytes are system ones, false otherwise
*/
private boolean areTransactionBytesSystemOnes(@NonNull final Transaction transaction) {
final var transactionBytes = transaction.getApplicationTransaction();

if (transactionBytes.length() == 0) {
return false;

Check warning on line 225 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java#L225

Added line #L225 was not covered by tests
}

return transactionBytes.getByte(0) == SYSTEM_TRANSACTION_MARKER;
}

private void consumeSystemTransaction(
@NonNull final Transaction transaction,
@NonNull final Event event,
@NonNull
final Consumer<ScopedSystemTransaction<StateSignatureTransaction>>
stateSignatureTransactionCallback) {
try {
final Bytes transactionBytes = transaction.getApplicationTransaction();
final Bytes strippedSystemTransactionBytes = transactionBytes.slice(1, transactionBytes.length() - 1);
final StateSignatureTransaction stateSignatureTransaction =
StateSignatureTransaction.PROTOBUF.parse(strippedSystemTransactionBytes);
stateSignatureTransactionCallback.accept(new ScopedSystemTransaction<>(
event.getCreatorId(), event.getSoftwareVersion(), stateSignatureTransaction));
} catch (final ParseException e) {
logger.error("Failed to parse StateSignatureTransaction", e);

Check warning on line 245 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java#L244-L245

Added lines #L244 - L245 were not covered by tests
}
}

private void maybeDelay() {
if (SYNTHETIC_HANDLE_TIME) {
final long start = System.nanoTime();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2022-2024 Hedera Hashgraph, LLC
* Copyright (C) 2022-2025 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -165,7 +165,7 @@
final byte[] sig = exSig.getSignature();

transactions[i] = TransactionCodec.encode(alg, transactionId, sig, data);
} catch (SignatureException e) {
} catch (final SignatureException e) {

Check warning on line 168 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/SttTransactionPool.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/SttTransactionPool.java#L168

Added line #L168 was not covered by tests
// If we are unable to sign the transaction then log the failure and create an unsigned transaction
logger.error(
EXCEPTION.getMarker(),
Expand Down Expand Up @@ -202,7 +202,7 @@
if (algorithm.isAvailable()) {
activeAlgorithms.put(algorithm.getId(), algorithm);
}
} catch (Exception ex) {
} catch (final Exception ex) {

Check warning on line 205 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/SttTransactionPool.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/SttTransactionPool.java#L205

Added line #L205 was not covered by tests
logger.error(
EXCEPTION.getMarker(),
"Failed to Activate Signing Algorithm [ id = {}, class = {} ]",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2022-2024 Hedera Hashgraph, LLC
* Copyright (C) 2022-2025 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -30,14 +30,15 @@
* The core transaction encoder and decoder implementation. See below for the binary transaction format specification.
* <p>
* Transaction Structure:
* ------------------------------------------------------------------------------------------------------------
* | 8 bytes | 1 byte | 1 byte | 4 bytes | pklen bytes | 4 bytes | siglen bytes | 4 bytes | datalen bytes |
* |---------|--------|----------|-----------|-------------|---------|--------------|---------|---------------|
* | id | signed | sigAlgId | pklen | pk | siglen | sig | datalen | data |
* ------------------------------------------------------------------------------------------------------------
* ---------------------------------------------------------------------------------------------------------------------
* | 1 byte | 8 bytes | 1 byte | 1 byte | 4 bytes | pklen bytes | 4 bytes | siglen bytes | 4 bytes | datalen bytes |
* |--------|---------|--------|----------|-----------|-------------|---------|--------------|---------|---------------|
* | marker | id | signed | sigAlgId | pklen | pk | siglen | sig | datalen | data |
* ---------------------------------------------------------------------------------------------------------------------
*/
final class TransactionCodec {

public static final byte APPLICATION_TRANSACTION_MARKER = 1;
public static final byte NO_ALGORITHM_PRESENT = -1;

private static final int PREAMBLE_SIZE = Long.BYTES + (Byte.BYTES * 2);
Expand Down Expand Up @@ -78,11 +79,14 @@ public static int overheadSize(final SigningAlgorithm algorithm) {

public static byte[] encode(
final SigningAlgorithm algorithm, final long transactionId, final byte[] signature, final byte[] data) {
final ByteBuffer buffer = ByteBuffer.allocate(bufferSize(algorithm, (data != null) ? data.length : 0));
final ByteBuffer buffer = ByteBuffer.allocate(1 + bufferSize(algorithm, (data != null) ? data.length : 0));
final boolean signed =
algorithm != null && algorithm.isAvailable() && signature != null && signature.length > 0;

buffer.putLong(transactionId)
// Add a marker byte in the very beginning to indicate the start of an application transaction. This is used
// to later differentiate between application transactions and system transactions.
buffer.put(APPLICATION_TRANSACTION_MARKER)
.putLong(transactionId)
.put((signed) ? (byte) 1 : 0)
.put((signed) ? algorithm.getId() : NO_ALGORITHM_PRESENT)
.putInt((signed) ? algorithm.getPublicKeyLength() : 0);
Expand Down
Loading