Skip to content

Latest commit

 

History

History
205 lines (157 loc) · 7.48 KB

CALIPER.md

File metadata and controls

205 lines (157 loc) · 7.48 KB

Benchmarking the performance using Hyperledger Caliper

This document introduces how to use Hyperledger Caliper to benchmark the performance of the Hyperledger Fabric environment created with test-network-k8s.

Fabric adapter manual of Hyperledger Caliper v0.6.0 only describes how to connect to test-network. Furthermore, these chaincodes need to be executed as services to run in a K8s environment, but this is not supported by default, requiring customization. So we will explain how to benchmark the performance of the Kubernetes test network using Hyperledger Caliper and Asset Transfer Basic chaincode, which is most basic in current sample chaincodes.

The following documentation assumes that test-network-k8s and Hyperledger Caliper v0.6.0 are located on the same host.

Setting of test-network-k8s side

As described in the README of test-network-k8s, launch the network, create a channel, and deploy and invoke the basic-asset-transfer smart contract:

./network kind

./network cluster init

./network up

./network channel create

./network chaincode deploy asset-transfer-basic ../asset-transfer-basic/chaincode-java

./network chaincode invoke asset-transfer-basic '{"Args":["InitLedger"]}'

REST API will not be used in the procedure described below, but the connection profile will be generated by launching it:

./network rest-easy

Setting of Hyperledger Caliper side

Following Install manual of Hyperledger Caliper v0.6.0, install Hyperledger Caliper from npm:

git clone https://github.com/hyperledger/caliper-benchmarks.git
cd caliper-benchmarks
npm install --only=prod @hyperledger/[email protected]
npx caliper bind --caliper-bind-sut fabric:fabric-gateway

Copy the connection profile created in test-network-k8s environment to Caliper environment.

cp <fabric-samples install dir>/test-network-k8s/build/fabric-rest-sample-config/HLF_CONNECTION_PROFILE_ORG1 networks/fabric/connection-profile.json

Replace *.test-network.svc.cluster.local with *.localho.st in "url" and "grpcOptions" section of connection-profile.json as below:

    "peers": {
        "org1-peers": {
            "url": "grpcs://org1-peer1.localho.st:443",
            "tlsCACerts": {
                "pem": <contents of pem file>"
            },
            "grpcOptions": {
                "ssl-target-name-override": "org1-peer1.localho.st",
                "hostnameOverride": "org1-peer1.localho.st"
            }
        }
    },

*.localho.st is wildcard domain defined for accessing K8s pod from external network via Nginx ingress controller. Please see Working with Kubernetes document for details.

Open networks/fabric/test-network.yaml and edit it as below:

name: Caliper Benchmarks
version: "2.0.0"

caliper:
  blockchain: fabric

channels:
  # channelName of mychannel matches the name of the channel created by test network
  - channelName: mychannel
    # the chaincodeIDs of all the fabric chaincodes in caliper-benchmarks
    contracts:
    - id: fabcar
    - id: fixed-asset
    - id: marbles
    - id: simple
    - id: smallbank
    - id: asset-transfer-basic

organizations:
  - mspid: Org1MSP
    # Identities come from cryptogen created material for test-network
    identities:
      certificates:
      - name: 'User1'
        clientPrivateKey:
          path: '<fabric-samples install absolute dir>/test-network-k8s/build/enrollments/org1/users/org1admin/msp/keystore/key.pem'

        clientSignedCert:
          path: '<fabric-samples install absolute dir>/test-network-k8s/build/enrollments/org1/users/org1admin/msp/signcerts/cert.pem'

    connectionProfile:
      path: 'networks/fabric/connection-profile.json'
      discover: true

Currently, sample code for running a performance benchmark targeting asset-transfer-basic is not published on the Caliper repository. Therefore, use the sample code published in Caliper's user manual to build a test workload.

As shown in "Step 3" of the above document, create workload/readAsset.js file and edit it as below:

'use strict';

const { WorkloadModuleBase } = require('@hyperledger/caliper-core');

class MyWorkload extends WorkloadModuleBase {
    constructor() {
        super();
    }

    async initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext) {
        await super.initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext);

        for (let i=0; i<this.roundArguments.assets; i++) {
            const assetID = `${this.workerIndex}_${i}`;
            console.log(`Worker ${this.workerIndex}: Creating asset ${assetID}`);
            const request = {
                contractId: this.roundArguments.contractId,
                contractFunction: 'CreateAsset',
                invokerIdentity: 'User1',
                contractArguments: [assetID,'blue','20','penguin','500'],
                readOnly: false
            };

            await this.sutAdapter.sendRequests(request);
        }
    }

    async submitTransaction() {
        const randomId = Math.floor(Math.random()*this.roundArguments.assets);
        const myArgs = {
            contractId: this.roundArguments.contractId,
            contractFunction: 'ReadAsset',
            invokerIdentity: 'User1',
            contractArguments: [`${this.workerIndex}_${randomId}`],
            readOnly: true
        };

        await this.sutAdapter.sendRequests(myArgs);
    }

    async cleanupWorkloadModule() {
        for (let i=0; i<this.roundArguments.assets; i++) {
            const assetID = `${this.workerIndex}_${i}`;
            console.log(`Worker ${this.workerIndex}: Deleting asset ${assetID}`);
            const request = {
                contractId: this.roundArguments.contractId,
                contractFunction: 'DeleteAsset',
                invokerIdentity: 'User1',
                contractArguments: [assetID],
                readOnly: false
            };

            await this.sutAdapter.sendRequests(request);
        }
    }
}

function createWorkloadModule() {
    return new MyWorkload();
}

module.exports.createWorkloadModule = createWorkloadModule;

As shown in "Step 4" of the above document, create benchmarks/myAssetBenchmark.yaml file and edit it as below:

test:
    name: basic-contract-benchmark
    description: test benchmark
    workers:
      number: 2
    rounds:
      - label: readAsset
        description: Read asset benchmark
        txDuration: 30
        rateControl:
          type: fixed-load
          opts:
            transactionLoad: 2
        workload:
          module: workload/readAsset.js
          arguments:
            assets: 10
            contractId: asset-transfer-basic

After creating the above files, you can run a performance benchmark with the following command, which will create some assets and measure the time it takes to reference the assets.

npx caliper launch manager --caliper-workspace . --caliper-benchconfig benchmarks/myAssetBenchmark.yaml --caliper-networkconfig networks/fabric/test-network.yaml