Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark #1487

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/benchmarks/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ sidebar_position: 1

# Benchmarks & Limitations

This section provides up-to-date data about the known limitations Hydra Head on-chain protocol. Cardano transactions (and blocks) have limits on the transaction size, execution cost, number of inputs and outputs which are dependent on the network parameters and which impact the capabilities of the Head protocol: How many parties can take part in a Head, how many UTxO can be committed to the Head, how many can be fan-out... As the on-chain scripts and transactions mature and are optimised, and the underlying Cardano chain evolves with increased parameters and more efficient scripts execution, those limits will change.
This section provides up-to-date data regarding the known limitations of the Hydra Head on-chain protocol. Cardano transactions and blocks are subject to constraints on transaction size, execution cost, and the number of inputs and outputs. These constraints are determined by network parameters and significantly influence the capabilities of the Head protocol, such as the maximum number of parties that can participate, the amount of UTXOs that can be committed, and the extent to which these UTXOs can be fanned out. As on-chain scripts and transactions are further optimized, and as the underlying Cardano blockchain evolves with expanded parameters and enhanced script execution efficiency, these limitations are expected to evolve.

The data in this section is _generated_ through Hydra's [Continuous Integration](https://github.com/input-output-hk/hydra/actions/workflows/ci-nix.yaml) process, ensuring that it accurately reflects the current state of the code.

The data provided in those pages is _generated_ by Hydra's [Continuous Integration](https://github.com/input-output-hk/hydra/actions/workflows/ci-nix.yaml) process and thus guaranteed to reflect the current state of the code.

```mdx-code-block
import DocCardList from '@theme/DocCardList';
import {useDocsSidebar} from '@docusaurus/theme-common/internal';
<DocCardList items={useDocsSidebar().items.filter(({ docId }) => docId != "index").map(item => {if (item.label == "tests") item.label = "Test Results" ; return item})}/>
```
```
3 changes: 1 addition & 2 deletions docs/benchmarks/ledger.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Ledger micro-benchmarks

These micro-benchmarks details the time for various key aspects of applying transactions in the Cardano ledger in a Hydra Head.
The absolute values are relatively meaningless, what's important is the relative cost of the various operations involved in the end-to-end process of submitting a new transaction to a Hydra Head and applying it to the ledger.
This section details the time taken for various key processes involved in applying transactions in the Cardano ledger within a Hydra Head. The absolute values are relatively meaningless instead, the focus is on understanding the relative costs of the different operations that make up the end-to-end process of submitting and applying a new transaction to a Hydra Head.

* :chart_with_upwards_trend: [Ledger Benchmarks](pathname:///ledger-bench.html)
57 changes: 18 additions & 39 deletions docs/benchmarks/profiling.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,22 @@ sidebar_position: 10

# Profiling Hydra scripts

This is a quick tutorial how to profile Hydra scripts and is intended for
contributors to the `hydra-node`.
This tutorial explains how to profile Hydra scripts and is intended for contributors to the `hydra-node`.

On every PR and also for the latest `master`, we do compute typical transaction
costs in size, memory and cpu usage of the Hydra protocol transactions on
Cardano. The latest results can be seen
[here](https://hydra.family/head-protocol/benchmarks/transaction-cost/).
## Overview

Such benchmarks provide a great overview of what "fits" into a given transaction
in terms of maximum transaction size, percent of maximum memory and cpu budget.
For the latter, we evaluates _all_ the scripts which will run in a given
transaction.
For every PR and the latest `master` branch, we compute typical transaction costs in terms of size, memory, and CPU usage of the Hydra protocol transactions on Cardano. You can view the latest results [here](https://hydra.family/head-protocol/benchmarks/transaction-cost/).

To get more detailed insights of _what exactly_ is resulting in excessive memory
or cpu usage, we need to profile the scripts as they are validating a
transaction.
Such benchmarks provide a comprehensive overview of the constraints for a given transaction, including maximum transaction size, and percent of maximum memory and CPU budget. For a detailed assessment, we analyze _all_ scripts that run within a given transaction.

To gain detailed insights into _what exactly_ results in excessive memory or CPU usage, we need to profile the scripts as they validate a transaction.

Follow the instructions provided by the [`plutus`](https://github.com/input-output-hk/plutus) project [here](https://plutus.readthedocs.io/en/latest/howtos/profiling-scripts.html), adapted for the `hydra` codebase.

This guide follows the instructions provided upstream by the
[`plutus`](https://github.com/input-output-hk/plutus) project
[here](https://plutus.readthedocs.io/en/latest/howtos/profiling-scripts.html),
but points out how this can be done in the `hydra` code base.

## Isolate a transaction to profile

To do any measurements, we need to isolate the actual cardano `Tx` which is to
be profiled. For example, let's investigate what the `collectCom` transaction
First, isolate the specific Cardano `TX` you want to profile. For example, let's investigate what the `collectCom` transaction
for `5` parties in the `tx-cost` benchmark is spending most time and memory on.

The benchmark computes many transactions with growing number of participants in `computeCollectComCost`:
Expand All @@ -44,28 +34,22 @@ computeCollectComCost =
-- [...]
```

The `tx` here would be the transaction we want to profile, so we can "isolate"
the transaction for `5` parties by changing the body of this function `maybe []
pure <$> compute 5`.
Here, isolate the transaction for `5` parties by altering the function to `maybe [] pure <$> compute 5`.

## Compiling a script for profiling

The `collectCom` transaction runs `vCommit` and `vHead` validator scripts, so we
need to add
The `collectCom` transaction utilizes the `vCommit` and `vHead` validator scripts. To enable profiling, add the following directive to the modules [`Hydra.Contract.Commit`](/haddock/hydra-plutus/Hydra-Contract-Commit.html) and [`Hydra.Contract.Head`](/haddock/hydra-plutus/Hydra-Contract-Head.html):

```
{-# OPTIONS_GHC -fplugin-opt PlutusTx.Plugin:profile-all #-}
```

to the corresponding modules [`Hydra.Contract.Commit`](/haddock/hydra-plutus/Hydra-Contract-Commit.html) and
[`Hydra.Contract.Head`](/haddock/hydra-plutus/Hydra-Contract-Head.html).

## Acquiring an executable script

This can be now achieved using
This can be achieved using
[`prepareTxScripts`](/haddock/hydra-node/Hydra-Ledger-Cardano-Evaluate.html#v:prepareTxScripts).
We can use this function to acquire and dump the fully applied scripts from the
transaction onto disk.
to acquire and save the fully applied scripts from the transaction onto disk:

```haskell
-- [...]
Expand All @@ -86,8 +70,7 @@ should make it clear at the latest.

## Running the script & analysing the results

The tools for this step can be acquired using nix (or alternatively compile the
upstream projects and use your distribution's package manager):
To perform this step, use the following tools available through Nix:

```
nix shell nixpkgs#flamegraph github:input-output-hk/plutus#x86_64-linux.plutus.library.plutus-project-924.hsPkgs.plutus-core.components.exes.traceToStacks github:input-output-hk/plutus#x86_64-linux.plutus.library.plutus-project-924.hsPkgs.plutus-core.components.exes.uplc
Expand All @@ -100,23 +83,19 @@ input format as `prepareTxScripts` retains the original name annotations.
uplc evaluate -t -i scripts-1.flat --if flat-namedDeBruijn --trace-mode LogsWithBudgets -o logs
```

This should have produced `logs` file. If not, double check that you have
compiled the script with profiling options via the language pragma above.
Check for a `logs` file output. If not present, ensure the script was compiled with profiling enabled as specified.

At this stage, we can inspect the logs or produce the flamegraph SVGs exactly as
described in the original tutorial:
Finally, you can inspect the logs or generate flamegraph SVGs as outlined in the original tutorial:

```
cat logs | traceToStacks | flamegraph.pl > cpu.svg
cat logs | traceToStacks --column 2 | flamegraph.pl > mem.svg
```

Here, for example the memory profile of a `5` party `collectCom` at the time of
writing:
Here's an example of a memory profile for a `5` party `collectCom`:

![](profile-mem.svg)

:::tip
Open the SVG in a browser to search and drill-down through the profile
interactively.
Open the SVG in a browser to interactively search and drill down through the profile.
:::