Skip to content

Metrics Definition

Manpreet edited this page Jul 22, 2023 · 15 revisions

The metrics definition input file is used to define all the metrics that the data generator should generate for the specified resources. Each metric type to be generated is defined in this file along with the following two fields which are applicable for all the metrics:

  • payloadFrequencySeconds: Number of seconds after which the metrics payloads are posted. You can override this at each metric level.
  • payloadCount: Number of payloads to be posted. You can override this at each metric level.

Love examples of the definition file are available here. The definition for each metric has the following fields:

Field Is Mandatory? Description
name Yes Name of the metric; is sent as it is in the OTel packet.
unit Yes Unit of the metric; is sent as it is in the OTel packet.
otelType Yes OpenTelemetry metric type. Supported types are: summary, sum, gauge and histogram.
valueFunction Yes Arithmetic expression to be evaluated to obtain the metric value. See Value Function Expressions.
reportingResources No List of resource names (which are defined in resource model definition) that should report this metric. Valid value must be provided for reportingResources or filteredReportingResources field.
filteredReportingResources No A map of resource names and a set of corresponding attribute value filters resulting in a list of resources that should report this metric. See this for example. If a resource name is mentioned in both reportingResources and filteredReportingResources field, the reportingResources one will take precedence, i.e. all resources of that type will be considered. Multiple filters, if specified for a resource, are treated as a combined filter, meaning only the resources matching all the filter conditions will be selected. Any invalid filters (not having exactly one '=') will simply be ignored and in case a resource type is specified with no valid filters, or, effectively an empty set of filters, all resources of that type will be selected.
aggregationTemporality Yes, only valid if otelType = sum Aggregation temporality of this sum metric. Valid values: delta, cumulative.
quantiles Yes, only required for otelType = summary Quantiles/Percentiles for which the corresponding values have to be sent for the summary type of metrics.
bounds Yes, only required for otelType = histogram Range of values that define buckets. For example bounds set to [5, 10] defines 3 buckets with ranges [-infinity, 5], [6, 10] and [11, +infinity].
isMonotonic No, only valid if otelType = sum Indicates if the value of this sum metric always increases.
isDouble No Double values are generated for the metric if set to true. Values for the summary type of metric are always double.
attributes No List of key-value pairs in the format <ATTRIBUTE_NAME, VALUE_EXPRESSION> where each pair defines an attribute to be sent for this span and an expression that should be evaluated to generate the value for that attribute. See Attribute Value Expressions.
copyResourceAttributes No A set of attribute names defined in the reporting resource(s) which should be copied to the metric attributes. If the attribute is not available in the reporting resource, it will be sent with an empty string value.
payloadFrequencySeconds No Optional override for this metric to specify the number of seconds after which this metric is posted. If not specified, the global payloadFrequencySeconds is used.
payloadCount No Optional override for this metric to specify the number of times this metric is posted. If not specified, the global payloadCount is used.

Each metric defined is posted as part of each payload for all resources which are of type reportingResources. For example, if payloadCount is 10, a metric is defined on the pod type of resource and the number of pods is set to 25. For each of the 10 payloads sent, the metric is sent for all 25 pods.


Value Function Expressions

The value function expression specified for each metric must be a defined using custom expression method that should return a double value when evaluated using the Jakarta Expression Language (JEL) processor.
It is important to note here that all the resources reporting the metric get the same metric value in the same payload. The valueFunction is evaluated for the metric and the same value is provided to all the resources. The metric generator threads are invoked every X number of seconds and that metric is reported in each invocation, for all of the resources specified in the definition. The next time that the thread is invoked, the value is updated based on the expression provided in the valueFunction. This is done to optimise functional test scenarios and the generator performance.
The following table lists functions that the test data generator tool provides which should be used in value functions:

Method Name Description
arithmeticSequence(double START_VALUE, double ADD_VALUE, String EXPRESSION) Gets the 'N'th value in an arithmetic progression where N = current payload count. The value thus obtained can be further modified with a basic arithmetic expression.
Example: arithmeticSequence(3, 2, "*5")
First payload → (3 + (20)) * 5 = 15
Second payload → (3 + (2
1)) * 5 = 25
Third payload → (3 + (2*2)) * 5 = 35 and so on.
geometricSequence(double START_VALUE, double COMMON_RATIO, String EXPRESSION) Gets the 'N'th value in a geometric progression where N = current payload count. The value thus obtained can be further modified with a basic arithmetic expression.
Example: geometricSequence(3, 2, "*5")
First payload → (3 * (2^0)) * 5 = 15
Second payload → (3 * (2^1)) * 5 = 30
Third payload → (3 * (2^2)) * 5 = 60 and so on.
exponentialSequence(double START_VALUE, double GROWTH_FACTOR, String EXPRESSION) Gets the 'N'th value in an exponential progression where N = current payload count. The value thus obtained can be further modified with a basic arithmetic expression.
Example: exponentialSequence(3, 2, "*5")
First payload → (3 * e^(20)) * 5 = 15
Second payload → (3 * e^(2
1)) * 5 = 110.83
Third payload → (3 * e^(2*2)) * 5 = 818.97 and so on.
logarithmicSequence(double START_VALUE, double GROWTH_FACTOR, String EXPRESSION) Gets the 'N'th value in a logarithmic progression where N = current payload count + 1 since log(0) leads to weird things. The value thus obtained can be further modified with a basic arithmetic expression.
Example: logarithmicSequence(3, 2, "*5")
First payload → (3 + 2log(1)) * 5 = 15
Second payload → (3 + 2
log(2)) * 5 = 21.93
Third payload → (3 + 2*log(3)) * 5 = 25.98 and so on.
absoluteSineSequence(String EXPRESSION) Gets the value of Math.abs(Math.sin(N)) where N = current payload count. The value obtainer can be further modified with a basic arithmetic expression.
Example: absoluteSineSequence("*7000")
First payload → Math.abs(Math.sin(0)) * 7000 = 0
Second payload → Math.abs(Math.sin(1)) * 7000 = 5890.3
Third payload → Math.abs(Math.sin(2)) * 7000 = 6365.08
absoluteCosineSequence(String EXPRESSION) Gets the value of Math.abs(Math.cos(N)) where N = current payload count. The value obtained can be further modified with a basic arithmetic expression.
Example: absoluteCosineSequence("*50")
First payload → Math.abs(Math.cos(0)) * 50 = 50
Second payload → Math.abs(Math.cos(1)) * 50 = 27.02
Third payload → Math.abs(Math.cos(2)) * 50 = 20.81
absoluteTangentSequence(String EXPRESSION) Gets the value of Math.abs(Math.tan(N)) where N = current payload count. The value obtainer can be further modified with a basic arithmetic expression.
Example: absoluteTangentSequence("*20+3")
First payload → Math.abs(Math.tan(0)) * 20 + 3 = 3
Second payload → Math.abs(Math.tan(1)) * 20 + 3 = 34.15
Third payload → Math.abs(Math.tan(2)) * 20 + 3 = 46.7
random(double MIN, double MAX, String EXPRESSION) Gets a random value between the specified MIN (inclusive) and MAX (exclusive) double values. The value thus obtained can be further modified with a basic arithmetic expression.
controlledRandom(double MIN, double MAX, String EXPRESSION) A controlled variant of random function which generates a random value between the specified range for the first term but all the subsequent values generated in the sequence are always in between the -20% to +20% range of their previous value.

For the gauge and sum metric types, the above functions work but each summary metric needs a number of values whose sum, count and quantiles are sent. To accommodate this, all of the above functions have a *Summary variant, for example:

  • arithmeticSequenceSummary(double START_VALUE, double ADD_VALUE, String EXPRESSION, int COUNT)
  • controlledRandomSummary(double MIN, double MAX, String EXPRESSION, int COUNT)

In such cases, the evaluation happens COUNT number of times to obtain the required values. For the deterministic sequence methods, these values follow the same formula and the N for the current payload count also remains the same. For example:
geometricSequenceSummary(3, 2, "*5", 5)
First payload values → [15, 30, 60, 120, 240]
Second payload values → [30, 60, 120, 240, 480]
Third payload values → [60, 120, 240, 480, 960] and so on.

If for any reason, a single value function is used for a summary metric, the same value is used with COUNT = 5 to obtain the summary statistics.

It is possible to implement and provide your own expression methods for generating metric values. See User Defined Expressions.