Skip to content

Add HA monitoring #1196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 15, 2025
+176 −178
Merged

Add HA monitoring #1196

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
758f0aa
Added HA metrics part
as51340 Mar 14, 2025
3237f6b
Update monitoring
as51340 Apr 4, 2025
22f13e7
Update HA metrics
as51340 Apr 4, 2025
2f4cdd0
Update pages/clustering/high-availability.mdx
matea16 Apr 14, 2025
387b3a3
Merge branch 'memgraph-3-2' into ha-monitoring
matea16 Apr 14, 2025
7d123be
Merge branch 'memgraph-3-2' into ha-monitoring
matea16 Apr 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion pages/clustering/high-availability.mdx
View file
Original file line number Diff line number Diff line change
@@ -52,6 +52,13 @@ since Raft, as a consensus algorithm, works by forming a majority in the decisio

</Callout>

## Observability

Monitoring the cluster state is very important and tracking various metrics can provide us with a valuable information. Currently, we track
metrics which reveal us p50, p90 and p99 latencies of RPC messages, the duration of recovery process and the time needed to react to changes
in the cluster. We are also counting the number of different RPC messages exchanged and the number of failed requests since this can give
us infomation about parts of the cluster that need further care. You can see the full list of metrics [here](/database-management/monitoring#system-metrics).

<Callout type="info">

When deploying coordinators to servers, you can use the instance of almost any size. Instances of 4GiB or 8GiB will suffice since coordinators'
@@ -61,7 +68,6 @@ but from the availability perspective, it is better to separate them physically.
</Callout>



## Bolt+routing

Directly connecting to the MAIN instance isn't preferred in the HA cluster since the MAIN instance changes due to various failures. Because of that, users
346 changes: 169 additions & 177 deletions pages/database-management/monitoring.mdx
View file
Original file line number Diff line number Diff line change
@@ -12,8 +12,9 @@ Monitoring applications and databases is essential to track the load and resourc
Memgraph currently supports:
1. **Real-time logs tracking via WebSocket server**: Log tracking is helpful for debugging purposes and monitoring the database operations.
2. **Metrics tracking via HTTP server (Enterprise Edition)**: In the Enterprise edition, besides log tracking, Memgraph allows tracking information about
transactions, query latencies, snapshot recovery latencies, triggers, Bolt
messages, indexes, streams, memory, operators and sesions, all by using an HTTP server.
transactions, query latencies, types of queries executed, snapshot recovery latencies, triggers, TTL data, Bolt
messages, indexes, streams, memory, operators and sesions. There are also many metrics which are trying to describe the state of high availability. These include metrics which
describe latencies of RPC messages, time needed to perform a failover and counters for events occurring throughout the lifetime of HA Memgraph.


## Real-time logs tracking via WebSocket server
@@ -184,7 +185,7 @@ privilege](/database-management/authentication-and-authorization/role-based-acce
## Metrics tracking via HTTP server (Enterprise Edition)

In the Enterprise Edition, Memgraph allows tracking information about
transactions, query latencies, snapshot recovery latencies, triggers, Bolt
high availability, transactions, query latencies, snapshot recovery latencies, triggers, Bolt
messages, indexes, streams, memory, operators and sesions, all by using using an HTTP server.

To retrieve data from the HTTP server, [enter a valid Memgraph
@@ -245,28 +246,28 @@ three different types:

#### General metrics

| Name | Type | Description |
| -------------- | ---------------- | ----------------------------------------------------------- |
| vertex_count | Gauge (uint64_t) | Number of nodes stored in the system. |
| edge_count | Gauge (uint64_t) | Number of relationships stored in the system. |
| average_degree | Gauge (double) | Average number of relationships of a single node. |
| memory_usage | Gauge (uint64_t) | Amount of RAM used reported by the OS (in bytes). |
| disk_usage | Gauge (uint64_t) | Amount of disk space used by the [data directory](/fundamentals/data-durability) (in bytes). |
| peak_memory_usage | Gauge (uint64_t) | Peak RAM memory usage in the system during the whole run (in bytes). |
| Name | Type | Description |
| ------------------------ | --------- | -------------------------------------------------------------------------------------------- |
| average_degree | Counter | Average number of relationships of a single node. |
| disk_usage | Gauge | Amount of disk space used by the [data directory](/fundamentals/data-durability) (in bytes). |
| edge_count | Counter | Number of relationships stored in the system. |
| memory_usage | Gauge | Amount of RAM used reported by the OS (in bytes). |
| peak_memory_usage | Gauge | Peak amount of RAM used reported by the OS (in bytes). |
| unreleased_delta_objects | Counter | Number of unreleased delta objects. |
| vertex_count | Counter | Number of nodes stored in the system. |
| SocketConnect_us_50p | Histogram | Latency of connecting to the socket, 50th percentile. |
| SocketConnect_us_90p | Histogram | Latency of connecting to the socket, 90th percentile. |
| SocketConnect_us_99p | Histogram | Latency of connecting to the socket, 99th percentile. |

#### Index metrics

| Name | Type | Description |
| -------------------------- | ------- | ------------------------------------------------------ |
| ActiveLabelIndices | Counter | Number of active label indexes in the system. |
| ActiveLabelPropertyIndices | Counter | Number of active label property indexes in the system. |
| ActivePointIndices | Counter | Number of active point indices in the system. |
| ActiveTextIndices | Counter | Number of active text indexes in the system. |

#### Memory metrics

| Name | Type | Description |
| ------------- | ---------------- | ------------------------------------------ |
| PeakMemoryRes | Gauge (uint64_t) | Peak RAM memory usage in the system during the whole run (in bytes). |

#### Operator metrics

@@ -277,52 +278,58 @@ database in order to yield the results for a given query. Every node within a
plan is known as [a logical operator](/querying/query-plan#query-plan-operators)
and describes a particular operation.

| Name | Type | Description |
| ----------------------------------- | ------- | -------------------------------------------------------------- |
| OnceOperator | Counter | Number of times Once operator was used. |
| CreateNodeOperator | Counter | Number of times CreateNode operator was used. |
| CreateExpandOperator | Counter | Number of times CreateExpand operator was used. |
| ScanAllOperator | Counter | Number of times ScanAll operator was used. |
| ScanAllByLabelOperator | Counter | Number of times ScanAllByLabel operator was used. |
| ScanAllByLabelPropertyRangeOperator | Counter | Number of times ScanAllByLabelPropertyRange operator was used. |
| ScanAllByLabelPropertyValueOperator | Counter | Number of times ScanAllByLabelPropertyValue operator was used. |
| ScanAllByLabelPropertyOperator | Counter | Number of times ScanAllByLabelProperty operator was used. |
| ScanAllByLabelIdOperator | Counter | Number of times ScanAllByLabelId operator was used. |
| ExpandOperator | Counter | Number of times Expand operator was used. |
| ExpandVariableOperator | Counter | Number of times ExpandVariable operator was used. |
| ConstructNamedPathOperator | Counter | Number of times ConstructNamedPath operator was used. |
| FilterOperator | Counter | Number of times Filter operator was used. |
| ProduceOperator | Counter | Number of times Produce operator was used. |
| DeleteOperator | Counter | Number of times Delete operator was used. |
| SetPropertyOperator | Counter | Number of times SetProperty operator was used. |
| SetPropertiesOperator | Counter | Number of times SetProperties operator was used. |
| SetLabelsOperator | Counter | Number of times SetLabels operator was used. |
| RemovePropertyOperator | Counter | Number of times RemoveProperty operator was used. |
| RemoveLabelsOperator | Counter | Number of times RemoveLabels operator was used. |
| EdgeUniquenessFilterOperator | Counter | Number of times EdgeUniquenessFilter operator was used. |
| EmptyResultOperator | Counter | Number of times EmptyResult operator was used. |
| AccumulateOperator | Counter | Number of times Accumulate operator was used. |
| AggregateOperator | Counter | Number of times Aggregate operator was used. |
| SkipOperator | Counter | Number of times Skip operator was used. |
| LimitOperator | Counter | Number of times Limit operator was used. |
| OrderByOperator | Counter | Number of times OrderBy operator was used. |
| MergeOperator | Counter | Number of times Merge operator was used. |
| OptionalOperator | Counter | Number of times Optional operator was used. |
| UnwindOperator | Counter | Number of times Unwind operator was used. |
| DistinctOperator | Counter | Number of times Distinct operator was used. |
| UnionOperator | Counter | Number of times Union operator was used. |
| CartesianOperator | Counter | Number of times Cartesian operator was used. |
| CallProcedureOperator | Counter | Number of times CallProcedureOperator operator was used. |
| ForeachOperator | Counter | Number of times Foreach operator was used. |
| EvaluatePatternFilterOperator | Counter | Number of times EvaluatePatternFilter operator was used. |
| ApplyOperator | Counter | Number of times Apply operator was used. |
| HashJoin | Counter | Number of times HashJoin operator was used. |
| IndexedJoin | Counter | Number of times IndexedJoin operator was used. |
| PeriodicCommit | Counter | Number of times PeriodicCommit operator was used. |
| PeriodicSubquery | Counter | Number of times PeriodicSubquery operator was used. |
| RollUpApplyOperator | Counter | Number of times RollUpApply operator was used. |
| ScanAllByEdgeIdOperator | Counter | Number of times ScanAllByEdgeId operator was used. |
| ScanAllByEdgeTypeOperator | Counter | Number of times ScanAllByEdgeType operator was used. |
| Name | Type | Description |
| ------------------------------------- | ------- | ------------------------------------------------------------------ |
| OnceOperator | Counter | Number of times Once operator was used. |
| CreateNodeOperator | Counter | Number of times CreateNode operator was used. |
| CreateExpandOperator | Counter | Number of times CreateExpand operator was used. |
| ScanAllOperator | Counter | Number of times ScanAll operator was used. |
| ScanAllByLabelOperator | Counter | Number of times ScanAllByLabel operator was used. |
| ScanAllByLabelPropertyRangeOperator | Counter | Number of times ScanAllByLabelPropertyRange operator was used. |
| ScanAllByLabelPropertyValueOperator | Counter | Number of times ScanAllByLabelPropertyValue operator was used. |
| ScanAllByLabelPropertyOperator | Counter | Number of times ScanAllByLabelProperty operator was used. |
| ScanAllByLabelIdOperator | Counter | Number of times ScanAllByLabelId operator was used. |
| ExpandOperator | Counter | Number of times Expand operator was used. |
| ExpandVariableOperator | Counter | Number of times ExpandVariable operator was used. |
| ConstructNamedPathOperator | Counter | Number of times ConstructNamedPath operator was used. |
| FilterOperator | Counter | Number of times Filter operator was used. |
| ProduceOperator | Counter | Number of times Produce operator was used. |
| DeleteOperator | Counter | Number of times Delete operator was used. |
| SetPropertyOperator | Counter | Number of times SetProperty operator was used. |
| SetPropertiesOperator | Counter | Number of times SetProperties operator was used. |
| SetLabelsOperator | Counter | Number of times SetLabels operator was used. |
| RemovePropertyOperator | Counter | Number of times RemoveProperty operator was used. |
| RemoveLabelsOperator | Counter | Number of times RemoveLabels operator was used. |
| EdgeUniquenessFilterOperator | Counter | Number of times EdgeUniquenessFilter operator was used. |
| EmptyResultOperator | Counter | Number of times EmptyResult operator was used. |
| AccumulateOperator | Counter | Number of times Accumulate operator was used. |
| AggregateOperator | Counter | Number of times Aggregate operator was used. |
| SkipOperator | Counter | Number of times Skip operator was used. |
| LimitOperator | Counter | Number of times Limit operator was used. |
| OrderByOperator | Counter | Number of times OrderBy operator was used. |
| MergeOperator | Counter | Number of times Merge operator was used. |
| OptionalOperator | Counter | Number of times Optional operator was used. |
| UnwindOperator | Counter | Number of times Unwind operator was used. |
| DistinctOperator | Counter | Number of times Distinct operator was used. |
| UnionOperator | Counter | Number of times Union operator was used. |
| CartesianOperator | Counter | Number of times Cartesian operator was used. |
| CallProcedureOperator | Counter | Number of times CallProcedureOperator operator was used. |
| ForeachOperator | Counter | Number of times Foreach operator was used. |
| EvaluatePatternFilterOperator | Counter | Number of times EvaluatePatternFilter operator was used. |
| ApplyOperator | Counter | Number of times Apply operator was used. |
| HashJoin | Counter | Number of times HashJoin operator was used. |
| IndexedJoin | Counter | Number of times IndexedJoin operator was used. |
| PeriodicCommit | Counter | Number of times PeriodicCommit operator was used. |
| PeriodicSubquery | Counter | Number of times PeriodicSubquery operator was used. |
| RollUpApplyOperator | Counter | Number of times RollUpApply operator was used. |
| ScanAllByEdgeIdOperator | Counter | Number of times ScanAllByEdgeId operator was used. |
| ScanAllByEdgeOperator | Counter | Number of times ScanAllByEdge operator was used. |
| ScanAllByEdgeTypeOperator | Counter | Number of times ScanAllByEdgeType operator was used. |
| ScanAllByEdgeTypePropertyOperator | Counter | Number of times ScanAllByEdgeTypeProperty operator was used. |
| ScanAllByEdgeTypePropertyRangeOperator | Counter | Number of times ScanAllByEdgeTypePropertyRange operator was used. |
| ScanAllByEdgeTypePropertyValueOperator | Counter | Number of times ScanAllByEdgeTypePropertyValue operator was used. |
| ScanAllByPointDistanceOperator | Counter | Number of times ScanAllByPointDistance operator was used. |
| ScanAllByPointWithinbboxOperator | Counter | Number of times ScanAllByPointWithinbbox operator was used. |

#### Query metrics

@@ -356,7 +363,7 @@ and describes a particular operation.
| Name | Type | Description |
| ------------------------------ | --------- | ------------------------------------------------------------ |
| SnapshotCreationLatency_us_50p | Histogram | Snapshot creation latency in microseconds (50th percentile). |
| SnapshotCreationLatency_us_90p | Histogram | Snapshot creation latency in microseconds (90th percentile)- |
| SnapshotCreationLatency_us_90p | Histogram | Snapshot creation latency in microseconds (90th percentile). |
| SnapshotCreationLatency_us_99p | Histogram | Snapshot creation latency in microseconds (99th percentile). |
| SnapshotRecoveryLatency_us_50p | Histogram | Snapshot recovery latency in microseconds (50th percentile). |
| SnapshotRecoveryLatency_us_90p | Histogram | Snapshot recovery latency in microseconds (90th percentile). |
@@ -389,121 +396,106 @@ and describes a particular operation.
| TriggersCreated | Counter | Number of Triggers created. |
| TriggersExecuted | Counter | Number of Triggers executed. |

### Example response

If there aren't any modifying configurations, by sending a GET request to
`localhost:9091` in the local Memgraph build will result in a response similar
to the one below.
#### TTL metrics

| Name | Type | Description |
| ------------ | ------- | ---------------------------- |
| DeletedNodes | Counter | Number of deleted TTL edges. |
| DeletedEdges | Counter | Number of deleted TTL nodes. |

#### HA metrics

| Name | Type | Description |
| --------------------------------- | --------- | -------------------------------------------------------------------------------------------------- |
| AppendDeltasRpc_us_50p | Histogram | AppendDeltasRpc latency in microseconds (50th percentile). |
| AppendDeltasRpc_us_90p | Histogram | AppendDeltasRpc latency in microseconds (90th percentile). |
| AppendDeltasRpc_us_99p | Histogram | AppendDeltasRpc latency in microseconds (99th percentile). |
| CurrentWalRpc_us_50p | Histogram | CurrentWalRpc latency in microseconds (50th percentile). |
| CurrentWalRpc_us_90p | Histogram | CurrentWalRpc latency in microseconds (90th percentile). |
| CurrentWalRpc_us_99p | Histogram | CurrentWalRpc latency in microseconds (99th percentile). |
| WalFilesRpc_us_50p | Histogram | WalFilesRpc latency in microseconds (50th percentile). |
| WalFilesRpc_us_90p | Histogram | WalFilesRpc latency in microseconds (90th percentile). |
| WalFilesRpc_us_99p | Histogram | WalFilesRpc latency in microseconds (99th percentile). |
| ReplicaStream_us_50p | Histogram | Time needed to construct AppendDeltasRpc stream (50th percentile). |
| ReplicaStream_us_90p | Histogram | Time needed to construct AppendDeltasRpc stream (90th percentile). |
| ReplicaStream_us_99p | Histogram | Time needed to construct AppendDeltasRpc stream (99th percentile). |
| SnapshotRpc_us_50p | Histogram | SnapshotRpc latency in microseconds (50th percentile). |
| SnapshotRpc_us_90p | Histogram | SnapshotRpc latency in microseconds (90th percentile). |
| SnapshotRpc_us_99p | Histogram | SnapshotRpc latency in microseconds (99th percentile). |
| FrequentHeartbeatRpc_us_50p | Histogram | FrequentHeartbeatRpc latency in microseconds (50th percentile). |
| FrequentHeartbeatRpc_us_90p | Histogram | FrequentHeartbeatRpc latency in microseconds (90th percentile). |
| FrequentHeartbeatRpc_us_99p | Histogram | FrequentHeartbeatRpc latency in microseconds (99th percentile). |
| HeartbeatRpc_us_50p | Histogram | HeartbeatRpc latency in microseconds (50th percentile). |
| HeartbeatRpc_us_90p | Histogram | HeartbeatRpc latency in microseconds (90th percentile). |
| HeartbeatRpc_us_99p | Histogram | HeartbeatRpc latency in microseconds (99th percentile). |
| SystemRecoveryRpc_us_50p | Histogram | SystemRecoveryRpc latency in microseconds (50th percentile). |
| SystemRecoveryRpc_us_90p | Histogram | SystemRecoveryRpc latency in microseconds (90th percentile). |
| SystemRecoveryRpc_us_99p | Histogram | SystemRecoveryRpc latency in microseconds (99th percentile). |
| ChooseMostUpToDateInstance_us_50p | Histogram | ChooseMostUpToDateInstance latency in microseconds (50th percentile). |
| ChooseMostUpToDateInstance_us_90p | Histogram | ChooseMostUpToDateInstance latency in microseconds (90th percentile). |
| ChooseMostUpToDateInstance_us_99p | Histogram | ChooseMostUpToDateInstance latency in microseconds (99th percentile). |
| GetHistories_us_50p | Histogram | GetHistories latency in microseconds (50th percentile). |
| GetHistories_us_90p | Histogram | GetHistories latency in microseconds (90th percentile). |
| GetHistories_us_99p | Histogram | GetHistories latency in microseconds (99th percentile). |
| InstanceFailCallback_us_50p | Histogram | InstanceFailCallback latency in microseconds (50th percentile). |
| InstanceFailCallback_us_90p | Histogram | InstanceFailCallback latency in microseconds (90th percentile). |
| InstanceFailCallback_us_99p | Histogram | InstanceFailCallback latency in microseconds (99th percentile). |
| InstanceSuccCallback_us_50p | Histogram | InstanceSuccCallback latency in microseconds (50th percentile). |
| InstanceSuccCallback_us_90p | Histogram | InstanceSuccCallback latency in microseconds (90th percentile). |
| InstanceSuccCallback_us_99p | Histogram | InstanceSuccCallback latency in microseconds (99th percentile). |
| DemoteMainToReplicaRpc_us_50p | Histogram | DemoteMainToReplicaRpc latency in microseconds (50th percentile). |
| DemoteMainToReplicaRpc_us_90p | Histogram | DemoteMainToReplicaRpc latency in microseconds (90th percentile). |
| DemoteMainToReplicaRpc_us_99p | Histogram | DemoteMainToReplicaRpc latency in microseconds (99th percentile). |
| EnableWritingOnMainRpc_us_50p | Histogram | EnableWritingOnMainRpc latency in microseconds (50th percentile). |
| EnableWritingOnMainRpc_us_90p | Histogram | EnableWritingOnMainRpc latency in microseconds (90th percentile). |
| EnableWritingOnMainRpc_us_99p | Histogram | EnableWritingOnMainRpc latency in microseconds (99th percentile). |
| GetDatabaseHistoriesRpc_us_50p | Histogram | GetDatabaseHistoriesRpc latency in microseconds (50th percentile). |
| GetDatabaseHistoriesRpc_us_90p | Histogram | GetDatabaseHistoriesRpc latency in microseconds (90th percentile). |
| GetDatabaseHistoriesRpc_us_99p | Histogram | GetDatabaseHistoriesRpc latency in microseconds (99th percentile). |
| PromoteToMainRpc_us_50p | Histogram | PromoteToMainRpc latency in microseconds (50th percentile). |
| PromoteToMainRpc_us_90p | Histogram | PromoteToMainRpc latency in microseconds (90th percentile). |
| PromoteToMainRpc_us_99p | Histogram | PromoteToMainRpc latency in microseconds (99th percentile). |
| RegisterReplicaOnMainRpc_us_50p | Histogram | RegisterReplicaOnMainRpc latency in microseconds (50th percentile). |
| RegisterReplicaOnMainRpc_us_90p | Histogram | RegisterReplicaOnMainRpc latency in microseconds (90th percentile). |
| RegisterReplicaOnMainRpc_us_99p | Histogram | RegisterReplicaOnMainRpc latency in microseconds (99th percentile). |
| StateCheckRpc_us_50p | Histogram | StateCheckRpc latency in microseconds (50th percentile). |
| StateCheckRpc_us_90p | Histogram | StateCheckRpc latency in microseconds (90th percentile). |
| StateCheckRpc_us_99p | Histogram | StateCheckRpc latency in microseconds (99th percentile). |
| UnregisterReplicaRpc_us_50p | Histogram | UnregisterReplicaRpc latency in microseconds (50th percentile). |
| UnregisterReplicaRpc_us_90p | Histogram | UnregisterReplicaRpc latency in microseconds (90th percentile). |
| UnregisterReplicaRpc_us_99p | Histogram | UnregisterReplicaRpc latency in microseconds (99th percentile). |
| BecomeLeaderSuccess | Counter | The number of times coordinators successfully became leaders. |
| FailedToBecomeLeader | Counter | The number of times coordinators failed to become leaders. |
| SuccessfulFailovers | Counter | The number of times failover was done successfully. |
| RaftFailedFailovers | Counter | The number of times failover failed because writing to Raft failed. |
| NoAliveInstanceFailedFailovers | Counter | The number of times failover failed because no instance was alive. |
| ShowInstance | Counter | The number of times `SHOW INSTANCE` query was called. |
| ShowInstances | Counter | The number of times `SHOW INSTANCES` query was called. |
| DemoteInstance | Counter | The number of times the user manually demoted instance. |
| UnregisterReplInstance | Counter | The number of times the user tried to unregister replication instance. |
| RemoveCoordInstance | Counter | The number of times the user tried to remove coordinator instance. |
| StateCheckRpcFail | Counter | The number of times coordinators received unsuccessful or no response to StateCheckRpc. |
| StateCheckRpcSuccess | Counter | The number of times coordinators received successful response to StateCheckRpc. |
| UnregisterReplicaRpcFail | Counter | The number of times coordinators received unsuccessful or no response to UnregisterReplicaRpc. |
| UnregisterReplicaRpcSuccess | Counter | The number of times coordinators received successful response to UnregisterReplicaRpc. |
| EnableWritingOnMainRpcFail | Counter | The number of times coordinators received unsuccessful or no response to EnableWritingOnMainRpc. |
| EnableWritingOnMainRpcSuccess | Counter | The number of times coordinators received successful response to EnableWritingOnMainRpc. |
| PromoteToMainRpcFail | Counter | The number of times coordinators received unsuccessful or no response to PromoteToMainRpc. |
| PromoteToMainRpcSuccess | Counter | The number of times coordinators received successful response to PromoteToMainRpc. |
| DemoteMainToReplicaRpcFail | Counter | The number of times coordinators received unsuccessful or no response to DemoteMainToReplicaRpc. |
| DemoteMainToReplicaRpcSuccess | Counter | The number of times coordinators received successful response to DemoteMainToReplicaRpc. |
| RegisterReplicaOnMainRpcFail | Counter | The number of times coordinators received unsuccessful or no response to RegisterReplicaOnMainRpc. |
| RegisterReplicaOnMainRpcSuccess | Counter | The number of times coordinators received successful response to RegisterReplicaOnMainRpc. |
| SwapMainUUIDRpcFail | Counter | The number of times coordinators received unsuccessful or no response to SwapMainUUIDRpc. |
| SwapMainUUIDRpcSuccess | Counter | The number of times coordinators received successful response to SwapMainUUIDRpc. |
| GetDatabaseHistoriesRpcFail | Counter | The number of times coordinators received unsuccessful or no response to GetDatabaseHistoriesRpc. |
| GetDatabaseHistoriesRpcSuccess | Counter | The number of times coordinators received successful response to GetDatabaseHistoriesRpc. |


All HA metrics with type `Counter` are aggregated for all coordinators. That makes it easier for users to track what is going on since they don't need to aggregate
by their own metrics specific to each coordinators. Also, after every pull, HA metrics with type `Counter` are in Memgraph reset to 0. This makes it possible to use `Gauge` on
the client side without worrying that on restart we will lose all data.

```json
{
"General": {
"average_degree": 0.0,
"disk_usage": 1417846,
"edge_count": 0,
"memory_usage": 36937728,
"peak_memory_usage": 100265984,
"vertex_count": 0
},
"Index": {
"ActiveLabelIndices": 0,
"ActiveLabelPropertyIndices": 0,
"ActiveTextIndices": 0
},
"Memory": {
"PeakMemoryRes": 100265984
},
"Operator": {
"AccumulateOperator": 0,
"AggregateOperator": 0,
"ApplyOperator": 0,
"CallProcedureOperator": 0,
"CartesianOperator": 0,
"ConstructNamedPathOperator": 0,
"CreateExpandOperator": 0,
"CreateNodeOperator": 0,
"DeleteOperator": 0,
"DistinctOperator": 0,
"EdgeUniquenessFilterOperator": 0,
"EmptyResultOperator": 0,
"EvaluatePatternFilterOperator": 0,
"ExpandOperator": 0,
"ExpandVariableOperator": 0,
"FilterOperator": 0,
"ForeachOperator": 0,
"HashJoinOperator": 0,
"IndexedJoinOperator": 0,
"LimitOperator": 0,
"MergeOperator": 0,
"OnceOperator": 0,
"OptionalOperator": 0,
"OrderByOperator": 0,
"ProduceOperator": 0,
"PeriodicCommit": 0,
"PeriodicSubquery": 0,
"RemoveLabelsOperator": 0,
"RemovePropertyOperator": 0,
"RollUpApplyOperator": 0,
"ScanAllByEdgeIdOperator": 0,
"ScanAllByEdgeTypeOperator": 0,
"ScanAllByIdOperator": 0,
"ScanAllByLabelOperator": 0,
"ScanAllByLabelPropertyOperator": 0,
"ScanAllByLabelPropertyRangeOperator": 0,
"ScanAllByLabelPropertyValueOperator": 0,
"ScanAllOperator": 0,
"SetLabelsOperator": 0,
"SetPropertiesOperator": 0,
"SetPropertyOperator": 0,
"SkipOperator": 0,
"UnionOperator": 0,
"UnwindOperator": 0
},
"Query": {
"QueryExecutionLatency_us_50p": 0,
"QueryExecutionLatency_us_90p": 0,
"QueryExecutionLatency_us_99p": 0
},
"QueryType": {
"ReadQuery": 0,
"ReadWriteQuery": 0,
"WriteQuery": 0
},
"Session": {
"ActiveBoltSessions": 0,
"ActiveSSLSessions": 0,
"ActiveSessions": 0,
"ActiveTCPSessions": 0,
"ActiveWebSocketSessions": 0,
"BoltMessages": 0
},
"Snapshot": {
"SnapshotCreationLatency_us_50p": 4860,
"SnapshotCreationLatency_us_90p": 4860,
"SnapshotCreationLatency_us_99p": 4860,
"SnapshotRecoveryLatency_us_50p": 628,
"SnapshotRecoveryLatency_us_90p": 628,
"SnapshotRecoveryLatency_us_99p": 628
},
"Stream": {
"MessagesConsumed": 0,
"StreamsCreated": 0
},
"Transaction": {
"ActiveTransactions": 0,
"CommitedTransactions": 0,
"FailedPrepare": 0,
"FailedPull": 0,
"FailedQuery": 0,
"RollbackedTransactions": 0,
"SuccessfulQuery": 0
},
"Trigger": {
"TriggersCreated": 0,
"TriggersExecuted": 0
}
}
```

## Prometheus exporter