CPU Issue with v0.13.0 version

[krishnaindani]

With the upgrade from v0.12.0 to v0.13.0 version, we are seeing a constant increase in CPU utilization for carbon-relay-ng.

With the v0.12.0 version, we used to have spiked in CPU when the network traffic and workload increases but v0.13.0 version CPU utilization is not coming down and constantly increasing.

From the snapshot above, between 08/18 to 08/19 20:00, it was at v0.12.0 version and after that even with the same network traffic CPU utilization is spiking.

We are using carbon relay as deployment in Kubernetes (v1.17).
Cabon relay config has some aggregation rules and metrics are sent to Grafana labs Graphite with the 10 seconds metrics aggregation rules.

This is another instance for example, where we have defined aggregation rules and in the route section, we are sending metrics to confluent cloud Kafka.

In this snapshot when CPU utilization came down, it was a restart and spiking again.

[Dieterbe]

can you confirm that if you revert to 0.12 the issue disappears?

[krishnaindani]

Yes, the issue disappears with rollback to the v0.12.0 version.

[krishnaindani]

This is the snap after rolling back

We can see that on 08/20 12:00, it was rolled back to v0.12.0 version and CPU looks good after that even though we saw some increase in network activity.

[cjonesshipt]

I updated to v0.13.0 on Friday in our Staging-Env, just to double check that I'm still seeing this, too. Looks, like it's still a problem for me.

Green line is the upgrade to v0.13.0, fwiw:

[Dieterbe]

is it possible to let it run without aggregator? reviewing the changelog, seems most changes for the 0.13.0 release was related to aggregators.
I suspect either that, or something to do with matchers (anything that has a prefix/substring/regex condition on it, such as a route), but more likely aggregator.