Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interval processor, spanlogs connector, and spanmetrics connector #1119

Merged
merged 8 commits into from
Jan 24, 2025
Merged

Add interval processor, spanlogs connector, and spanmetrics connector #1119

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8022d83
Add interval processor
petewall Jan 15, 2025
c18b921
Start adding spanmetrics and spanlogs
petewall Jan 22, 2025
ae72377
Rebuild the AppO11y pipeline so it's defined in a pipeline.tpl file. …
petewall Jan 23, 2025
7511ea5
Add spanlogs and spanmetrics connectors
petewall Jan 23, 2025
64025df
Add namespace field to spanmetrics
petewall Jan 24, 2025
314fa68
Update charts/k8s-monitoring/charts/feature-application-observability…
petewall Jan 24, 2025
76ae190
Update appo11y readme
petewall Jan 24, 2025
5fc2c38
Update the description for the grafanaCloudMetrics connector to highl…
petewall Jan 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 40 additions & 2 deletions charts/k8s-monitoring/charts/feature-application-observability/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,41 @@

## Values

### Connectors: Grafana Cloud Host Info

Check warning on line 43 in charts/k8s-monitoring/charts/feature-application-observability/README.md

View workflow job for this annotation

GitHub Actions / runner / alex 

[alex] reported by reviewdog 🐶
`Host` may be insensitive, use `Presenter`, `Entertainer`, `Emcee` instead                                                           host-hostess  retext-equality

Raw Output:
    43:31-43:35  warning  `Host` may be insensitive, use `Presenter`, `Entertainer`, `Emcee` instead                                                           host-hostess  retext-equality

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| connectors.grafanaCloudMetrics.enabled | bool | `true` | Generate host info metrics from telemetry data. These metrics are required for using Application Observability in Grafana Cloud. Note: Enabling this may incur additional costs. See https://grafana.com/docs/grafana-cloud/monitor-applications/application-observability/pricing/ |

Check failure on line 47 in charts/k8s-monitoring/charts/feature-application-observability/README.md

View workflow job for this annotation

GitHub Actions / runner / markdownlint 

[markdownlint] reported by reviewdog 🐶
MD034/no-bare-urls Bare URL used [Context: "https://grafana.com/docs/grafa..."]

Raw Output:
charts/k8s-monitoring/charts/feature-application-observability/README.md:47:241 MD034/no-bare-urls Bare URL used [Context: "https://grafana.com/docs/grafa..."]
petewall marked this conversation as resolved.
Show resolved Hide resolved

### Connectors: Span Logs

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| connectors.spanLogs.enabled | bool | `false` | Use a span logs connector which creates logs from spans. |
| connectors.spanLogs.labels | list | `[]` | A list of keys that will be logged as labels. |
| connectors.spanLogs.process | bool | `false` | Log one line for every process. |
| connectors.spanLogs.processAttributes | list | `[]` | Additional process attributes to log. |
| connectors.spanLogs.roots | bool | `false` | Log one line for every root span of a trace. |
| connectors.spanLogs.spanAttributes | list | `[]` | Additional span attributes to log. |
| connectors.spanLogs.spans | bool | `false` | Create a log line for each span. This can lead to a large number of logs. |

### Connectors: Span Metrics

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| connectors.spanMetrics.dimensions | list | `[]` | Define dimensions to be added. Some are set internally by default: [service.name, span.name, span.kind, status.code] Example: - name: "http.status_code" - name: "http.method" default: "GET" |
| connectors.spanMetrics.dimensionsCacheSize | int | `1000` | How many dimensions to cache |
| connectors.spanMetrics.enabled | bool | `false` | Use a span metrics connector which creates metrics from spans. |
| connectors.spanMetrics.events.enabled | bool | `false` | Capture events metrics, which track span events. |
| connectors.spanMetrics.exemplars.enabled | bool | `false` | Attach exemplars to histograms. |
| connectors.spanMetrics.exemplars.maxPerDataPoint | number | `nil` | Limits the number of exemplars that can be added to a unique dimension set. |
| connectors.spanMetrics.histogram.enabled | bool | `true` | Capture histogram metrics, derived from spans’ durations. |
| connectors.spanMetrics.histogram.explicit.buckets | list | `["2ms","4ms","6ms","8ms","10ms","50ms","100ms","200ms","400ms","800ms","1s","1400ms","2s","5s","10s","15s"]` | The histogram buckets to use. |
| connectors.spanMetrics.histogram.exponential.maxSize | int | `160` | Maximum number of buckets per positive or negative number range. |
| connectors.spanMetrics.histogram.type | string | `"explicit"` | Type of histograms to create. Must be either "explicit" or "exponential". |
| connectors.spanMetrics.histogram.unit | string | `"ms"` | The histogram unit. |
| connectors.spanMetrics.namespace | string | `"traces.span.metrics"` | The Metric namespace. |
petewall marked this conversation as resolved.
Show resolved Hide resolved

### General settings

| Key | Type | Default | Description |
Expand All @@ -55,11 +90,14 @@
| processors.batch.size | int | `16384` | What batch size to use, in bytes |
| processors.batch.timeout | string | `"2s"` | How long before sending (Processors) |

### Processors: Grafana Cloud Host Info
### Processors: Interval

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| processors.grafanaCloudMetrics.enabled | bool | `true` | Generate host info metrics from telemetry data, used in Application Observability in Grafana Cloud. |
| processors.interval.enabled | bool | `false` | Utilize an interval processor to aggregate metrics and periodically forward the latest values to the next component in the pipeline. |
| processors.interval.interval | string | `"60s"` | The interval at which to emit aggregated metrics. |
| processors.interval.passthrough.gauge | bool | `false` | Determines whether gauge metrics should be passed through as they are or aggregated. |
| processors.interval.passthrough.summary | bool | `false` | Determines whether summary metrics should be passed through as they are or aggregated. |

### Processors: K8s Attributes

Expand Down
13 changes: 13 additions & 0 deletions .../k8s-monitoring/charts/feature-application-observability/schema-mods/types-and-enums.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"properties": {
"connectors": {
"properties": {
"spanMetrics": {
"properties": {
"histogram": {"properties": {"type": {"enum": ["explicit", "exponential"]}}}
}
}
}
}
}
}
4 changes: 2 additions & 2 deletions ...8s-monitoring/charts/feature-application-observability/templates/_connector_host_info.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ otelcol.connector.host_info "{{ .name | default "default" }}" {
host_identifiers = [ "k8s.node.name" ]
output {
{{- if and .metricsOutput .Values.metrics.enabled }}
metrics = {{ .metricsOutput }}
{{- if and .metrics .Values.metrics.enabled }}
metrics = {{ .metrics }}
{{- end }}
}
}
Expand Down
31 changes: 31 additions & 0 deletions ...8s-monitoring/charts/feature-application-observability/templates/_connector_span_logs.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{{/* Inputs: Values (values) metricsOutput, name */}}
{{/* https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.connector.spanlogs/ */}}
{{- define "feature.applicationObservability.connector.spanlogs.alloy.target" }}otelcol.connector.spanlogs.{{ .name | default "default" }}.input{{- end }}
{{- define "feature.applicationObservability.connector.spanlogs.alloy" }}
otelcol.connector.spanlogs "{{ .name | default "default" }}" {
{{- if .Values.connectors.spanLogs.spans }}
spans = true
{{- end }}
{{- if .Values.connectors.spanLogs.spansAttributes }}
spans_attributes = {{ .Values.connectors.spanLogs.spansAttributes | toJson }}
{{- end }}
{{- if .Values.connectors.spanLogs.roots }}
roots = true
{{- end }}
{{- if .Values.connectors.spanLogs.process }}
process = true
{{- end }}
{{- if .Values.connectors.spanLogs.processAttributes }}
process_attributes = {{ .Values.connectors.spanLogs.processAttributes | toJson }}
{{- end }}
{{- if .Values.connectors.spanLogs.labels }}
labels = {{ .Values.connectors.spanLogs.labels | toJson }}
{{- end }}

output {
{{- if and .logs .Values.logs.enabled }}
logs = {{ .logs }}
{{- end }}
}
}
{{- end }}
51 changes: 51 additions & 0 deletions ...monitoring/charts/feature-application-observability/templates/_connector_span_metrics.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
{{/* Inputs: Values (values) metricsOutput, name */}}
{{/* https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.connector.spanmetrics/ */}}
{{- define "feature.applicationObservability.connector.spanmetrics.alloy.target" }}otelcol.connector.spanmetrics.{{ .name | default "default" }}.input{{- end }}
{{- define "feature.applicationObservability.connector.spanmetrics.alloy" }}
otelcol.connector.spanmetrics "{{ .name | default "default" }}" {
{{- range $dimension := .Values.connectors.spanMetrics.dimensions }}
dimension {
name = {{ $dimension.name | quote }}
{{- if $dimension.default }}
default = {{ $dimension.default | quote }}
{{- end }}
}
{{- end }}
dimensions_cache_size = {{ .Values.connectors.spanMetrics.dimensionsCacheSize }}
namespace = {{ .Values.connectors.spanMetrics.namespace | quote }}
{{- if .Values.connectors.spanMetrics.events.enabled }}
events {
enabled = true
}
{{- end }}
{{ if .Values.connectors.spanMetrics.exemplars.enabled }}
exemplars {
enabled = true
{{- if .Values.connectors.spanMetrics.exemplars.maxPerDataPoint }}
max_per_data_point = {{ .Values.connectors.spanMetrics.exemplars.maxPerDataPoint }}
{{- end }}
}
{{- end }}
{{- if .Values.connectors.spanMetrics.histogram.enabled }}
histogram {
disable = false
unit = {{ .Values.connectors.spanMetrics.histogram.unit | quote }}
{{- if eq .Values.connectors.spanMetrics.histogram.type "explicit" }}
explicit {
buckets = {{ .Values.connectors.spanMetrics.histogram.explicit.buckets | toJson }}
}
{{- else if eq .Values.connectors.spanMetrics.histogram.type "exponential" }}
exponential {
max_size = {{ .Values.connectors.spanMetrics.histogram.exponential.maxSize }}
}
{{- end }}
}
{{- end }}

output {
{{- if and .metrics .Values.metrics.enabled }}
metrics = {{ .metrics }}
{{- end }}
}
}
{{- end }}
87 changes: 18 additions & 69 deletions charts/k8s-monitoring/charts/feature-application-observability/templates/_module.alloy.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,4 @@
{{- define "feature.applicationObservability.module" }}
{{- $metricsNext := "" }}
{{- $logsNext := "" }}
{{- $resourceDetection := include "feature.applicationObservability.processor.resourcedetection.alloy.target" dict }}
{{- $k8sAttributes := include "feature.applicationObservability.processor.k8sattributes.alloy.target" dict }}
{{- $grafanaCloudMetrics := include "feature.applicationObservability.connector.host_info.alloy.target" dict }}
{{- $transform := include "feature.applicationObservability.processor.transform.alloy.target" dict }}
{{- $filter := include "feature.applicationObservability.processor.filter.alloy.target" dict }}
{{- $batch := include "feature.applicationObservability.processor.batch.alloy.target" dict }}
{{- $memoryLimiter := include "feature.applicationObservability.processor.memory_limiter.alloy.target" dict }}
declare "application_observability" {
argument "metrics_destinations" {
comment = "Must be a list of metrics destinations where collected metrics should be forwarded to"
Expand All @@ -20,66 +11,24 @@ declare "application_observability" {
argument "traces_destinations" {
comment = "Must be a list of trace destinations where collected trace should be forwarded to"
}

// Receivers --> Resource Detection Processor
{{- $next := printf "[%s]" $resourceDetection }}
{{- include "feature.applicationObservability.receiver.otlp.alloy" (dict "Values" $.Values "metricsOutput" $next "logsOutput" $next "tracesOutput" $next ) | indent 2 }}
{{- include "feature.applicationObservability.receiver.jaeger.alloy" (dict "Values" $.Values "tracesOutput" $next ) | indent 2 }}
{{- include "feature.applicationObservability.receiver.zipkin.alloy" (dict "Values" $.Values "tracesOutput" $next ) | indent 2 }}

// Resource Detection Processor --> K8s Attribute Processor
{{- $next = printf "[%s]" $k8sAttributes }}
{{- include "feature.applicationObservability.processor.resourcedetection.alloy" (dict "Values" $.Values "metricsOutput" $next "logsOutput" $next "tracesOutput" $next ) | indent 2 }}

// K8s Attribute Processor --> Transform Processor
{{- $tracesNext := list $transform }}
{{- $next = printf "[%s]" $transform }}
{{- if .Values.processors.grafanaCloudMetrics.enabled }}
// Resource Detection Processor Traces --> Host Info Connector
{{- $tracesNext = append $tracesNext $grafanaCloudMetrics }}
{{- end -}}
{{- $tracesNext = printf "[%s]" ($tracesNext | join ", ")}}
{{- include "feature.applicationObservability.processor.k8sattributes.alloy" (dict "Values" $.Values "metricsOutput" $next "logsOutput" $next "tracesOutput" $tracesNext ) | indent 2 }}

{{- if .Values.processors.grafanaCloudMetrics.enabled }}
// Host Info Connector --> Batch Processor
{{- $next = printf "[%s]" $batch }}
{{- include "feature.applicationObservability.connector.host_info.alloy" (dict "Values" $.Values "metricsOutput" $next ) | indent 2 }}
{{- end }}

{{ if eq (include "feature.applicationObservability.processor.filter.enabled" .) "true" }}
// Transform Processor --> Filter Processor
{{- $next = printf "[%s]" $filter }}
{{- else }}
// Transform Processor --> Batch Processor
{{- $next = printf "[%s]" $batch }}
{{- end }}
{{- include "feature.applicationObservability.processor.transform.alloy" (dict "Values" $.Values "metricsOutput" $next "logsOutput" $next "tracesOutput" $next ) | indent 2 }}
{{ if eq (include "feature.applicationObservability.processor.filter.enabled" .) "true" }}
// Filter Processor --> Batch Processor
{{- $next = printf "[%s]" $batch }}
{{- include "feature.applicationObservability.processor.filter.alloy" (dict "Values" $.Values "metricsOutput" $next "logsOutput" $next "tracesOutput" $next ) | indent 2 }}
{{- end }}

{{- if .Values.processors.memoryLimiter.enabled }}
// Batch Processor --> Memory Limiter
{{- $metricsNext = printf "[%s]" $memoryLimiter }}
{{- $logsNext = printf "[%s]" $memoryLimiter }}
{{- $tracesNext = printf "[%s]" $memoryLimiter }}
{{- else }}
// Batch Processor --> Destinations
{{- $metricsNext = "argument.metrics_destinations.value" }}
{{- $logsNext = "argument.logs_destinations.value" }}
{{- $tracesNext = "argument.traces_destinations.value" }}
{{- end }}
{{- include "feature.applicationObservability.processor.batch.alloy" (dict "Values" $.Values "metricsOutput" $metricsNext "logsOutput" $logsNext "tracesOutput" $tracesNext ) | indent 2 }}

{{- if .Values.processors.memoryLimiter.enabled }}
// Memory Limiter --> Destinations
{{- $metricsNext = "argument.metrics_destinations.value" }}
{{- $logsNext = "argument.logs_destinations.value" }}
{{- $tracesNext = "argument.traces_destinations.value" }}
{{- include "feature.applicationObservability.processor.memory_limiter.alloy" (dict "Values" $.Values "metricsOutput" $metricsNext "logsOutput" $logsNext "tracesOutput" $tracesNext ) | indent 2 }}
{{- $pipeline := include "feature.applicationObservability.pipeline" . | fromYamlArray }}
{{- range $component := $pipeline }}
{{- $args := (dict "Values" $.Values "name" $component.name) }}

{{- range $dataType := (list "metrics" "logs" "traces")}}
{{- if kindIs "string" (index $component.targets $dataType) }}
{{- $args = merge $args (dict $dataType (index $component.targets $dataType)) }}
{{- else if kindIs "slice" (index $component.targets $dataType) }}
{{- $targets := list }}
{{- range $target := (index $component.targets $dataType) }}
{{- $targets = append $targets (include (printf "feature.applicationObservability.%s.alloy.target" $target.component) $target) }}
{{- end }}
{{- $args = merge $args (dict $dataType (printf "[%s]" (join ", " $targets))) }}
{{- end }}
{{- end }}

// {{ $component.description }}
{{- include (printf "feature.applicationObservability.%s.alloy" $component.component) $args | indent 2 }}
{{- end }}
}
{{- end }}
Expand Down
Loading
Loading