Skip to content
Merged
Show file tree
Hide file tree
Changes from 121 commits
Commits
Show all changes
171 commits
Select commit Hold shift + click to select a range
c64f4d8
chore: Renovate support
a-thaler Jan 16, 2025
dbbf6bf
Update renovate.json
a-thaler Jan 17, 2025
3a514d7
Update renovate.json
a-thaler Jan 17, 2025
901fb01
Update renovate.json
a-thaler Jan 17, 2025
37bec0c
Update renovate.json
a-thaler Jan 17, 2025
348f6ef
Update renovate.json
a-thaler Jan 17, 2025
6ad3e25
Update renovate.json
a-thaler Jan 17, 2025
f5ff171
Update renovate.json
a-thaler Jan 17, 2025
4fb14c9
Update renovate.json
a-thaler Jan 17, 2025
28e56f0
docs: revamp
a-thaler Jul 29, 2025
d35f1cf
fix
a-thaler Jul 29, 2025
d3d90da
redesign
a-thaler Jul 30, 2025
6168b27
Merge branch 'main' into docs
a-thaler Jul 31, 2025
3a4a176
fix
a-thaler Aug 4, 2025
8053983
fix
a-thaler Aug 4, 2025
ef241af
fixes
a-thaler Aug 6, 2025
6908baa
Merge branch 'main' into docs
a-thaler Aug 6, 2025
0b83bcd
fix
a-thaler Aug 6, 2025
cac01a4
Merge branch 'main' into docs
a-thaler Aug 8, 2025
0b05b74
added migration guide
a-thaler Aug 11, 2025
02cb528
Merge branch 'main' into docs
a-thaler Aug 12, 2025
c3a8c7b
Review feedback
a-thaler Aug 18, 2025
28adee5
added istio
a-thaler Aug 19, 2025
b3400f6
fix
a-thaler Aug 20, 2025
4ebdc4f
Merge branch 'main' into docs
a-thaler Aug 20, 2025
e08fcfd
improved doc titles
a-thaler Aug 20, 2025
aac8797
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
a-thaler Aug 20, 2025
b5bbc18
new diagram
a-thaler Aug 21, 2025
4cac1b7
fix diagram
a-thaler Aug 21, 2025
6f74592
migrate cls guide
a-thaler Aug 22, 2025
e23b7aa
Merge branch 'main' into docs
a-thaler Sep 4, 2025
381db8f
Review migration guide
NHingerl Sep 9, 2025
edc4254
temporarily move the guide to root for review
NHingerl Sep 18, 2025
fc01206
adjust title
NHingerl Sep 18, 2025
6141f81
language review
NHingerl Sep 18, 2025
12ebdcf
language
NHingerl Sep 18, 2025
077eec6
language
NHingerl Sep 18, 2025
513d9a6
Merge branch 'main' into docs
a-thaler Sep 26, 2025
776de3e
first batch of content updates
NHingerl Sep 29, 2025
40b1aae
Merge pull request #22 from NHingerl/pr-2341-ninas-changes
a-thaler Oct 8, 2025
9bcd945
two more docs
NHingerl Oct 8, 2025
c06f299
update metrics docs
NHingerl Oct 8, 2025
e2fa150
health monitoring
NHingerl Oct 8, 2025
4803f4e
mov and rename monitor pipeline health
NHingerl Oct 8, 2025
698c1b9
delete sample files
NHingerl Oct 8, 2025
fb5b0df
migrated architecture
a-thaler Oct 8, 2025
3eb43c0
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
a-thaler Oct 8, 2025
b578453
troubleshooting
a-thaler Oct 8, 2025
2b6c53e
add filtering docs
NHingerl Oct 8, 2025
75e19ec
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
NHingerl Oct 8, 2025
57dba81
renamed docs
a-thaler Oct 8, 2025
4de6da5
Merge branch 'main' into docs
a-thaler Oct 8, 2025
2efe997
adjusted links
a-thaler Oct 8, 2025
f33581a
renemd pipelines
a-thaler Oct 8, 2025
73fb78b
rephrase redirects
NHingerl Oct 8, 2025
8377e4f
fix link
NHingerl Oct 8, 2025
ec7a19a
fix link titles
NHingerl Oct 8, 2025
0edcd8f
fix links
NHingerl Oct 8, 2025
35edaab
a word too much
NHingerl Oct 8, 2025
09be9c1
a word more
NHingerl Oct 8, 2025
7784dde
integration guide
a-thaler Oct 9, 2025
c713f33
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
a-thaler Oct 9, 2025
67527db
finalized integration guide
a-thaler Oct 9, 2025
d2da808
fix
a-thaler Oct 9, 2025
3b0e4f2
fix readme
a-thaler Oct 9, 2025
f7014e3
adjust to Help Portal version
NHingerl Oct 9, 2025
a1e5601
align main README with Help Portal version
NHingerl Oct 9, 2025
1a47775
align pipeline doc with Help Portal
NHingerl Oct 9, 2025
81ea3a5
align otlp input doc with help portal
NHingerl Oct 9, 2025
8c7ab82
align Collecting Logs with HP
NHingerl Oct 9, 2025
4236dea
align application input with Help Portal
NHingerl Oct 9, 2025
71efd00
remove curl from prerequisites
NHingerl Oct 9, 2025
1edbfda
align istio logs with Help Portal
NHingerl Oct 9, 2025
683e5f9
align Limitations
NHingerl Oct 9, 2025
4976797
adjust Istio Traces
NHingerl Oct 9, 2025
88af9cd
align phrasing
NHingerl Oct 9, 2025
f06f1bd
align prometheus and istio metrics
NHingerl Oct 9, 2025
6819597
Update docs/user/integration/sap-cloud-logging/README.md
a-thaler Oct 10, 2025
dacfaed
Update docs/user/integration/sap-cloud-logging/README.md
a-thaler Oct 10, 2025
1ecc118
Merge branch 'main' into docs
a-thaler Oct 10, 2025
8ea5768
add missing metrics
a-thaler Oct 10, 2025
73dd02f
adjusted bufferFillingUp
a-thaler Oct 10, 2025
e23df79
Merge branch 'main' into docs
a-thaler Oct 10, 2025
b5a5583
align diagnostic metrics descriptions
NHingerl Oct 13, 2025
7b0d22a
align Filter Metrics with HP
NHingerl Oct 13, 2025
d5b0ed2
align OTLP Log Transofrmation with HP doc
NHingerl Oct 13, 2025
713d8a1
align Data Enrichment with HP
NHingerl Oct 13, 2025
724837c
update otlp output doc
NHingerl Oct 13, 2025
05c338a
align the CLS integration guide
NHingerl Oct 13, 2025
0b9af3e
align migration guide with Help Portal
NHingerl Oct 13, 2025
8b19f15
align Istio Integration with HP
NHingerl Oct 13, 2025
51e72de
align Architecture with HP docs
NHingerl Oct 13, 2025
5adb6c7
update troubleshooting
NHingerl Oct 13, 2025
faae359
drop backticks for k8s resource names
NHingerl Oct 13, 2025
1596d1d
improved troubleshooting
a-thaler Oct 13, 2025
16d7f7c
fix
a-thaler Oct 13, 2025
e34d3ce
adjust metrics architecture
a-thaler Oct 13, 2025
a671d84
Merge branch 'main' into docs
a-thaler Oct 13, 2025
917d3e7
consistent use of "observability backend"
NHingerl Oct 13, 2025
4c58cf4
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
NHingerl Oct 13, 2025
cf87302
align with help portal doc
NHingerl Oct 13, 2025
53060e1
Apply suggestions from code review
a-thaler Oct 14, 2025
2b35c42
consistent into sections for input docs
NHingerl Oct 14, 2025
51edf25
format attributes/parameters as bold font (https://github.com/kyma-pr…
NHingerl Oct 14, 2025
de0d4e4
add links for Istio Telemety CRD
NHingerl Oct 14, 2025
b1df368
add lost sentence for Istio Tracing
NHingerl Oct 14, 2025
a2d096c
adjust ambiguous sentence
NHingerl Oct 14, 2025
8320aea
adjust sidebar
NHingerl Oct 14, 2025
8d84a87
language review
NHingerl Oct 14, 2025
01eebdc
update stages diagram
NHingerl Oct 15, 2025
9de5467
delete backup diagram
NHingerl Oct 15, 2025
ea0f674
fix diagram
NHingerl Oct 15, 2025
770ffcc
update pipeline structure
NHingerl Oct 15, 2025
2e82d0d
update OTLP Input and Telemetry Stages
NHingerl Oct 15, 2025
1b08cf5
review otlp output
NHingerl Oct 15, 2025
92a6c79
update CLS diagram
NHingerl Oct 15, 2025
1e1afe4
update telemetry architecture diagrams
NHingerl Oct 15, 2025
eb81852
update architecture diagrams
NHingerl Oct 15, 2025
8588c6d
Merge branch 'main' into docs
a-thaler Oct 16, 2025
75e1671
delete backup files
a-thaler Oct 16, 2025
7b558e0
review diagrams
a-thaler Oct 16, 2025
f739ce2
Merge branch 'main' into docs
a-thaler Oct 16, 2025
e36daef
review diagrams
a-thaler Oct 16, 2025
2a6d3da
fixes
a-thaler Oct 16, 2025
0621e9a
consistency in architecture
NHingerl Oct 16, 2025
3eefe59
font color
NHingerl Oct 16, 2025
8d0d331
fix link
a-thaler Oct 20, 2025
b2ea5d8
Merge branch 'main' into docs
a-thaler Oct 20, 2025
133ed90
fix
a-thaler Oct 20, 2025
7325ec4
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
a-thaler Oct 20, 2025
61655bd
Merge branch 'main' into docs
a-thaler Oct 22, 2025
d034770
move "app logs from system namespaces" to Filter topic, add link
NHingerl Oct 22, 2025
6fc1ae7
adjust formatting for namespace names and placeholders in Tracing
NHingerl Oct 22, 2025
d4b2dc6
system namespaces included, not excluded
NHingerl Oct 22, 2025
af21478
add Filter... link to metrics inputs
NHingerl Oct 22, 2025
023e40a
consistent namespace placeholders
NHingerl Oct 22, 2025
1512726
improve and simplify namespace filtering
NHingerl Oct 22, 2025
84ea0aa
consistent phrasing for OTLP filters
NHingerl Oct 22, 2025
23e4f5d
istio metrics exclude system namespaces by default
NHingerl Oct 22, 2025
19e282a
review
NHingerl Oct 22, 2025
086d037
align with help portal docs
NHingerl Oct 22, 2025
216dd7d
language review
NHingerl Oct 22, 2025
ef58c85
add a sentence
NHingerl Oct 22, 2025
d877973
Apply suggestions from code review
a-thaler Oct 23, 2025
308f3fb
switch order for application log filters
NHingerl Oct 23, 2025
eee6fbe
Merge branch 'main' into docs
a-thaler Oct 27, 2025
b1a23ea
TW review
NHingerl Oct 27, 2025
fd440e0
update with G's review
NHingerl Oct 27, 2025
880df04
remove duplicate entry inn sidebar
NHingerl Oct 27, 2025
a14c814
update repo README
NHingerl Oct 27, 2025
b9e5dd3
update User README and resources README acc G's suggestions
NHingerl Oct 27, 2025
d4c4e24
fix typo in resources README
NHingerl Oct 27, 2025
43a3c12
fix error in resources readme
NHingerl Oct 27, 2025
ad5e879
no capital A
NHingerl Oct 27, 2025
49f0d95
apply Grzesiu's suggestions
NHingerl Oct 27, 2025
ebe5ec9
Apply Rakesh's HP suggestion about consistent resource name and note box
NHingerl Oct 27, 2025
e107823
fix formatting of notes, tips, and warnings
NHingerl Oct 27, 2025
5914af9
fix tip formatting
NHingerl Oct 27, 2025
4cf5f66
fix warning formatting
NHingerl Oct 27, 2025
c8d6ba2
rephrase column titles
NHingerl Oct 27, 2025
300566f
Merge branch 'main' into docs
a-thaler Oct 28, 2025
6e10e5b
test gitleaks
a-thaler Oct 28, 2025
96728c4
fix note and tip formatting
NHingerl Oct 29, 2025
8027bbe
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
NHingerl Oct 29, 2025
3311df4
fix
a-thaler Oct 29, 2025
00a2781
Merge branch 'main' into docs
a-thaler Oct 29, 2025
f562bad
removed orphaned gitleaks variable
a-thaler Oct 29, 2025
e7ebcbd
apply Grzesiu's suggestions
NHingerl Oct 29, 2025
04c653a
Merge branch 'docs' of https://github.com/a-thaler/telemetry-manager …
NHingerl Oct 29, 2025
6ff2ba4
Merge branch 'main' into docs
a-thaler Oct 29, 2025
1e5c7ec
Update docs/user/resources/README.md
NHingerl Oct 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,25 @@

## Overview

[Telemetry Manager](docs/user/01-manager.md) is a Kubernetes [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) that fulfils the [Kyma module interface](https://kyma-project.io/#/06-modules/README.md). It provides APIs for a managed agent/gateway setup for log, trace, and metric ingestion and dispatching into 3rd-party backend systems, in order to reduce the pain of orchestrating such setup on your own. Read more on the [manager](./docs/user/01-manager.md) itself or the general [usage](docs/user/README.md) of the module.
[Telemetry Manager](docs/user/01-manager.md) is a Kubernetes [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) that fulfils the [Kyma module interface](https://kyma-project.io/#/06-modules/README.md). It provides APIs for a managed agent/gateway setup for log, trace, and metric ingestion and dispatching into 3rd-party backend systems, in order to reduce the pain of orchestrating such setup on your own. Read more on the [Architecture](./docs/user/architecture/README.md) itself or the general [usage](docs/user/README.md) of the module.

### Logs

The logging controllers generate a Fluent Bit DaemonSet and configuration from one or more LogPipeline and LogParser custom resources. The controllers ensure that all Fluent Bit Pods run the current configuration by restarting Pods after the configuration has changed. See all [CRD attributes](apis/telemetry/v1alpha1/logpipeline_types.go) and some [examples](samples).
The logging controllers generate a [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) DaemonSet And Deployment and configuration from one or more LogPipeline custom resources. The controllers ensure that all OTel Collector Pods run the current configuration by restarting Pods after the configuration has changed. See all [CRD attributes](apis/telemetry/v1alpha1/logpipeline_types.go) and some [examples](./samples).

For more information, see [Logs](./docs/user/02-logs.md).
For more information, see [Logs](./docs/user/collecting-logs/README.md).

### Traces

The trace controller creates an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) deployment and related Kubernetes objects from a `TracePipeline` custom resource. The collector is configured to receive traces using the [OpenTelemetry Protocol (OTLP)](https://opentelemetry.io/docs/specs/otel/protocol/), and forwards the received traces to a configurable OTLP backend.
The trace controller creates an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) Deployment and related Kubernetes objects from a `TracePipeline` custom resource. The collector is configured to receive traces using the [OpenTelemetry Protocol (OTLP)](https://opentelemetry.io/docs/specs/otel/protocol/), and forwards the received traces to a configurable OTLP backend.

For more information, see [Traces](./docs/user/03-traces.md).
For more information, see [Traces](./docs/user/collecting-traces/README.md).

### Metrics

The metric controller creates an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) and related Kubernetes objects from a `MetricPipeline` custom resource. The collector is deployed as a [Gateway](https://opentelemetry.io/docs/collector/deployment/#gateway). The controller is configured to receive metrics in the OTLP protocol and forward them to a configurable OTLP backend.
The metric controller creates an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) DaemonSet And Deployment and related Kubernetes objects from a `MetricPipeline` custom resource. The collector is deployed as a [Gateway](https://opentelemetry.io/docs/collector/deployment/#gateway). The controller is configured to receive metrics in the OTLP protocol and forward them to a configurable OTLP backend.

For more information, see [Metrics](./docs/user/04-metrics.md).
For more information, see [Metrics](./docs/user/collecting-metrics/README.md).

## Installation

Expand Down
24 changes: 1 addition & 23 deletions docs/user/01-manager.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,3 @@
# Telemetry Manager

As the core element of the Telemetry module, Telemetry Manager manages the lifecycle of other Telemetry module components by watching user-created resources.

## Module Lifecycle

The Telemetry module includes Telemetry Manager, a Kubernetes [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) that's described by a custom resource of type Telemetry. Telemetry Manager has the following tasks:

1. Watch the module configuration for changes and sync the module status to it.
2. Watch for the user-created Kubernetes resources LogPipeline, TracePipeline, and MetricPipeline. In these resources, you specify what data of a signal type to collect and where to ship it.
3. Manage the lifecycle of the self monitor and the user-configured agents and gateways.
For example, only if you defined a LogPipeline resource, the Fluent Bit DaemonSet is deployed as log agent.

![Manager](assets/manager-resources.drawio.svg)

### Self Monitor

The Telemetry module contains a self monitor, based on [Prometheus](https://prometheus.io/), to collect and evaluate metrics from the managed gateways and agents. Telemetry Manager retrieves the current pipeline health from the self monitor and adjusts the status of the pipeline resources and the module status.
Additionally, you can monitor the health of your pipelines in an integrated backend like [SAP Cloud Logging](./integration/sap-cloud-logging/README.md#use-sap-cloud-logging-alerts): To set up alerts and reports in the backend, use the [pipeline health metrics](./04-metrics.md#5-monitor-pipeline-health) emitted by your MetricPipeline.

![Self-Monitor](assets/manager-arch.drawio.svg)

## Module Configuration and Status

For configuration options and the overall status of the module, see the specification of the related [Telemetry resource](./resources/01-telemetry.md).
This content moved to [Architecture](./architecture/README.md).
7 changes: 5 additions & 2 deletions docs/user/02-logs.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# Application Logs (Fluent Bit)

> [!NOTE]
> The following API uses the LogPipeline output `http` and `custom` based on a FluentBit agent. As a new user, start with the approach based on OpenTelemetry: [Collecting Logs](./collecting-logs/README.md).

With application logs, you can debug an application and derive the internal state of an application. When logs are emitted with the correct severity level and context, they're essential for observing an application.

## Overview

The Telemetry module provides the [Fluent Bit](https://fluentbit.io/) log agent for the collection and shipment of application logs of any container running in the Kyma runtime.

You can configure the log agent with external systems using runtime configuration with a dedicated Kubernetes API ([CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)) named `LogPipeline`. With the LogPipeline's HTTP output, you can natively integrate with vendors that support this output, or with any vendor using a [Fluentd integration](https://medium.com/hepsiburadatech/fluent-logging-architecture-fluent-bit-fluentd-elasticsearch-ca4a898e28aa).
You can configure the log agent with external systems using runtime configuration with a dedicated Kubernetes API ([CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)) named `LogPipeline`. With the LogPipeline's HTTP output, you can natively integrate with vendors that support this output, or with any vendor using a Fluentd integration.

The feature is optional, if you don't want to use the Logs feature, simply don't set up a LogPipeline.

Expand Down Expand Up @@ -478,7 +481,7 @@ The enriched timestamp attributes have the following meaning:

The Telemetry module ensures that the log agent instances are operational and healthy at any time, for example, with buffering and retries. However, there may be situations when the instances drop logs, or cannot handle the log load.

To detect and fix such situations, check the [pipeline status](./resources/02-logpipeline.md#logpipeline-status) and check out [Troubleshooting](#troubleshooting). If you have set up [pipeline health monitoring](./04-metrics.md#5-monitor-pipeline-health), check the alerts and reports in an integrated backend like [SAP Cloud Logging](./integration/sap-cloud-logging/README.md#use-sap-cloud-logging-alerts).
To detect and fix such situations, check the [pipeline status](./resources/02-logpipeline.md#logpipeline-status) and check out [Troubleshooting](#troubleshooting). If you have set up [pipeline health monitoring](./monitor-pipeline-health.md), check the alerts and reports in an integrated backend like [SAP Cloud Logging](./integration/sap-cloud-logging/README.md#use-sap-cloud-logging-alerts).

> [!WARNING]
> It's not recommended to access the metrics endpoint of the used FluentBit instances directly, because the exposed metrics are no official API of the Kyma Telemetry module. Breaking changes can happen if the underlying FluentBit version introduces such.
Expand Down
Loading