Skip to content

Commit c68961b

Browse files
committed
docs: update docs for loki.process with regex fields
1 parent 2e77c56 commit c68961b

File tree

2 files changed

+16
-11
lines changed

2 files changed

+16
-11
lines changed

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ Main (unreleased)
2222

2323
- Add htpasswd file based authentication for `otelcol.auth.basic` (@pkarakal)
2424

25+
- Add a `regex` argument to the `logfmt` and `json` stages in `loki.process` to extract log fields matching a regular expression. (@timonegk)
26+
2527
### Enhancements
2628

2729
- update promtail converter to use `file_match` block for `loki.source.file` instead of going through `local.file_match`. (@kalleep)

docs/sources/reference/components/loki/loki.process.md

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -496,13 +496,16 @@ The following arguments are supported:
496496

497497
| Name | Type | Description | Default | Required |
498498
| ---------------- | ------------- | ----------------------------------------------------- | ------- | -------- |
499-
| `expressions` | `map(string)` | Key-value pairs of JMESPath expressions. | | yes |
499+
| `expressions` | `map(string)` | Key-value pairs of JMESPath expressions. | | no |
500+
| `regex` | `string` | Regular expression matched against JSON keys. | | no |
500501
| `drop_malformed` | `bool` | Drop lines whose input can't be parsed as valid JSON. | `false` | no |
501502
| `source` | `string` | Source of the data to parse as JSON. | `""` | no |
502503

503504
The `expressions` field is the set of key-value pairs of JMESPath expressions to run.
504505
The map key defines the name with which the data is extracted, while the map value is the expression used to populate the value.
505506

507+
The `regex` field is a regular expression. All keys in the JSON source matching the regular expression are extracted.
508+
506509
When configuring a JSON stage, the `source` field defines the source of data to parse as JSON.
507510
By default, this is the log line itself, but it can also be a previously extracted value.
508511

@@ -653,19 +656,19 @@ The `stage.logfmt` inner block configures a processing stage that reads incoming
653656

654657
The following arguments are supported:
655658

656-
| Name | Type | Description | Default | Required |
657-
| --------- | ------------- | ---------------------------------------------- | ------- | -------- |
658-
| `mapping` | `map(string)` | Key-value pairs of `logmft` fields to extract. | | yes |
659-
| `source` | `string` | Source of the data to parse as `logfmt`. | `""` | no |
659+
| Name | Type | Description | Default | Required |
660+
| --------- | ------------- | ----------------------------------------------- | ------- | -------- |
661+
| `mapping` | `map(string)` | Key-value pairs of `logmft` fields to extract. | | no |
662+
| `regex` | `string` | Regular expression matched against logfmt keys. | | no |
663+
| `source` | `string` | Source of the data to parse as `logfmt`. | `""` | no |
660664

661-
The `source` field defines the source of data to parse as `logfmt`.
662-
When `source` is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value.
665+
The `mapping` field is the set of key-value pairs.
666+
The map key defines the name with which the data is extracted, while the map value is the logfmt field used to populate the value.
663667

664-
This stage uses the [go-logfmt][] unmarshaler, so that numeric or boolean types are unmarshalled into their correct form.
665-
The stage doesn't perform any other type conversions.
666-
If the extracted value is a complex type, it's treated as a string.
668+
The `regex` field is a regular expression. All logfmt fields matching the regular expression are extracted.
667669

668-
[go-logfmt]: https://github.com/go-logfmt/logfmt
670+
The `source` field defines the source of data to parse as `logfmt`.
671+
When `source` is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value.
669672

670673
The following log line and stages demonstrates how this works.
671674

0 commit comments

Comments
 (0)