ECS 1.7.0

Experimental Changes

A few months ago, we introduced the RFC process. This process is meant to fully vet big additions or changes to ECS. A key aspect of this process is that proposals advance in stages. Each stage represents the vetting and maturity of the proposal.

We won’t go over the process in detail here, but one of its key aspects is that accepted “stage 2” proposals appear in “experimental” ECS artifacts. They don’t yet appear officially in ECS documentation. Proposals that reach “stage 3” are the ones that will officially appear in ECS documentation.

ECS 1.7 is the first release that includes RFCs that have reached stage 2 / experimental changes. A new directory has therefore been added, where all the usual generated artifacts are published including the experimental changes. This is at experimental/generated.

This release includes experimental changes from two RFCs reaching stage 2:

• Replace the keyword type on many existing ECS fields with the new wildcard type.
• Adding more places where user fields can be nested, in order to capture privilege escalations & demotions as well as IAM. These experimental nestings are user.effective.*, user.target.*, and user.changes.*.

“Normal” Changes

Contrary to the new experimental changes described above, the following changes are reflected in the documentation.

Two new fields are introduced: http.[request|response].mime_type/ and threat.technique.subtechnique.

Both the network.direction and event.category fields add support for additional allowed values.

The ECS generator script adds two new arguments, --oss and --strict. See usage for more details and examples.

Lastly, we have changed the index pattern of the sample Elasticsearch template from ecs-* to try-ecs-* to avoid conflicting with Logstash' template when run in ECS compatibility mode.

Changelog

Schema Changes

Bugfixes

• The protocol allowed value under event.type should not have the expected_event_types defined. #964
• Clarify the definition of file.extension (no dots). #1016

Added

• Added Mime Type fields to HTTP request and response. #944
• Added network directions ingress and egress. #945
• Added threat.technique.subtechnique to capture MITRE ATT&CK® subtechniques. #951
• Added configuration as an allowed event.category. #963
• Added a new directory with experimental artifacts, which includes all changes
  from RFCs that have reached stage 2. #993, #1053, #1115, #1117, #1118

Improvements

• Expanded field set definitions for source.* and destination.*. #967
• Provided better guidance for mapping network events. #969
• Added the field .subdomain under client, destination, server, source
  and url, to match its presence at dns.question.subdomain. #981
• Clarified ambiguity in guidance on how to use x509 fields for connections with
  only one certificate. #1114

Tooling and Artifact Changes

Breaking changes

• Changed the index pattern of the sample Elasticsearch template from ecs-* to
  try-ecs-* to avoid conflicting with Logstash' ecs-logstash-*. #1048

Bugfixes

• Addressed issue where foreign reuses weren't using the user-supplied as value for their destination. #960
• Experimental artifacts failed to install due to event.original index setting. #1053

Added

• Introduced --strict flag to perform stricter schema validation when running the generator script. #937
• Added check under --strict that ensures composite types in example fields are quoted. #966
• Added ignore_above and normalizer support for keyword multi-fields. #971
• Added --oss flag for users who want to generate ECS templates for use on OSS clusters. #991

Improvements

• Field details Jinja2 template components have been consolidated into one template #897
• Add [discrete] marker before each section header in field details. #989
• --ref now loads experimental/schemas based on git ref in addition to schemas. #1063