v11.3.0

Features

• add gzip handling to http_input connector
• adds advanced logging configuration
  • add configurable log format
  • add configurable datetime formate in logs
  • makes hostname available in custom log formats
  • add fine grained log level configuration for every logger instance

Improvements

• rename logprep.event_generator module to logprep.generator
• shorten logger instance names

Bugfix

• fixes exposing OpenSearch/ElasticSearch stacktraces in log when errors happen by making loglevel configurable for loggers opensearch and elasticsearch
• fixes the logprep quickstart profile

Details

• Handle Opensearch Stacktraces by @dtrai2 in #577
• add gzip handling to http_input by @ekneg54 in #581
• Add Flush Timeout to s3 Connector by @saegel in #553
• rename event generator module by @ekneg54 in #585
• advanced logging configuration by @ekneg54 in #584
• fix logprep quickstart profile by @ekneg54 in #586
• prepare release 11.3.0 by @ekneg54 in #588

Full Changelog: v11.2.1...v11.3.0

v11.2.1

Bugfix

• fixes bug, that leads to spawning exporter http server always on localhost

Details

• fix bug prometheus exporter server uvicorn only on localhost by @ekneg54 in #580

Full Changelog: v11.2.0...v11.2.1

v11.2.0

Features

• expose metrics via uvicorn webserver
  • makes all uvicorn configuration options possible
  • add security best practices to server configuration
• add following metrics to http_input connector
  • nummer_of_http_requests
  • message_backlog_size

Bugfix

• fixes a bug in grokker rules, where common field prefixes wasn't possible
• fixes bug where missing key in credentials file leads to AttributeError

Details

• Fix AttributeError due to missing key in credentials file by @djkhl in #574
• Fix grokker rules that have fields with common prefixes by @dtrai2 in #571
• export metrics via uvicorn asgi app by @ekneg54 in #576
• add http input connector metrics by @ekneg54 in #578

Full Changelog: v11.1.0...v11.2.0

v11.1.0

11.1.0

Features

• new documentation part with security best practices which compiles to user_manual/security/best_practices.html
  • also comes with excel export functionality of given best practices
• add basic auth to http_input

Bugfix

• fixes a bug in http connector leading to only first process working
• fixes the broken gracefull shutdown behaviour

Details

• fix http connector only first process working by @ekneg54 in #566
• Add overview with best practices to docs by @dtrai2 in #559
• fix broken gracefull shutdown by @ekneg54 in #568
• Add Basic Auth to HTTP Connector by @herrfeder in #569

Full Changelog: v11.0.1...v11.1.0

v11.0.1

Bugfix

• fixes a bug where the pipeline index increases on every restart of a failed pipeline
• fixes closed log queue issue by run logging in an extra process

Details

• Fix mtls credentials warning by @djkhl in #558
• remove fastapi from depenencies by @ekneg54 in #557
• make pipeline index stable by @ekneg54 in #562
• fix multiprocessing logging by @ekneg54 in #563

Full Changelog: v11.0.0...v11.0.1

v11.0.0

Breaking

• configuration of Authentication for getters is now done by new introduced credentials file

Features

• introducing an additional file to define the credentials for every configuration source
• retrieve oauth token automatically from different oauth endpoints
• retrieve configruation with mTLS authentication
• reimplementation of HTTP Input Connector with following Features:
  • Wildcard based HTTP Request routing
  • Regex based HTTP Request routing
  • Improvements in thread-based runtime
  • Configuration and possibility to add metadata

Improvements

• remove versioneer dependency in favor of setuptools-scm

Bugfix

• fix version string of release versions
• fix version string of container builds for feature branches
• fix merge of config versions for multiple configs

Details

• fix documentation examples by @ekneg54 in #548
• Update FDA to version 0.21.0 in quickstart example by @clumsy9 in #544
• Automatic oauth2 token retrieval by @dtrai2 in #531
• exchange version string generation from versioneer to setuptools-scm by @ekneg54 in #549
• fix merge of version string for multiple configs by @ekneg54 in #556
• Expand authentication with mTLS by @djkhl in #554
• Reimplementation HTTP Connector by @herrfeder in #538

Full Changelog: v10.0.4...v11.0.0

v10.0.4

Improvements

• refactor logprep build process and requirements management

Bugfix

• fix generic_adder not creating new field from type list

Details

• pyproject.toml dependency and build managment by @ekneg54 in #526
• Fix FatalOutputError caused by serializing KafkaError by @clumsy9 in #536
• fix generic adder not creating initial list field by @ekneg54 in #540
• prepare release 10.0.4 by @djkhl in #541
• fix pyproject.toml by @ekneg54 in #542

Full Changelog: v10.0.3...v10.0.4

v10.0.3

Bugfix

• fix loading of configuration inside the AutoRuleCorpusTester for logprep test integration
• fix auto rule tester (test unit), which was broken after adding support for multiple configuration files and resolving paths in configuration files

Details

• remove multiple pipeline tests for http_input by @ekneg54 in #525
• fix call auto_rule_corpus_tester with tuple instead of str by @ekneg54 in #527
• Fix auto rule tester by @ppcad in #530
• Prepare Release 10.0.3 by @djkhl in #532

Full Changelog: v10.0.2...v10.0.3

v10.0.2

Bugfix

• fix versioneer import
• fix logprep does not complain about missing PROMETHEUS_MULTIPROC_DIR

Details

• Fix versioneer and PROMETHEUS_MULTIPROC_DIR bug by @djkhl in #524

Full Changelog: v10.0.1...v10.0.2

v10.0.1

v10.0.1

Bugfix

• fix entrypoint in setup.py that corrupted the install

Details

• fix setup.py by @djkhl in #523

Full Changelog: v10.0.0...v10.0.1