Logprep

Latest version: v14.0.0

Safety actively analyzes 682404 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 2 of 8

12.0.0

Breaking

* `pseudonymizer` change rule config field `pseudonyms` to `mapping`
* `clusterer` change rule config field `target` to `source_fields`
* `generic_resolver` change rule config field `append_to_list` to `extend_target_list`
* `hyperscan_resolver` change rule config field `append_to_list` to `extend_target_list`
* `calculator` now adds the error tag `_calculator_missing_field_warning` to the events tag field instead of `_calculator_failure` in case of missing field in events
* `domain_label_extractor` now writes `_domain_label_extractor_missing_field_warning` tag to event tags in case of missing fields
* `geoip_enricher` now writes `_geoip_enricher_missing_field_warning` tag to event tags in case of missing fields
* `grokker` now writes `_grokker_missing_field_warning` tag to event tags instead of `_grokker_failure` in case of missing fields
* `requester` now writes `_requester_missing_field_warning` tag to event tags instead of `_requester_failure` in case of missing fields
* `timestamp_differ` now writes `_timestamp_differ_missing_field_warning` tag to event tags instead of `_timestamp_differ_failure` in case of missing fields
* `timestamper` now writes `_timestamper_missing_field_warning` tag to event tags instead of `_timestamper_failure` in case of missing fields
* rename `--thread_count` parameter to `--thread-count` in http generator
* removed `--report` parameter and feature from http generator
* when using `extend_target_list` in the `field manager`the ordering of the given source fields is now preserved
* logprep now exits with a negative exit code if pipeline restart fails 5 times
* this was implemented because further restart behavior should be configured on level of a system init service or container orchestrating service like k8s
* the `restart_count` parameter is configurable. If you want the old behavior back, you can set this parameter to a negative number
* logprep now exits with a exit code of 2 on configuration errors

Features

* add UCL into the quickstart setup
* add logprep http output connector
* add pseudonymization tools to logprep -> see: `logprep pseudo --help`
* add `restart_count` parameter to configuration
* add option `mode` to `pseudonymizer` processor and to pseudonymization tools to chose the AES Mode for encryption and decryption
* add retry mechanism to opensearch parallel bulk, if opensearch returns 429 `rejected_execution_exception`

Improvements

* remove logger from Components and Factory signatures
* align processor architecture to use methods like `write_to_target`, `add_field_to` and `get_dotted_field_value` when reading and writing from and to events
* required substantial refactoring of the `hyperscan_resolver`, `generic_resolver` and `template_replacer`
* change `pseudonymizer`, `pre_detector`, `selective_extractor` processors and `pipeline` to handle `extra_data` the same way
* refactor `clusterer`, `pre_detector` and `pseudonymizer` processors and change `rule_tree` so that the processor do not require `process` override
* required substantial refactoring of the `clusterer`
* handle missing fields in processors via `_handle_missing_fields` from the field_manager
* add `LogprepMPQueueListener` to outsource logging to a separate process
* add a single `Queuehandler` to root logger to ensure all logs were handled by `LogprepMPQueueListener`
* refactor `http_generator` to use a logprep http output connector
* ensure all `cached_properties` are populated during setup time

Bugfix

* make `--username` and `--password` parameters optional in http generator
* fixes a bug where `FileNotFoundError` is raised during processing

11.3.0

Features

* add gzip handling to `http_input` connector
* adds advanced logging configuration
* add configurable log format
* add configurable datetime formate in logs
* makes `hostname` available in custom log formats
* add fine grained log level configuration for every logger instance

Improvements

* rename `logprep.event_generator` module to `logprep.generator`
* shorten logger instance names

Bugfix

* fixes exposing OpenSearch/ElasticSearch stacktraces in log when errors happen by making loglevel configurable for loggers `opensearch` and `elasticsearch`
* fixes the logprep quickstart profile

11.2.1

Bugfix

* fixes bug, that leads to spawning exporter http server always on localhost

11.2.0

Features

* expose metrics via uvicorn webserver
* makes all uvicorn configuration options possible
* add security best practices to server configuration
* add following metrics to `http_input` connector
* `nummer_of_http_requests`
* `message_backlog_size`

Bugfix

* fixes a bug in grokker rules, where common field prefixes wasn't possible
* fixes bug where missing key in credentials file leads to AttributeError

11.1.0

Features

* new documentation part with security best practices which compiles to `user_manual/security/best_practices.html`
* also comes with excel export functionality of given best practices
* add basic auth to http_input

Bugfix

* fixes a bug in http connector leading to only first process working
* fixes the broken gracefull shutdown behaviour

11.0.1

Bugfix

* fixes a bug where the pipeline index increases on every restart of a failed pipeline
* fixes closed log queue issue by run logging in an extra process

Page 2 of 8

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.