Pysigma-backend-secops

Bug Fixes
- Fixed parenthesis/grouping for various conditions

Major Changes
- Added `!=` expression in place of NOT expressions
- Using `NOT` with regex caused inconsistent results when compared to using `!=`, so all `NOT`s have been swapped with `!=`

Major Changes
- Improved regex handling in UDM searches:
- Removed unnecessary leading/trailing `.*` patterns
- Added proper forward slash escaping
- Fixed case sensitivity handling

New Features
- Added field mappings for:
- Grandparent process fields
- Common hash algorithm fields outside of `Hashes` (md5, sha1, sha256, sha512, imphash)

Technical Improvements
- Fixed NOT operation handling in UDM searches
- Fixed IN expression conversion for UDM compatibility
- Improved regex escaping in command line arguments

- Fixed error when using `yara_l` output format with `sigma-cli`

📌 Summary
This release marks a significant milestone for the pySigma Google SecOps (Chronicle) Backend, introducing major improvements in event type determination, field mappings, and output formats. We've enhanced the backend's ability to generate more accurate and flexible queries, while also introducing support for YARA-L 2.0 output.

🚀 New Features

🧠 Advanced Event Type Determination
- Implemented `SetRuleEventTypeFromLogsourceTransformation` and `SetRuleEventTypeFromEventIDTransformation`
- Improved logic to determine event types based on logsource categories and EventIDs

🗺️ Dynamic Field Mapping
- Introduced `get_field_mappings_by_event_type` function for more flexible field mappings
- Added support for various event types including process, network, file, authentication, and registry events

📤 YARA-L 2.0 Output Support
- Added new `yara_l_pipeline()` for generating YARA-L 2.0 format output
- Implemented `YaraLPostprocessingTransformation` for formatting YARA-L rules

🔄 Enum Value Conversion
- New `ConvertEnumValueTransformation` to map enum values to their UDM equivalents

🔧 Improvements

📊 Pipeline Enhancements
- Added `PrependMetadataPostprocessingTransformation` for more flexible query generation
- Implemented `SetPrependMetadataTransformation` to control metadata prepending
- New `RemoveHashAlgoFromValueTransformation` for cleaning up hash fields

🧹 Code Cleanup and Optimization
- Refactored and optimized various utility functions
- Improved overall code structure and readability

🐛 Error Handling
- Enhanced error reporting for invalid UDM fields

🧪 Testing

- Added comprehensive test suite for the SecOps pipeline
- Expanded backend tests to cover new functionalities, including YARA-L output

📚 Documentation

- Updated README with new features and usage examples
- Added more detailed comments and docstrings throughout the codebase

🔮 Coming Soon

- More robust field mapping logic
- Enhanced YARA-L output with improved readability and structure

Introduction

This is the initial release (v0.0.2) of the pySigma backend for Google SecOps, formally Chronicle. It is still in active development, so use at your own risk. See [Development Status](https://github.com/AttackIQ/pySigma-backend-secops/blob/main/README.md#development-status) for current/planned development items. This release includes the following items:

Features

Core Functionality
- **SecOpsBackend Class**
- Located in `sigma.backends.secops`.
- Converts Sigma rules to Google SecOps UDM queries.

Processing Pipelines
- **secops_udm_pipeline**
- Converts Sigma rules into Google SecOps UDM format.
- Located in `sigma.pipelines.secops`.

Output Formats
- **Default:** Generates plain Google SecOps queries.
- **YARA-L v2.0:** *(In Progress)* Adds support for YARA-L v2.0 output.

Enhanced Matching
- **Regex Customization:** Supports `contains`, `startswith`, `endswith`, and other regex-based matching.
- **Case Insensitive Matching:** Implements `nocase` for case-insensitive rule matching.

Schema Tools
- **UDM Schema Parser and Validator**
- **Validators:**
- Located in `sigma/pipelines/secops/validators.py`.
- Provides functions to validate field paths and values against the UDM schema.
- **Schema Extractor:**
- Located in `utils/get_field_schema_from_docs.py`.
- Parses and extracts the UDM schema from Google's documentation to ensure accurate field definitions.

Testing
- **Backend Testing:** Comprehensive tests to ensure reliability and functionality.
- **Schema Validation Testing:** Comprehensive tests to ensure converted Sigma fields and enums are valid for UDM
---

**Note:** This alpha release is intended for testing and feedback purposes. Features may change, and the backend is not yet stable for production use.