Sigmaiq

New Features
* Added Google SecOps (Chronicle) backend support with UDM pipeline
* Fixed LLM dependencies as optional install:
bash
pip install sigmaiq[llm]


Improvements
* Added automatic Sigma v1 to v2 schema conversion util
* Enhanced handling of nested SigmaCollections
* Updated pipeline resolver to handle None values more gracefully

Infrastructure
* Added pytest configuration and async test support
* Added VSCode and test files to gitignore

Dependencies
* Updated pySigma to 0.11.18
* Updated various backend dependencies to latest versions
* Added pytest-asyncio for testing

- Pinned latest pySigma version (v0.11.17) in `pyproject.toml` to fix pyparsing import error found in previous pySigma version
- Fixes 12

🚀 Major Updates

Dependency Upgrades
- Upgraded to Python 3.9+ support (previously 3.8.1+)
- Updated numerous core dependencies to latest versions, including:
- pysigma (0.11.14)
- pysigma backends and pipelines
- langchain (0.2.16)
- openai

New Backends
- Added Kusto backend support:
- Microsoft Defender XDR
- Microsoft Sentinel ASIM
- Azure Monitor
- Added Netwitness backend

Enhanced Crowdstrike Support
- Added Crowdstrike Logscale backend
- Updated Crowdstrike Splunk backend to use FDR pipeline

🔧 Improvements & Changes

Backend Refinements
- Removed deprecated Microsoft365Defender backend
- Updated Elasticsearch backend to support additional pipelines:
- ecs_kubernetes
- ecs_windows_old
- ecs_zeek_beats
- ecs_zeek_corelight
- zeek_raw

LLM Module Enhancements
- Expanded README with detailed feature descriptions and usage guidelines
- Default LLM model changed to gpt-4o

🐛 Bug Fixes
- Various minor bug fixes and code improvements

📚 Documentation
- Updated installation instructions and requirements
- Enhanced LLM module documentation with examples and known issues

🛠 Development Tools
- Updated development dependencies (pytest, black, ruff)
- Refined project configuration (pyproject.toml, ruff settings)

---

This release significantly enhances SigmAIQ's capabilities, especially in backend support and LLM integration. Users are encouraged to review the updated documentation for new features and potential breaking changes due to dependency updates.

What's Changed
* New LLM tool added to convert a SIEM/Product query into a Sigma Rule (a.k.a reverse conversion)
* Default LLM models have been updated from `gpt-3.5-turbo` to `gpt-4o`
* Rule Creation prompt has been updated
* Ensures better rules are created when user asks about threat group, malware activity
* Schema URL given to prompt and instructed to look it up if LLM is unsure of correct schema for rule output
* The Sigma Schema is already provided in the prompt, but this just gives it all the context it would need if required
* Created rules should now include the original author and related rule IDs if rules were used as context for creating the new rule. This is to ensure the detection rule license is enforced
* pySigma core version increased to v0.10.10. Backend and pipeline versions were increased to their maximum allowed versions for this pySigma version.

Upcoming
* pySigma will be updated to at least v0.11.3. Backends and pipelines will be updated to the latest allowed version with this change.
* This will also allow us to update `langchain` and the LLM libraries to the latest versions, due to a conflict with the `packaging` dependency pinned versions in `langchain` and `pysigma` that was fixed in `pysigma 0.11.3`.


**Full Changelog**: https://github.com/AttackIQ/SigmAIQ/compare/v0.3.0...v0.4.0

With this release, we've added LLM / OpenAI functionality! Here's some of the highlights:
- Added an rule updater to download the latest SigmaHQ Rule release
- Added base LLM class to create embeddings from downloaded rules and store in a local VectorStore
- Added simple similarity searching for Sigma Rules in a VectorStore from user input
- Added a langchain Toolkit and Tools for use with a langchain Agent/bot to perform the following:
- Automatically convert a Sigma Rule to any SigmAIQ supported backend, pipeline, and output format via user input
- Automatically create brand new Sigma Rules based on a user's input and similar rules in the VectorStore

This is still very much a work in progress, but we are excited to share this with the community and keep working on its development.

For more information, please see the LLM specific README [here](https://github.com/AttackIQ/SigmAIQ/blob/master/sigmaiq/llm/README.md)

- Fixed improper pipeline creation when setting new pipeline in created SigmAIQBackend object