Contentctl

This release fixes spacing issues that could appear in savedsearches.conf or es_investigations.conf and result in failed parsing of certain stanzas and failures/errors when running btool.
This issue was initially reported by a customer. Thank you to them for their feedback!

Support for the upcoming Enterprise Security

- Added support for an upcoming Enterprise Security, including the `actions.correlationsearch.metadata` field for version, date, and other relevant information.

Improved Validation and Performance

- Upgraded to Pydantic2 for enhanced object validation of detections, stories, lookups, and other content.
- Leveraged newer Pydantic field validation technology to introduce strongly typed content fields, eliminating string references.
- Moved many checks from runtime to validation time, allowing for faster error detection (3 seconds instead of 1-2 hour wait for failure).
- Implemented Atomic GUID validation and support for detection tests.
- Added strict checks for risk and threat object configuration, including type, allowed fields, and mandatory victim definition.
- Introduced notable and risk message validation during runtime when Enterprise Security triggers the detection.

Bug Fixes and Consistency Improvements

- Resolved inconsistencies between file names and detection names through programmatic enforcement.
- Fixed broken references to analytic stories and detections that were mistyped or no longer existed.
- Ensured all detections create correct risk and threat objects, addressing previous bugs.
- Removed backspace usage for new lines in the detection description, fixing rendering issues and eliminating the need for escape characters (\\).

Code Cleanup and Optimization

- Cleaned out old, duplicated code.
- Significantly improved execution speed across all content workflows, including validation and build generation.

Developer Experience Enhancements

- Upgraded to Python 3.12 for improved performance and compatibility.
- Added an app template folder for customization of the app generated by `contentctl init`, allowing users to add their own images, dashboards, panels, etc.
- Improved command line ergonomics and documentation for a better developer experience.

Miscellaneous Updates

- Improved release notes generation for easier tracking of changes between versions.

These updates aim to enhance the overall functionality, performance, and user experience of Contentctl, while addressing bugs and inconsistencies present in previous versions.

This version includes a small change that writes additional information to savedsearches.conf.
The file now includes a metadata field showing:
1. Whether or not a detection is deprecated
2. The current version of the detection
3. the UUID of the detection.

Here are some examples:



Please see the following link for more detail on the PR:
https://github.com/splunk/contentctl/pull/132#issuecomment-2073451293

This release includes improvements around Integration Testing Functionality and now properly respects the manual_test flag. Previously, detections labeled as manual_test would show as errors at the conclusion of the test, but they now render as SKIPPED.

Release note generation is also improved.

Remove verbose print which could expose sensitive arguments during ACS deploy.

Update how long we wait for the appinspect api to return results.
First, wait for 40 seconds, then check every subsequent second for the results (appinspects always take at least 40 seconds).
This reduces the wait time for an appinspect without creating undue stress on the appinspect endpoint.