Osquery

[Git Commits](https://github.com/osquery/osquery/compare/5.12.1...5.12.2)

This release is a hot fix. It reverts 8233, which had inadvertently broken ATC tables under some conditions.

Representing commits from 3 contributors! Thank you all.

Bug Fixes

- Revert Don't add ATC table name to registry until after sqlite DB initialization 8233 ([8334](https://github.com/osquery/osquery/pull/8334))

Build

- CI: Fix macOS python dependencies install step ([8308](https://github.com/osquery/osquery/pull/8308))

<a name="5.12.1"></a>

[Git Commits](https://github.com/osquery/osquery/compare/5.11.0...5.12.1)

Representing commits from 11 contributors! Thank you all.

New Features

- New flag `logger_tls_backoff_max` to configure the retry backoff for TLS logger plugin ([8230](https://github.com/osquery/osquery/pull/8230))

Table Changes

- Port the `battery` table to Windows ([8267](https://github.com/osquery/osquery/pull/8267))
- Update `homebrew_packages` table to include Casks ([8276](https://github.com/osquery/osquery/pull/8276))
- Update `cpu_info` to include `load_percentage` on windows ([8275](https://github.com/osquery/osquery/pull/8275))
- Check path exists first in `vscode_extensions` ([8292](https://github.com/osquery/osquery/pull/8292))
- `deb_packages` to ignore non existent admindirs ([8288](https://github.com/osquery/osquery/pull/8288))
- Add missing path separator in Safari Extensions table generator ([8273](https://github.com/osquery/osquery/pull/8273))
- Add windows UBR to `os_version` table ([8265](https://github.com/osquery/osquery/pull/8265))

Under the Hood improvements

- Persist query performance stats ([8250](https://github.com/osquery/osquery/pull/8250))
- Deprecate `worker_threads` flag ([8278](https://github.com/osquery/osquery/pull/8278))
- Change message from warning to error when extension could not be loaded ([8260](https://github.com/osquery/osquery/pull/8260))
- Refactor macOS system profile report retrieval ([8251](https://github.com/osquery/osquery/pull/8251))
- Clear performance stats when modifying scheduled/pack query ([8239](https://github.com/osquery/osquery/pull/8239))

Bug Fixes

- Fix version collate returning incorrect value when last character is a delimiter ([8283](https://github.com/osquery/osquery/pull/8283))
- Fix a memory leak in `unified_log` ([8274](https://github.com/osquery/osquery/pull/8274))
- Don't add ATC table name to registry until after sqlite DB initialization ([8233](https://github.com/osquery/osquery/pull/8233))

Documentation

- Update Jinja dependency for docs ([8285](https://github.com/osquery/osquery/pull/8285))
- Remove Zercurity from fleet managers list ([8293](https://github.com/osquery/osquery/pull/8293))
- Fix missing spaces in `kernel_keys` column descriptions ([8289](https://github.com/osquery/osquery/pull/8289))
- Update description for amperage in battery table. ([8253](https://github.com/osquery/osquery/pull/8253))

Packs

- Fix packs to check for platform before including queries ([7461](https://github.com/osquery/osquery/pull/7461))

Build

- Downgrade sqlite to 3.42 to prevent a regression with required columns ([8295](https://github.com/osquery/osquery/pull/8295))
- cve: Remove libxml2 dependency ([8282](https://github.com/osquery/osquery/pull/8282))
- cve: Update libexpat to 2.6.0 ([8281](https://github.com/osquery/osquery/pull/8281))
- cve: Update sqlite to 3.45.0 ([8259](https://github.com/osquery/osquery/pull/8259))
- cve: Update openssl to 3.2.1 ([8262](https://github.com/osquery/osquery/pull/8262))
- ci: Use all available cores and print more stats ([8248](https://github.com/osquery/osquery/pull/8248))
- cmake: Pass the osquery python path to googletest ([8237](https://github.com/osquery/osquery/pull/8237))
- test: Fix vscodeExtensions.test_sanity test ([8236](https://github.com/osquery/osquery/pull/8236))
- cmake: Correct typo, semvar -> semver ([8234](https://github.com/osquery/osquery/pull/8234))

<a name="5.11.0"></a>

[Git Commits](https://github.com/osquery/osquery/compare/5.10.2...5.11.0)

Representing commits from 11 contributors! Thank you all.

Table Changes

- Add new table `vscode_extensions` ([8150](https://github.com/osquery/osquery/pull/8150))
- Add support for additional Apple Silicon columns in `secureboot` table ([8215](https://github.com/osquery/osquery/pull/8215))
- Add Shortcut metadata parsing on Windows in the `file` table ([8143](https://github.com/osquery/osquery/pull/8143))
- Remove `atom_packages` table ([8181](https://github.com/osquery/osquery/pull/8181))
- Add additional chrome extensions paths ([8170](https://github.com/osquery/osquery/pull/8170)) to pick up extensions for Chrome Beta, Chrome Dev, and Vivaldi.

Under the Hood improvements

- Add version collations to column definitions ([8222](https://github.com/osquery/osquery/pull/8222))
- Add support for additional collations in column definitions ([8214](https://github.com/osquery/osquery/pull/8214))
- Add version collate functions ([8168](https://github.com/osquery/osquery/pull/8168))
- Added cache and throttling for `certificates`, `keychain_acls`, and `keychain_items` tables ([8192](https://github.com/osquery/osquery/pull/8192)). This is intended to reduce the occurrence of keychain corruption due to broken macOS APIs.
- process_open_sockets: Mark pid column as additional instead of index ([8191](https://github.com/osquery/osquery/pull/8191))

Bug Fixes

- Add stricter checks to JSON parsing ([8229](https://github.com/osquery/osquery/pull/8229))
- Fix signed/unsigned mismatch in powershell_events ([8225](https://github.com/osquery/osquery/pull/8225))
- Fix a crash in firefox_addons ([8227](https://github.com/osquery/osquery/pull/8227))
- Correct the aws_sts_region behavior ([8184](https://github.com/osquery/osquery/pull/8184))

Documentation

- Update building.md prereqs for Windows ([8216](https://github.com/osquery/osquery/pull/8216))
- Correct link to a PR in the 4.7.0 changelog ([8186](https://github.com/osquery/osquery/pull/8186))
- Call out in the CHANGELOG the format changes of the status logs decorations ([8174](https://github.com/osquery/osquery/pull/8174))
- Remove some duplicated lines from 5.8.1 changelog ([8172](https://github.com/osquery/osquery/pull/8172))
- Fix typo in table specs ([8163](https://github.com/osquery/osquery/pull/8163))
- Keychain cache and throttling documentation. ([8205](https://github.com/osquery/osquery/pull/8205))
- Changelog 5.10.2 ([8171](https://github.com/osquery/osquery/pull/8171))


Build / Dependencies

- Update libxml2 to v2.12.3 ([8223](https://github.com/osquery/osquery/pull/8223))
- Update zlib to 1.3 and ignore a CVE ([8218](https://github.com/osquery/osquery/pull/8218))
- Update openssl to 3.2.0 ([8212](https://github.com/osquery/osquery/pull/8212))
- Update nvdlib to use the latest NVD APIs ([8207](https://github.com/osquery/osquery/pull/8207))
- Fix Linux build ([8208](https://github.com/osquery/osquery/pull/8208))
- Correct job order ([8185](https://github.com/osquery/osquery/pull/8185))
- Re-enable tools_tests_testrelease ([8221](https://github.com/osquery/osquery/pull/8221))
- Enable client certificate verification in the TLS tests ([8211](https://github.com/osquery/osquery/pull/8211))
- Temporary workaround to build with XCode 15 ([8197](https://github.com/osquery/osquery/pull/8197))


<a name="5.10.2"></a>

[Git Commits](https://github.com/osquery/osquery/compare/5.9.1...5.10.2)

This release has several updates and bugfixes. Several improvements to various tables, and their handling.

One potential breaking change, is in how [the watchdog calculates CPU utilization](https://github.com/osquery/osquery/pull/8104).
Previously, this calculation was based on physical CPUs, now it is based on virtual cores. We believe this makes more sense with modern CPUs.

A second potential breaking change, is in PR [8102](https://github.com/osquery/osquery/pull/8102). In addition to allowing decorations to the top level of the status logs, this PR normalizes the decorations format to the results log. In practice, this means that the `unixTime`, `severity` and `line` JSON fields are now numbers instead of strings.

Representing commits from 18 contributors! Thank you all.

New Features

- Add `--enable_watchdog_debug` flag and improve watchdog error messages ([8070](https://github.com/osquery/osquery/pull/8070))
- Add `--aws_enforce_fips` to enforce AWS FIPS endpoints ([8075](https://github.com/osquery/osquery/pull/8075))
- Add new AWS valid regions ([8110](https://github.com/osquery/osquery/pull/8110))
- Implement `decorations_top_level` flag for status logs ([8102](https://github.com/osquery/osquery/pull/8102))

Table Changes

- Add new macOS SIP config flags ([8101](https://github.com/osquery/osquery/pull/8101))
- Added `cloud`_id to `ycloud_instance_metadata` - the vm metadata table for Yandex Cloud ([8086](https://github.com/osquery/osquery/pull/8086))
- Allow querying of kernel and filesystem drivers ([8119](https://github.com/osquery/osquery/pull/8119))
- Update `es_process_file_events` adding support for open events, and for only triggering on `file_paths` ([8114](https://github.com/osquery/osquery/pull/8114))
- Update `firefox_addons` to use rapidjson to parse and don't block on read ([8089](https://github.com/osquery/osquery/pull/8089))
- Update macOS `es_process_events` table: quote spaces in command line and environment variables ([8054](https://github.com/osquery/osquery/pull/8054))
- Update linux `disk_encryption` to recursively query parent crypt status ([8052](https://github.com/osquery/osquery/pull/8052))
- Add, and revert, indexing on `block_devices` ([8037](https://github.com/osquery/osquery/pull/8037), [#8151](https://github.com/osquery/osquery/pull/8151))

Under the Hood improvements

- Add warnings when an enrollment secret cannot be found ([8082](https://github.com/osquery/osquery/pull/8082))
- Avoid blocking when reading plist files ([8099](https://github.com/osquery/osquery/pull/8099))
- Fix named virtual table create statement ([8139](https://github.com/osquery/osquery/pull/8139))
- Remove forensicReadFile ([8085](https://github.com/osquery/osquery/pull/8085))
- Substitute the TEXT macro with SQL_TEXT in table code ([8091](https://github.com/osquery/osquery/pull/8091))
- Use JSON member iterator instead of rescanning ([8122](https://github.com/osquery/osquery/pull/8122))
- core: Avoid checking if a file exists before opening ([8087](https://github.com/osquery/osquery/pull/8087))
- improvement: Avoid unnecessary string conversions ([8093](https://github.com/osquery/osquery/pull/8093))
- watchdog: Use virtual cores to calculate CPU utilization limit ([8104](https://github.com/osquery/osquery/pull/8104))

Bug Fixes

- Always lock event_index_mutex when accessing event_index map ([8077](https://github.com/osquery/osquery/pull/8077))
- Check audit return values with <= ([8125](https://github.com/osquery/osquery/pull/8125))
- Fix `wifi_survey` table not to crash if the ssid cannot be retrieved ([8153](https://github.com/osquery/osquery/pull/8153))
- Fix macOS EndpointSecurity FIM mute inversion for file paths ([8166](https://github.com/osquery/osquery/pull/8166))

Documentation

- Add a list of Osquery fleet managers ([7781](https://github.com/osquery/osquery/pull/7781))
- Add basic file carving documentation ([8118](https://github.com/osquery/osquery/pull/8118))
- Changelog for 5.9.1 ([8088](https://github.com/osquery/osquery/pull/8088))
- Changelog 5.10.1 ([8155](https://github.com/osquery/osquery/pull/8155))
- Fixed small doc error ([8147](https://github.com/osquery/osquery/pull/8147))
- Update Automatic Table Construction example ([8094](https://github.com/osquery/osquery/pull/8094))
- Update XCode version mentions to the proper one ([8128](https://github.com/osquery/osquery/pull/8128))
- Update the description of `serial_number` in `connected_displays` ([8113](https://github.com/osquery/osquery/pull/8113))

Build

- Fix openssl build arch for Windows ARM64 ([8134](https://github.com/osquery/osquery/pull/8134))
- Fix python test http server use `SSLContext.wrap_socket()` instead of deprecated `ssl.wrap_socket()` ([8169](https://github.com/osquery/osquery/pull/8169))
- GitHub Action to cleanup at stale ec2 runners ([8156](https://github.com/osquery/osquery/pull/8156))
- Ignore CVE-2023-30571 ([8065](https://github.com/osquery/osquery/pull/8065))
- Missing pragma/header guard for boottime.h ([8117](https://github.com/osquery/osquery/pull/8117))
- Permit cross compiling for x86_64 on Apple Silicon ([8136](https://github.com/osquery/osquery/pull/8136))
- build: update macos hosted github runner to macos-12 monterey ([8100](https://github.com/osquery/osquery/pull/8100))
- ci: Fix DistributedTests.test_run_queries_with_denylisted_query test ([8154](https://github.com/osquery/osquery/pull/8154))
- ci: Increase aarch64 available space by splitting the build ([8131](https://github.com/osquery/osquery/pull/8131))
- ci: Increase disk space on the Linux x86_64 runner ([8133](https://github.com/osquery/osquery/pull/8133))
- ci: Remove flakyness when removing unused packages on Linux ([8144](https://github.com/osquery/osquery/pull/8144))
- cve: Fix the expat product name in the libraries manifest ([8158](https://github.com/osquery/osquery/pull/8158))
- cve: Ignore dbus CVE-2023-34969 ([8126](https://github.com/osquery/osquery/pull/8126))
- cve: Ignore libcap CVE-2023-2603 ([8127](https://github.com/osquery/osquery/pull/8127))
- cve: Update expat to version 2.5.0 ([8159](https://github.com/osquery/osquery/pull/8159))
- cve: Update libmagic to 5.45 ([8142](https://github.com/osquery/osquery/pull/8142))
- cve: Update lzma to 5.4.4 ([8135](https://github.com/osquery/osquery/pull/8135))
- cve: Update openssl to 3.1.3 ([8141](https://github.com/osquery/osquery/pull/8141))
- libs: Fix openssl build on aarch64 ([8084](https://github.com/osquery/osquery/pull/8084))
- libs: Update openssl to 3.1.1 ([8081](https://github.com/osquery/osquery/pull/8081))
- libs: Update openssl to 3.1.2 ([8124](https://github.com/osquery/osquery/pull/8124))
- test: Fix leaks in inotify and rocksdb tests ([8080](https://github.com/osquery/osquery/pull/8080))


<a name="5.9.1"></a>

[Git Commits](https://github.com/osquery/osquery/compare/5.8.2...5.9.1)

Big shoutout for the Windows Arm port!

Representing commits from 14 contributors! Thank you all.

New Features

- Add support for Windows on Arm ([7918](https://github.com/osquery/osquery/pull/7918))
- logger: Add new `string_batch` request type to compliment existing `string` type ([8027](https://github.com/osquery/osquery/pull/8027))

Table Changes

- Add `connected_displays` table on macOS ([7946](https://github.com/osquery/osquery/pull/7946))
- Add `windows_search` table ([7990](https://github.com/osquery/osquery/pull/7990))
- Restore functionality of `crashes` table on macOS 12 and newer ([7819](https://github.com/osquery/osquery/pull/7819))
- Update `keychain_items` to include data about key types ([8002](https://github.com/osquery/osquery/pull/8002))
- Update `os_version` to include Apple RSR fields using native API ([8011](https://github.com/osquery/osquery/pull/8011))
- Update `safari_extensions` to handle the current app extensions pattern ([7991](https://github.com/osquery/osquery/pull/7991))
- Update `system_info` to include the nnumber of sockets ([8038](https://github.com/osquery/osquery/pull/8038))
- Update `unified_log` table to add `predicate` column and optimize timestamp constraint ([8019](https://github.com/osquery/osquery/pull/8019))

Under the Hood improvements

- Improving `listDirectoriesInDirectory` by using `std::fs` ([7974](https://github.com/osquery/osquery/pull/7974))
- Do not consider a 404 as an error in ec2-instance-metadata ([8025](https://github.com/osquery/osquery/pull/8025))
- Release objects and free memory obtained from COM ([7999](https://github.com/osquery/osquery/pull/7999))
- Do not pass wstring::c_str() to wstringToString function ([8000](https://github.com/osquery/osquery/pull/8000))
- Do not copy process arguments into vector for CreateProcess call ([7956](https://github.com/osquery/osquery/pull/7956))

Bug Fixes

- Fix `version` column in `homebrew_packages` ([8057](https://github.com/osquery/osquery/pull/8057))
- Improve extended_attributes implementation for Linux and macOS ([8046](https://github.com/osquery/osquery/pull/8046))
- Update event tables to mark time column as "additional" ([8020](https://github.com/osquery/osquery/pull/8020))

Documentation

- Update expired Slack invite ([8051](https://github.com/osquery/osquery/pull/8051))
- Update `es_process_file_events.table` description ([7978](https://github.com/osquery/osquery/pull/7978))
- CHANGELOG 5.8.2 ([7986](https://github.com/osquery/osquery/pull/7986))

Build

- cve: Update to openssl 1.1.1u ([8050](https://github.com/osquery/osquery/pull/8050))
- cmake: Add an option to disable shallow git clone operations ([8026](https://github.com/osquery/osquery/pull/8026))
- Fix the aarch64 workflow ([8036](https://github.com/osquery/osquery/pull/8036))
- test: Fix a leak in ExtendedAttributesTableTests SetUp function ([8045](https://github.com/osquery/osquery/pull/8045))
- cve: Update libxml2 to v2.11.2 ([8023](https://github.com/osquery/osquery/pull/8023))
- libs: Bring out LZ4 from rdkafka and update it to v1.9.4 ([7996](https://github.com/osquery/osquery/pull/7996))
- ci: Update python version and docs build tools ([7969](https://github.com/osquery/osquery/pull/7969))
- ci: Update aarch64 runner to Ubuntu 20.04 and update badges ([7984](https://github.com/osquery/osquery/pull/7984))
- Add few unit tests for the hashing component ([7993](https://github.com/osquery/osquery/pull/7993))


<a name="5.8.2"></a>

[Git Commits](https://github.com/osquery/osquery/compare/5.8.1...5.8.2)

Representing commits from 6 contributors! Thank you all.


Bug Fixes

- Fix empty batch result set reporting ([7958](https://github.com/osquery/osquery/pull/7958))
- Fix COM security initialization by setting COM security per interface level ([7963](https://github.com/osquery/osquery/pull/7963))
- Fix username field in managed_policy table ([7944](https://github.com/osquery/osquery/pull/7944))

Documentation

- CHANGELOG 5.8.1 ([7957](https://github.com/osquery/osquery/pull/7957))

Build

- test: Do not always expect a row from the secureboot table ([7967](https://github.com/osquery/osquery/pull/7967))
- cmake: Only link against the experiments loader when needed ([7959](https://github.com/osquery/osquery/pull/7959))
- tests: Fix some tests becoming osquery shells ([7964](https://github.com/osquery/osquery/pull/7964))
- test: Fix SystemdUnitsTest missing the unit_file_state column ([7965](https://github.com/osquery/osquery/pull/7965))
- tests: Do not always build root tests on Linux ([7966](https://github.com/osquery/osquery/pull/7966))


<a name="5.8.1"></a>