Aperture-py

Changelog

List of aperture PRs merged since 2.18.1 release. For the full list of changes, see [list of changes][changes]

Remove policy marshalling from aperturectl (2744)

- Refactor: Updated `applyPolicy` and `createAndApplyPolicy` functions
in `cmd/aperturectl/cmd/apply/policy.go` and
`cmd/aperturectl/cmd/cloud/apply/policy.go` to use policy bytes instead
of policy objects, improving flexibility and consistency.
- Refactor: Modified `GetPolicy` and `UpdatePolicyUsingAPI` functions in
`cmd/aperturectl/cmd/utils/policies.go` to handle policy bytes,
enhancing code maintainability.
- New Feature: Changed `UpsertPolicy` function in
`pkg/policies/controlplane/policy-service.go` to support YAML formatted
policies, expanding the range of acceptable input formats.

Handle UNKNOWN service while passing to Check calls (2738)

This change allows both `""` and no `service` field as input in the
selectors and stops reporting `UNKNOWN` as service

- **Refactor**: Simplified YAML comparison in `CompareConfigMap`
function using `MatchYAML` matcher, improving code readability.
- **New Feature**: Added `ParseServicesFromString` and
`ParseServicesFromAddress` methods to the `ServiceGetter` interface,
enhancing modularity and simplifying service retrieval logic.
- **Bug Fix**: Updated `ServicesFromSocketAddress` method to return a
`nil` slice instead of `[]string{"UNKNOWN"}` when IP address is invalid
or no entity is associated with it, providing more accurate results.
- **Refactor**: Replaced direct service retrieval from addresses with
new `ParseServicesFromString` and `ParseServicesFromAddress` methods in
`checkhttp.go` and `authz.go`, improving code maintainability.
- **Chore**: Added logging statements in `checkhttp.go` and `authz.go`
for better visibility of source and destination services.

Preemption metric fixes (2743)

Description of change
* Unregister
* Init the metrics to get zero counters which ensure that the preemption
query which combines all of the metrics returns a valid values even if
some of the counters did not ever increment

- Refactor: Renamed and updated the `Delete` method in the
`SchedulerMetrics` struct for better readability.
- New Feature: Added `appendWorkloadLabel` method to `SchedulerMetrics`
struct, enhancing metric labeling with workload information.
- Refactor: Updated `NewScheduler` function in `Factory` struct to
initialize preempt metrics for each workload, improving metrics
accuracy.
- Refactor: Modified `Decide` method in `Scheduler` struct to use
`getWorkloadLabel` function, providing more precise workload
identification.

Send policy string in the UpsertPolicy req to validate before putting in etcd (2739)

Description of change

This change does not send policy proto in the UpsertPolicy request.
Instead, sends the raw policy string. This way, the string is
unmarshalled to languagev1.Policy on the controller side before putting
in etcd. We get correct validation and proper defaults handling.

- New Feature: Added support for specifying policy as a JSON string in
`UpsertPolicyRequest` with the new `policy_string` field.
- Refactor: Updated package and type names related to policy language
across multiple files for better organization and clarity. The package
`languagev1` has been renamed to `policylangv1`.
- Refactor: Improved handling of policy data in `UpsertPolicy` function,
including better error handling and support for field masks.
- Chore: Replaced usage of `proto.Marshal()` with `MarshalJSON()` for
policy serialization, providing more consistent and efficient data
handling.

add validation checks for deprecated fields (2741)

- **Deprecation**: The `AdaptiveLoadScheduler` component in the
`FlowControl` message has been deprecated. Users are advised to use the
"AIMD Load Scheduler" instead for better performance and efficiency.
- **Deprecation**: The `telemetry_collectors` field in the `Resources`
message and the `differentiator` field in the `Component` message have
been marked as deprecated. Future updates will remove these fields.
- **Documentation**: Updated the configuration specification document to
reflect the deprecation of the "Adaptive Load Scheduler" and
"TelemetryCollector".
- **Refactor**: Updated blueprint metadata to include a new
`deprecation_message` key, providing users with more information about
deprecated components.

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v2.18.1...releases/aperture-controller/v2.18.2-rc.1

Changelog

List of aperture PRs merged since 2.18.0 release. For the full list of changes, see [list of changes][changes]

use policy name in background circuit job name (2734)

- Refactor: Enhanced the naming convention for background jobs in the
control plane runtime. The job names now include the policy name along
with the policy hash, making them more descriptive and unique. This
change improves traceability and debugging by providing clearer
identification of each job.

Scheduler preemption metrics (2731)

**New Features:**
- Added new metrics related to workload requests and tokens, providing
more detailed insights into system performance. These include
`RequestInQueueDurationMetricName`, `WorkloadPreemptedTokensMetricName`,
`WorkloadDelayedTokensMetricName`, and `WorkloadOnTimeMetricName`.
- Introduced a new `preemptionMetrics` struct in the `WFQScheduler` for
tracking token counters related to preemption and delay metrics.

**Refactor:**
- Updated the `queuedRequest` struct with two new fields:
`tokensInQueue` and `tokensAllowed` to better manage workload requests.
- Enhanced the `queueRequest`, `cancelRequest`, and `scheduleRequest`
functions to handle the new preemption and delay metrics.

**Documentation:**
- Improved clarity and consistency of the metrics generated by Aperture
Agents in the documentation.

aperturectl: apply all command for blueprints (2733)

- New Feature: Introduced `--values-dir` flag to specify a directory
path containing multiple values files.
- New Feature: Modified `--values-file` flag to accept a single file
path instead of a blueprint name.
- New Feature: Added `URIToRawContentURL` function that converts a URI
to a raw content URL, specifically for GitHub.
- Refactor: Renamed `GetPolicyTUIModel` to `GetPoliciesTUIModel` across
the codebase for improved clarity.
- Refactor: Updated logic for handling values files and directories in
`generateCmd` function.
- Refactor: `setupOutputDir` function now creates a temporary directory
if `outputDir` is not provided.
- Documentation: Updated command-line options of the `aperturectl` tool
in the documentation.

Add sections under Adaptive Service Protection based on metrics origin (2577)

Description of change

<img width="1898" alt="postgresql-dashboards-signal"
src="https://github.com/fluxninja/aperture/assets/1553055/cab7cb69-4ca7-42d3-bb26-2c30772a726b">
<img width="1895" alt="postgresql-dashboards-tokens"
src="https://github.com/fluxninja/aperture/assets/1553055/3d7e53de-cbb5-4860-be12-b4c5d60fdff4">

- New Feature: Added new Markdown files for Adaptive Service Protection
use cases, including database, infrastructure, message queue, and
service. These documents provide detailed explanations and guides for
each use case.
- Documentation: Updated various documentation files to improve clarity,
readability, and consistency. This includes changes in wording, fixing
typos, updating links, and rephrasing sentences.
- Refactor: Updated comments and struct field names in
`cmd/aperture-agent/config/types.go` for better clarity.
- Chore: Updated file paths in `operator/hack/create_policy_sample.sh`
and `apertureVersion.js`.
- Documentation: Updated the sidebar position in "Percentage Rollouts"
tutorial page.
- New Feature: Added a new PostgreSQL policy guide in
`docs/content/guides/database-load-management/postgresql.md`.
- Refactor: Modified the `vuStages` array in
`playground/scenarios/graceful-js/load-generator/test.js` to adjust the
target values for different durations.
- Documentation: Updated the load management capabilities section in
`docs/content/introduction.md` with revised feature names and
descriptions.

---------

Co-authored-by: Sudhanshu Prajapati <spsupraj2dgmail.com>

Expose policy hash function (2730)


Fix data race in Controller shutdown (2729)

- Refactor: Simplified the creation of watcher instances in
`policy-factory.go` by removing unnecessary variables and consolidating
the creation process.
- Refactor: Removed the `NotifierLifecycle` function from `watcher.go`,
indicating a refactoring or relocation of the functionality for adding
and removing prefix notifiers.
- Style: Updated job group name from "promql_jobs_scheduler" to
"background_jobs_scheduler" in `background-scheduler.go` for better
clarity and consistency.

change default for delayedInitialFill (2706)


Fix deadlock in peers shutdown (2703)

- Refactor: Improved concurrency in the `PeerDiscovery` struct in
`pkg/peers/peers.go`. Introduced separate locks for `peers` and
`services` to prevent data races and enhance performance.
- Refactor: Renamed methods `RegisterSelf` and `DeregisterSelf` to
`registerSelf` and `deregisterSelf` respectively, enhancing code
readability and maintainability.
- Refactor: Updated various methods in `PeerDiscovery` to use the new
lock fields, improving code consistency and reliability.

reject when req tokens are more than bucketCapacity (2702)


global token bucket: improve wait time precision (2699)



[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v2.18.0...releases/aperture-controller/v2.18.1

Changelog

List of aperture PRs merged since 2.18.0 release. For the full list of changes, see [list of changes][changes]

use policy name in background circuit job name (2734)

- Refactor: Enhanced the naming convention for background jobs in the
control plane runtime. The job names now include the policy name along
with the policy hash, making them more descriptive and unique. This
change improves traceability and debugging by providing clearer
identification of each job.

Scheduler preemption metrics (2731)

**New Features:**
- Added new metrics related to workload requests and tokens, providing
more detailed insights into system performance. These include
`RequestInQueueDurationMetricName`, `WorkloadPreemptedTokensMetricName`,
`WorkloadDelayedTokensMetricName`, and `WorkloadOnTimeMetricName`.
- Introduced a new `preemptionMetrics` struct in the `WFQScheduler` for
tracking token counters related to preemption and delay metrics.

**Refactor:**
- Updated the `queuedRequest` struct with two new fields:
`tokensInQueue` and `tokensAllowed` to better manage workload requests.
- Enhanced the `queueRequest`, `cancelRequest`, and `scheduleRequest`
functions to handle the new preemption and delay metrics.

**Documentation:**
- Improved clarity and consistency of the metrics generated by Aperture
Agents in the documentation.

aperturectl: apply all command for blueprints (2733)

- New Feature: Introduced `--values-dir` flag to specify a directory
path containing multiple values files.
- New Feature: Modified `--values-file` flag to accept a single file
path instead of a blueprint name.
- New Feature: Added `URIToRawContentURL` function that converts a URI
to a raw content URL, specifically for GitHub.
- Refactor: Renamed `GetPolicyTUIModel` to `GetPoliciesTUIModel` across
the codebase for improved clarity.
- Refactor: Updated logic for handling values files and directories in
`generateCmd` function.
- Refactor: `setupOutputDir` function now creates a temporary directory
if `outputDir` is not provided.
- Documentation: Updated command-line options of the `aperturectl` tool
in the documentation.

Add sections under Adaptive Service Protection based on metrics origin (2577)

Description of change

<img width="1898" alt="postgresql-dashboards-signal"
src="https://github.com/fluxninja/aperture/assets/1553055/cab7cb69-4ca7-42d3-bb26-2c30772a726b">
<img width="1895" alt="postgresql-dashboards-tokens"
src="https://github.com/fluxninja/aperture/assets/1553055/3d7e53de-cbb5-4860-be12-b4c5d60fdff4">

- New Feature: Added new Markdown files for Adaptive Service Protection
use cases, including database, infrastructure, message queue, and
service. These documents provide detailed explanations and guides for
each use case.
- Documentation: Updated various documentation files to improve clarity,
readability, and consistency. This includes changes in wording, fixing
typos, updating links, and rephrasing sentences.
- Refactor: Updated comments and struct field names in
`cmd/aperture-agent/config/types.go` for better clarity.
- Chore: Updated file paths in `operator/hack/create_policy_sample.sh`
and `apertureVersion.js`.
- Documentation: Updated the sidebar position in "Percentage Rollouts"
tutorial page.
- New Feature: Added a new PostgreSQL policy guide in
`docs/content/guides/database-load-management/postgresql.md`.
- Refactor: Modified the `vuStages` array in
`playground/scenarios/graceful-js/load-generator/test.js` to adjust the
target values for different durations.
- Documentation: Updated the load management capabilities section in
`docs/content/introduction.md` with revised feature names and
descriptions.

---------

Co-authored-by: Sudhanshu Prajapati <spsupraj2dgmail.com>

Expose policy hash function (2730)


Fix data race in Controller shutdown (2729)

- Refactor: Simplified the creation of watcher instances in
`policy-factory.go` by removing unnecessary variables and consolidating
the creation process.
- Refactor: Removed the `NotifierLifecycle` function from `watcher.go`,
indicating a refactoring or relocation of the functionality for adding
and removing prefix notifiers.
- Style: Updated job group name from "promql_jobs_scheduler" to
"background_jobs_scheduler" in `background-scheduler.go` for better
clarity and consistency.

change default for delayedInitialFill (2706)


Fix deadlock in peers shutdown (2703)

- Refactor: Improved concurrency in the `PeerDiscovery` struct in
`pkg/peers/peers.go`. Introduced separate locks for `peers` and
`services` to prevent data races and enhance performance.
- Refactor: Renamed methods `RegisterSelf` and `DeregisterSelf` to
`registerSelf` and `deregisterSelf` respectively, enhancing code
readability and maintainability.
- Refactor: Updated various methods in `PeerDiscovery` to use the new
lock fields, improving code consistency and reliability.

reject when req tokens are more than bucketCapacity (2702)


global token bucket: improve wait time precision (2699)



[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v2.18.0...releases/aperture-controller/v2.18.1-rc.1

Changelog

List of aperture PRs merged since 2.17.0 release. For the full list of changes, see [list of changes][changes]

Made service optional in flow-control preview and other fixes (2695)

- New Feature: Added `aperturectl cloud apply dynamic-config` command to
apply Aperture DynamicConfig to a Policy.
- New Feature: Introduced `autoscale` and `decisions` commands to the
`CloudCmd` command-line interface for querying information about active
AutoScale integrations and getting Aperture Decisions respectively.
- Refactor: Updated `PreviewCmd` command to accept only one argument,
the control point, and added a new flag `service` to specify the service
Fully Qualified Domain Name (FQDN) for the preview.
- Documentation: Added comprehensive documentation for all new commands
and updated existing ones to reflect changes in command usage and
arguments.
- Bug Fix: Modified `GetPolicies` function to log invalid policies and
continue processing valid ones instead of returning an error.

Use User API token in aperturectl and separate `aperturectl cloud` subcommand (2687)

- New Feature: Added new commands to the CLI tool for listing agents,
control points, entities, and policies.
- New Feature: Introduced utility functions for managing Aperture
policies including retrieving, parsing, updating, checking for updates,
deleting, and listing policies.
- Refactor: Improved modularity and code reuse by moving logic to
utility functions in the `utils` package.
- Documentation: Updated documentation to reflect changes in
command-line options and added new sections for various commands related
to the Cloud Controller.
- Bug Fix: Corrected a typo in a log message in the self-hosting
controller docker file.

Fixed policy update using API (2693)


refactor: use proto format when writing proto messages to etcd (2691)

Use proto format when writing proto messages to etcd.

- New Feature: Transitioned from JSON/YAML serialization to Protocol
Buffers (protobuf) for policy data, infra meter configurations, and peer
information. This change enhances performance and compatibility with
other systems.
- Refactor: Updated the unmarshalling process across various modules to
use a new `ProtobufUnmarshaller` implementation, improving code
maintainability.
- Refactor: Simplified the policy validation and compilation process by
introducing separate functions for handling YAML and protobuf inputs.
- Bug Fix: Corrected the application of field masks to policies in the
control plane module, enhancing data accuracy and consistency.

Controller fixes (2690)

Description
* Fix Controller restart getting stuck because of older leader results.
* Fix graceful shutdown issue in Controller because of `GracefulStop` in
grpc server.
* Reduce etcd lease TTL from 60s to 10s so that older (left over in-case
of non-graceful shutdown) leader entry expires before readiness failure
of Controller.

Fix rate limiter when continuous_fill is set to false (2688)

- New Feature: Added a `disable_delayed_filling` field in the
`RateLimiter` message to allow disabling the delayed filling of the
token bucket, providing more control over rate limiting behavior.
- Refactor: Simplified blueprint generation by removing unused
`metadata` parameters and variables. This change improves code
readability and maintainability.
- Refactor: Introduced a new function `getWaitTime` in the
`GlobalTokenBucket` struct to calculate wait time for tokens, improving
code readability.
- Bug Fix: Fixed deep copy operation in the `DeepCopyInto` method of the
`AgentConfigSpec` struct, eliminating unnecessary operations.
- Documentation: Updated documentation for the RateLimiter component to
reflect changes in functionality and configuration options.

---------

Co-authored-by: Harjot Gill <harjot.gillfluxninja.com>

Disable auto scale by default and adds its permission only when enabled (2681)

- New Feature: Introduced a new configuration option
`agent.config.auto_scale.kubernetes.enabled` to control the auto-scaling
feature of the Aperture Agent.
- Refactor: Changed the default value of the auto-scaling feature in
Kubernetes from `true` to `false`. This means that the auto-scaling
functionality will now be disabled by default, providing users with more
control over their resource management.
- Documentation: Updated documentation to reflect these changes in the
configuration parameters.

Fixes and improvements on documentation (2680)

This PR contains below changes:
1. Make Helm the default option for installation and aperturectl
secondary
2. Fixed some of the docs issues observed during installation
3. Added separate policy apply command per installation time
4. Fixed one issue where apimeta.IsNoMatchError was not working as
expected

- Refactor: Replaced usage of "apimeta.IsNoMatchError" with a custom
utility function "utils.IsNoMatchError" across multiple files for better
error handling.
- Documentation: Updated and improved the installation instructions for
Aperture Agent and Controller in various deployment scenarios, providing
clearer steps for both `aperturectl` CLI tool and Helm.
- New Feature: Added new configuration options in the self-hosting agent
configuration file, allowing users to disable certain Kubernetes
scrapers and control service discovery.
- Chore: Swapped the order of `Helm` and `aperturectl` in installation
instructions across multiple documentation files for consistency.

Fixed issue in aperturectl when both uri and version is provided with blueprints generate (2682)

"Bug Fix: Enhanced error handling in blueprint generation command

- The update introduces a more robust check for the presence of
`blueprintsVersion` and `blueprintsURI`.
- If both are missing, the system now attempts to retrieve these values
from the `values` map.
- In case the required information is still not found, an explicit error
message is returned, improving user feedback and troubleshooting.
- This fix enhances the reliability of the blueprint generation process
and provides clearer guidance when input parameters are missing."

Set workload index as fairness label (2676)

- Refactor: Simplified the assignment of `fairnessLabel` in the `Decide`
function of the `Scheduler` struct by removing unnecessary string
concatenation.
- Refactor: Streamlined the assignment of `request.FairnessLabel` to
`metricsLabels[metrics.WorkloadIndexLabel]` in `wfq.go`, eliminating the
need for the `strings` package.

These changes enhance code readability and maintainability without
altering the underlying logic or functionality.

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v2.17.0...releases/aperture-controller/v2.18.0

Changelog

List of aperture PRs merged since 2.17.0 release. For the full list of changes, see [list of changes][changes]

Made service optional in flow-control preview and other fixes (2695)

- New Feature: Added `aperturectl cloud apply dynamic-config` command to
apply Aperture DynamicConfig to a Policy.
- New Feature: Introduced `autoscale` and `decisions` commands to the
`CloudCmd` command-line interface for querying information about active
AutoScale integrations and getting Aperture Decisions respectively.
- Refactor: Updated `PreviewCmd` command to accept only one argument,
the control point, and added a new flag `service` to specify the service
Fully Qualified Domain Name (FQDN) for the preview.
- Documentation: Added comprehensive documentation for all new commands
and updated existing ones to reflect changes in command usage and
arguments.
- Bug Fix: Modified `GetPolicies` function to log invalid policies and
continue processing valid ones instead of returning an error.

Use User API token in aperturectl and separate `aperturectl cloud` subcommand (2687)

- New Feature: Added new commands to the CLI tool for listing agents,
control points, entities, and policies.
- New Feature: Introduced utility functions for managing Aperture
policies including retrieving, parsing, updating, checking for updates,
deleting, and listing policies.
- Refactor: Improved modularity and code reuse by moving logic to
utility functions in the `utils` package.
- Documentation: Updated documentation to reflect changes in
command-line options and added new sections for various commands related
to the Cloud Controller.
- Bug Fix: Corrected a typo in a log message in the self-hosting
controller docker file.

Fixed policy update using API (2693)


refactor: use proto format when writing proto messages to etcd (2691)

Use proto format when writing proto messages to etcd.

- New Feature: Transitioned from JSON/YAML serialization to Protocol
Buffers (protobuf) for policy data, infra meter configurations, and peer
information. This change enhances performance and compatibility with
other systems.
- Refactor: Updated the unmarshalling process across various modules to
use a new `ProtobufUnmarshaller` implementation, improving code
maintainability.
- Refactor: Simplified the policy validation and compilation process by
introducing separate functions for handling YAML and protobuf inputs.
- Bug Fix: Corrected the application of field masks to policies in the
control plane module, enhancing data accuracy and consistency.

Controller fixes (2690)

Description
* Fix Controller restart getting stuck because of older leader results.
* Fix graceful shutdown issue in Controller because of `GracefulStop` in
grpc server.
* Reduce etcd lease TTL from 60s to 10s so that older (left over in-case
of non-graceful shutdown) leader entry expires before readiness failure
of Controller.

Fix rate limiter when continuous_fill is set to false (2688)

- New Feature: Added a `disable_delayed_filling` field in the
`RateLimiter` message to allow disabling the delayed filling of the
token bucket, providing more control over rate limiting behavior.
- Refactor: Simplified blueprint generation by removing unused
`metadata` parameters and variables. This change improves code
readability and maintainability.
- Refactor: Introduced a new function `getWaitTime` in the
`GlobalTokenBucket` struct to calculate wait time for tokens, improving
code readability.
- Bug Fix: Fixed deep copy operation in the `DeepCopyInto` method of the
`AgentConfigSpec` struct, eliminating unnecessary operations.
- Documentation: Updated documentation for the RateLimiter component to
reflect changes in functionality and configuration options.

---------

Co-authored-by: Harjot Gill <harjot.gillfluxninja.com>

Disable auto scale by default and adds its permission only when enabled (2681)

- New Feature: Introduced a new configuration option
`agent.config.auto_scale.kubernetes.enabled` to control the auto-scaling
feature of the Aperture Agent.
- Refactor: Changed the default value of the auto-scaling feature in
Kubernetes from `true` to `false`. This means that the auto-scaling
functionality will now be disabled by default, providing users with more
control over their resource management.
- Documentation: Updated documentation to reflect these changes in the
configuration parameters.

Fixes and improvements on documentation (2680)

This PR contains below changes:
1. Make Helm the default option for installation and aperturectl
secondary
2. Fixed some of the docs issues observed during installation
3. Added separate policy apply command per installation time
4. Fixed one issue where apimeta.IsNoMatchError was not working as
expected

- Refactor: Replaced usage of "apimeta.IsNoMatchError" with a custom
utility function "utils.IsNoMatchError" across multiple files for better
error handling.
- Documentation: Updated and improved the installation instructions for
Aperture Agent and Controller in various deployment scenarios, providing
clearer steps for both `aperturectl` CLI tool and Helm.
- New Feature: Added new configuration options in the self-hosting agent
configuration file, allowing users to disable certain Kubernetes
scrapers and control service discovery.
- Chore: Swapped the order of `Helm` and `aperturectl` in installation
instructions across multiple documentation files for consistency.

Fixed issue in aperturectl when both uri and version is provided with blueprints generate (2682)

"Bug Fix: Enhanced error handling in blueprint generation command

- The update introduces a more robust check for the presence of
`blueprintsVersion` and `blueprintsURI`.
- If both are missing, the system now attempts to retrieve these values
from the `values` map.
- In case the required information is still not found, an explicit error
message is returned, improving user feedback and troubleshooting.
- This fix enhances the reliability of the blueprint generation process
and provides clearer guidance when input parameters are missing."

Set workload index as fairness label (2676)

- Refactor: Simplified the assignment of `fairnessLabel` in the `Decide`
function of the `Scheduler` struct by removing unnecessary string
concatenation.
- Refactor: Streamlined the assignment of `request.FairnessLabel` to
`metricsLabels[metrics.WorkloadIndexLabel]` in `wfq.go`, eliminating the
need for the `strings` package.

These changes enhance code readability and maintainability without
altering the underlying logic or functionality.

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v2.17.0...releases/aperture-controller/v2.18.0-rc.2

Changelog

List of aperture PRs merged since 2.17.0 release. For the full list of changes, see [list of changes][changes]

Made service optional in flow-control preview and other fixes (2695)

- New Feature: Added `aperturectl cloud apply dynamic-config` command to
apply Aperture DynamicConfig to a Policy.
- New Feature: Introduced `autoscale` and `decisions` commands to the
`CloudCmd` command-line interface for querying information about active
AutoScale integrations and getting Aperture Decisions respectively.
- Refactor: Updated `PreviewCmd` command to accept only one argument,
the control point, and added a new flag `service` to specify the service
Fully Qualified Domain Name (FQDN) for the preview.
- Documentation: Added comprehensive documentation for all new commands
and updated existing ones to reflect changes in command usage and
arguments.
- Bug Fix: Modified `GetPolicies` function to log invalid policies and
continue processing valid ones instead of returning an error.

Use User API token in aperturectl and separate `aperturectl cloud` subcommand (2687)

- New Feature: Added new commands to the CLI tool for listing agents,
control points, entities, and policies.
- New Feature: Introduced utility functions for managing Aperture
policies including retrieving, parsing, updating, checking for updates,
deleting, and listing policies.
- Refactor: Improved modularity and code reuse by moving logic to
utility functions in the `utils` package.
- Documentation: Updated documentation to reflect changes in
command-line options and added new sections for various commands related
to the Cloud Controller.
- Bug Fix: Corrected a typo in a log message in the self-hosting
controller docker file.

Fixed policy update using API (2693)


refactor: use proto format when writing proto messages to etcd (2691)

Use proto format when writing proto messages to etcd.

- New Feature: Transitioned from JSON/YAML serialization to Protocol
Buffers (protobuf) for policy data, infra meter configurations, and peer
information. This change enhances performance and compatibility with
other systems.
- Refactor: Updated the unmarshalling process across various modules to
use a new `ProtobufUnmarshaller` implementation, improving code
maintainability.
- Refactor: Simplified the policy validation and compilation process by
introducing separate functions for handling YAML and protobuf inputs.
- Bug Fix: Corrected the application of field masks to policies in the
control plane module, enhancing data accuracy and consistency.

Controller fixes (2690)

Description
* Fix Controller restart getting stuck because of older leader results.
* Fix graceful shutdown issue in Controller because of `GracefulStop` in
grpc server.
* Reduce etcd lease TTL from 60s to 10s so that older (left over in-case
of non-graceful shutdown) leader entry expires before readiness failure
of Controller.

Fix rate limiter when continuous_fill is set to false (2688)

- New Feature: Added a `disable_delayed_filling` field in the
`RateLimiter` message to allow disabling the delayed filling of the
token bucket, providing more control over rate limiting behavior.
- Refactor: Simplified blueprint generation by removing unused
`metadata` parameters and variables. This change improves code
readability and maintainability.
- Refactor: Introduced a new function `getWaitTime` in the
`GlobalTokenBucket` struct to calculate wait time for tokens, improving
code readability.
- Bug Fix: Fixed deep copy operation in the `DeepCopyInto` method of the
`AgentConfigSpec` struct, eliminating unnecessary operations.
- Documentation: Updated documentation for the RateLimiter component to
reflect changes in functionality and configuration options.

---------

Co-authored-by: Harjot Gill <harjot.gillfluxninja.com>

Disable auto scale by default and adds its permission only when enabled (2681)

- New Feature: Introduced a new configuration option
`agent.config.auto_scale.kubernetes.enabled` to control the auto-scaling
feature of the Aperture Agent.
- Refactor: Changed the default value of the auto-scaling feature in
Kubernetes from `true` to `false`. This means that the auto-scaling
functionality will now be disabled by default, providing users with more
control over their resource management.
- Documentation: Updated documentation to reflect these changes in the
configuration parameters.

Fixes and improvements on documentation (2680)

This PR contains below changes:
1. Make Helm the default option for installation and aperturectl
secondary
2. Fixed some of the docs issues observed during installation
3. Added separate policy apply command per installation time
4. Fixed one issue where apimeta.IsNoMatchError was not working as
expected

- Refactor: Replaced usage of "apimeta.IsNoMatchError" with a custom
utility function "utils.IsNoMatchError" across multiple files for better
error handling.
- Documentation: Updated and improved the installation instructions for
Aperture Agent and Controller in various deployment scenarios, providing
clearer steps for both `aperturectl` CLI tool and Helm.
- New Feature: Added new configuration options in the self-hosting agent
configuration file, allowing users to disable certain Kubernetes
scrapers and control service discovery.
- Chore: Swapped the order of `Helm` and `aperturectl` in installation
instructions across multiple documentation files for consistency.

Fixed issue in aperturectl when both uri and version is provided with blueprints generate (2682)

"Bug Fix: Enhanced error handling in blueprint generation command

- The update introduces a more robust check for the presence of
`blueprintsVersion` and `blueprintsURI`.
- If both are missing, the system now attempts to retrieve these values
from the `values` map.
- In case the required information is still not found, an explicit error
message is returned, improving user feedback and troubleshooting.
- This fix enhances the reliability of the blueprint generation process
and provides clearer guidance when input parameters are missing."

Set workload index as fairness label (2676)

- Refactor: Simplified the assignment of `fairnessLabel` in the `Decide`
function of the `Scheduler` struct by removing unnecessary string
concatenation.
- Refactor: Streamlined the assignment of `request.FairnessLabel` to
`metricsLabels[metrics.WorkloadIndexLabel]` in `wfq.go`, eliminating the
need for the `strings` package.

These changes enhance code readability and maintainability without
altering the underlying logic or functionality.

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v2.17.0...releases/aperture-controller/v2.18.0-rc.1