Aperture-py

Changelog

List of aperture PRs merged since 1.4.0 release. For the full list of changes, see [list of changes][changes]

support defaults on maps (1790)


refactor flowcontrol engine (1776)


Don't pass metric deletion error in circuit (1774)

We delete some metrics for invalid readings. So they get deleted twice.
Deleting a deleted metric is not bad, so we can just handle this and log
in setup.

Bug fix
- Improved error handling in circuit setup function by logging errors
during metrics deletion and returning nil instead of multi-error when no
errors are present.

> 🎉 Oh, rejoice, for we have caught the bug! 🐛
> In circuit setup, it hid snug. 🌿
> With better logs, we now unveil 📜
> The errors that once made us fail. 💪

Fixed certificate issues on Operator restarts (1770)

Description of change

This change persists the certificates generated by the Operator for
webhooks and Controller so that we don't see the certificate errors
during upgrades or operator pod restarts.

Fixes 1694

**Bug fix:**
- Persist certificates generated by the Operator for webhooks and
Controller to prevent certificate issues on restarts

**Documentation:**
- Update installation documentation to reflect certificate persistence
changes

**Refactor:**
- Improve code readability and maintainability

> 🎉 Certificates persist, no more errors resist! 🚀
> With docs updated, and code refactored,
> Our Operator's journey, now unencumbered. 🌟

Update policy and dynamic-config APIs (1767)

Description of change

- Update etcd calls to be blocking using `etcdClient` directly
- Move away from adding/updating multiple policies and dynamic-configs
in a single call
- Combine a POST and PATCH into 1 call, by using
`google.protobuf.FieldMask`, still keeping both methods and backwards
compatibility with `aperturectl`
- Add FieldMask related helper functions in utils
- Replace "no matches for kind" string check with
`k8s.io/apimachinery/pkg/api/meta` errors check

**New Feature:**
- Combine POST and PATCH into a single call using
`google.protobuf.FieldMask`
- Add FieldMask helper functions in the `utils` package

**Refactor:**
- Update etcd calls to be blocking
- Simplify policy and dynamic-config updates by separating them
- Improve error handling with `k8s.io/apimachinery/pkg/api/meta` errors
check

> 🎉 A merge of POST and PATCH we see, 🔄
> With FieldMask helpers, oh so nifty! 🛠️
> Etcd calls now block, as they should, ⏱️
> And error checks improved, all understood. 🚀

Namespace scoped installation (1764)

Description of change

This PR contains all the changes required for running agent and
controller in namespace scoped:
- Option in Agent and Controller helm chart to install only namespace
scoped installation
- aperturectl support for installation in namespace scoped
- Disable Policy CR watcher conditionally in Controller
- Policy Post and Delete APIs for managing policies

Fixes 1692

Default Playground SS:


Aperture Running in namespace scoped SS with Envoy:


**New Feature:**
- Namespace-scoped installation of Aperture Agent and Controller
- New APIs for managing policies and dynamic configs
- Conditional disabling of Policy CR watcher in Controller
- `aperturectl delete` command for deleting policies

**Documentation:**
- Updated documentation for new installation modes, APIs, and commands

> 🎉 Namespace-scoped, we now install,
> With new APIs, our features fulfill.
> Policies managed, with ease and grace,
> Aperture's updates, we embrace! 🚀

---------

Co-authored-by: Hasit Mistry <hasitfluxninja.com>

GRPC -> gRPC (1762)


Fix metrics (1759)

scheduler: rely on client timeouts (1757)

Description of change

Scheduler logic is changed to ignore the existing `timeout_factor` and
`max_timeout` fields. Instead, the Check() call's client must decide the
timeout for each request. To work with fail-open design of Aperture
clients, `decision_deadline_margin` parameter is added instead.

**New Feature:**
- Rely on client timeouts for scheduler logic
- Add `decision_deadline_margin` parameter for fail-open design

**Refactor:**
- Remove unused imports and minor modifications in load generator test
files

> 🎉 A new dawn for scheduling, we embrace,
> 🕒 Client timeouts now lead the race.
> ⏱️ With margins set, decisions are wise,
> 🧹 Cleaned up imports, a welcomed surprise!

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v1.4.0...releases/aperture-controller/v1.5.0

Changelog

List of aperture PRs merged since 1.4.0 release. For the full list of changes, see [list of changes][changes]

support defaults on maps (1790)


refactor flowcontrol engine (1776)


Don't pass metric deletion error in circuit (1774)

We delete some metrics for invalid readings. So they get deleted twice.
Deleting a deleted metric is not bad, so we can just handle this and log
in setup.

Bug fix
- Improved error handling in circuit setup function by logging errors
during metrics deletion and returning nil instead of multi-error when no
errors are present.

> 🎉 Oh, rejoice, for we have caught the bug! 🐛
> In circuit setup, it hid snug. 🌿
> With better logs, we now unveil 📜
> The errors that once made us fail. 💪

Fixed certificate issues on Operator restarts (1770)

Description of change

This change persists the certificates generated by the Operator for
webhooks and Controller so that we don't see the certificate errors
during upgrades or operator pod restarts.

Fixes 1694

**Bug fix:**
- Persist certificates generated by the Operator for webhooks and
Controller to prevent certificate issues on restarts

**Documentation:**
- Update installation documentation to reflect certificate persistence
changes

**Refactor:**
- Improve code readability and maintainability

> 🎉 Certificates persist, no more errors resist! 🚀
> With docs updated, and code refactored,
> Our Operator's journey, now unencumbered. 🌟

Update policy and dynamic-config APIs (1767)

Description of change

- Update etcd calls to be blocking using `etcdClient` directly
- Move away from adding/updating multiple policies and dynamic-configs
in a single call
- Combine a POST and PATCH into 1 call, by using
`google.protobuf.FieldMask`, still keeping both methods and backwards
compatibility with `aperturectl`
- Add FieldMask related helper functions in utils
- Replace "no matches for kind" string check with
`k8s.io/apimachinery/pkg/api/meta` errors check

**New Feature:**
- Combine POST and PATCH into a single call using
`google.protobuf.FieldMask`
- Add FieldMask helper functions in the `utils` package

**Refactor:**
- Update etcd calls to be blocking
- Simplify policy and dynamic-config updates by separating them
- Improve error handling with `k8s.io/apimachinery/pkg/api/meta` errors
check

> 🎉 A merge of POST and PATCH we see, 🔄
> With FieldMask helpers, oh so nifty! 🛠️
> Etcd calls now block, as they should, ⏱️
> And error checks improved, all understood. 🚀

Namespace scoped installation (1764)

Description of change

This PR contains all the changes required for running agent and
controller in namespace scoped:
- Option in Agent and Controller helm chart to install only namespace
scoped installation
- aperturectl support for installation in namespace scoped
- Disable Policy CR watcher conditionally in Controller
- Policy Post and Delete APIs for managing policies

Fixes 1692

Default Playground SS:


Aperture Running in namespace scoped SS with Envoy:


**New Feature:**
- Namespace-scoped installation of Aperture Agent and Controller
- New APIs for managing policies and dynamic configs
- Conditional disabling of Policy CR watcher in Controller
- `aperturectl delete` command for deleting policies

**Documentation:**
- Updated documentation for new installation modes, APIs, and commands

> 🎉 Namespace-scoped, we now install,
> With new APIs, our features fulfill.
> Policies managed, with ease and grace,
> Aperture's updates, we embrace! 🚀

---------

Co-authored-by: Hasit Mistry <hasitfluxninja.com>

GRPC -> gRPC (1762)


Fix metrics (1759)

scheduler: rely on client timeouts (1757)

Description of change

Scheduler logic is changed to ignore the existing `timeout_factor` and
`max_timeout` fields. Instead, the Check() call's client must decide the
timeout for each request. To work with fail-open design of Aperture
clients, `decision_deadline_margin` parameter is added instead.

**New Feature:**
- Rely on client timeouts for scheduler logic
- Add `decision_deadline_margin` parameter for fail-open design

**Refactor:**
- Remove unused imports and minor modifications in load generator test
files

> 🎉 A new dawn for scheduling, we embrace,
> 🕒 Client timeouts now lead the race.
> ⏱️ With margins set, decisions are wise,
> 🧹 Cleaned up imports, a welcomed surprise!

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v1.4.0...releases/aperture-controller/v1.5.0-rc.3

Changelog

List of aperture PRs merged since 1.4.0 release. For the full list of changes, see [list of changes][changes]

refactor flowcontrol engine (1776)


Don't pass metric deletion error in circuit (1774)

We delete some metrics for invalid readings. So they get deleted twice.
Deleting a deleted metric is not bad, so we can just handle this and log
in setup.

Bug fix
- Improved error handling in circuit setup function by logging errors
during metrics deletion and returning nil instead of multi-error when no
errors are present.

> 🎉 Oh, rejoice, for we have caught the bug! 🐛
> In circuit setup, it hid snug. 🌿
> With better logs, we now unveil 📜
> The errors that once made us fail. 💪

Fixed certificate issues on Operator restarts (1770)

Description of change

This change persists the certificates generated by the Operator for
webhooks and Controller so that we don't see the certificate errors
during upgrades or operator pod restarts.

Fixes 1694

**Bug fix:**
- Persist certificates generated by the Operator for webhooks and
Controller to prevent certificate issues on restarts

**Documentation:**
- Update installation documentation to reflect certificate persistence
changes

**Refactor:**
- Improve code readability and maintainability

> 🎉 Certificates persist, no more errors resist! 🚀
> With docs updated, and code refactored,
> Our Operator's journey, now unencumbered. 🌟

Update policy and dynamic-config APIs (1767)

Description of change

- Update etcd calls to be blocking using `etcdClient` directly
- Move away from adding/updating multiple policies and dynamic-configs
in a single call
- Combine a POST and PATCH into 1 call, by using
`google.protobuf.FieldMask`, still keeping both methods and backwards
compatibility with `aperturectl`
- Add FieldMask related helper functions in utils
- Replace "no matches for kind" string check with
`k8s.io/apimachinery/pkg/api/meta` errors check

**New Feature:**
- Combine POST and PATCH into a single call using
`google.protobuf.FieldMask`
- Add FieldMask helper functions in the `utils` package

**Refactor:**
- Update etcd calls to be blocking
- Simplify policy and dynamic-config updates by separating them
- Improve error handling with `k8s.io/apimachinery/pkg/api/meta` errors
check

> 🎉 A merge of POST and PATCH we see, 🔄
> With FieldMask helpers, oh so nifty! 🛠️
> Etcd calls now block, as they should, ⏱️
> And error checks improved, all understood. 🚀

Namespace scoped installation (1764)

Description of change

This PR contains all the changes required for running agent and
controller in namespace scoped:
- Option in Agent and Controller helm chart to install only namespace
scoped installation
- aperturectl support for installation in namespace scoped
- Disable Policy CR watcher conditionally in Controller
- Policy Post and Delete APIs for managing policies

Fixes 1692

Default Playground SS:


Aperture Running in namespace scoped SS with Envoy:


**New Feature:**
- Namespace-scoped installation of Aperture Agent and Controller
- New APIs for managing policies and dynamic configs
- Conditional disabling of Policy CR watcher in Controller
- `aperturectl delete` command for deleting policies

**Documentation:**
- Updated documentation for new installation modes, APIs, and commands

> 🎉 Namespace-scoped, we now install,
> With new APIs, our features fulfill.
> Policies managed, with ease and grace,
> Aperture's updates, we embrace! 🚀

---------

Co-authored-by: Hasit Mistry <hasitfluxninja.com>

GRPC -> gRPC (1762)


Fix metrics (1759)

scheduler: rely on client timeouts (1757)

Description of change

Scheduler logic is changed to ignore the existing `timeout_factor` and
`max_timeout` fields. Instead, the Check() call's client must decide the
timeout for each request. To work with fail-open design of Aperture
clients, `decision_deadline_margin` parameter is added instead.

**New Feature:**
- Rely on client timeouts for scheduler logic
- Add `decision_deadline_margin` parameter for fail-open design

**Refactor:**
- Remove unused imports and minor modifications in load generator test
files

> 🎉 A new dawn for scheduling, we embrace,
> 🕒 Client timeouts now lead the race.
> ⏱️ With margins set, decisions are wise,
> 🧹 Cleaned up imports, a welcomed surprise!

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v1.4.0...releases/aperture-controller/v1.5.0-rc.2

Changelog

List of aperture PRs merged since 1.3.0 release. For the full list of changes, see [list of changes][changes]

Count invalid metrics readings in separate metrics (1736)

**New Feature:**
- Count invalid metrics readings separately and remove `valid="true"`
filter from Prometheus queries
- Add new metrics for invalid signal readings and flux meters
- Update `FluxMeter` interface and related structs with methods for
handling invalid readings

> 🎉 Invalid readings now have their place, 📊
> Separate metrics, a more accurate space. 🚀
> Flux meters adapt, with methods anew, 🌟
> A clearer picture of data comes into view. 🖼️✨

Use the `community/mitchellh-go-json` remote plugin (1749)

- Remove the deprecated remote plugin, move to newly added official
community plugin
- Ref: https://github.com/bufbuild/plugins/issues/489
- Break up apreture buf gen file because `buf generate` now complains if
there are more than 10 remote plugins
- langs for languages remote plugins
- grpc-ecosystem for grpc-ecosystem related remote plugins

Load Shaper component and Feature Rollout scenario (1738)

* New components
* `SignalGenerator`
* `FlowRegulator`
* `LoadShaper`
* New Feature Rollout blueprints
* A scenario and a tutorial

![Screenshot from 2023-04-13
21-50-54](https://user-images.githubusercontent.com/3925292/231948041-a98642cf-2928-4886-a3af-95f5332f1b2f.png)

**New Feature:**
- Added FlowRegulator and LoadShaper components for improved flow
control
- Introduced new feature rollout policies based on average latency, EMA
latency, percentile latency, and PromQL

**Documentation:**
- Updated and added documentation for new components and policies

**Refactor:**
- Improved error handling and function naming in various files

> 🎉 A new dawn for flow control we see, 🌅<br>
> With FlowRegulator and LoadShaper, so free! 🚀<br>
> Feature rollouts now smarter than ever, 🧠<br>
> Latency-based policies, oh so clever! 💡

---------

Co-authored-by: Harjot Gill <harjot.gillfluxninja.com>

add support for vale linting (1718)

Co-authored-by: Sudhanshu Prajapati <spsupraj2dgmail.com>
Co-authored-by: Michał Krasnoborski <michal.krasnoborskifluxninja.com>
Co-authored-by: Daria Bialobrzeska <88393714+DariaKunoichiusers.noreply.github.com>
Co-authored-by: Hasit Mistry <hasitfluxninja.com>

Use new official remote plugins (1739)

**Chore:**
- Updated `protobuf-java-util` dependency version to 3.22.2
- Updated `com.google.protobuf` plugin version to 0.9.2

> 🎉 A protobuf update, so swift and true,
> With versions refreshed, our code feels anew.
> Embrace the improvements, let's celebrate,
> For better compatibility, we now create! 🚀

Improve RequestToInput performance in Check call (1712)

Replaced usage of envoy's RequestToInput with our own which creates ast.Value for Classifier to use directly.

Shrink aperture-operator more by splitting some packages (1722)

Continuation to 1700. Operator shrank from 68M to 57M.

* Extracted config package out of discovery/static, jobs,
extensions/sentry & prometheus to avoid depending on pkg/status.
* stopped depending on batchprocessor in otelcollector/config.
* Removed accidental dependency on gomega.

scheduler: replace audit logic with timeout context (1735)

Refactor:

Replace the scheduler package with a new one
Use timeout context for scheduling requests in RunLimiter
Add Return method to the TokenManager interface and implement it in token bucket classes

Export metrics with pod count (1655)

**New Feature:**
- Added Prometheus metrics and improved error handling to the Kubernetes
Control Point Discovery module
- Introduced a `podNotifier` struct for leader change notifications

**Chore:**
- Updated Python version in CircleCI workflows
- Updated pre-commit hook versions and added codespell hook

> 🎉 A new dawn rises, metrics now gleam,
> 🌟 With pod notifiers, our code's a dream.
> 🐍 Python updated, hooks refined,
> 🚀 This PR's changes, truly shine!

Java SDK: replace envoy authz with checkHttp (1704)

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v1.3.0...releases/aperture-controller/v1.4.0

Changelog

List of aperture PRs merged since 1.3.0 release. For the full list of changes, see [list of changes][changes]

Count invalid metrics readings in separate metrics (1736)

**New Feature:**
- Count invalid metrics readings separately and remove `valid="true"`
filter from Prometheus queries
- Add new metrics for invalid signal readings and flux meters
- Update `FluxMeter` interface and related structs with methods for
handling invalid readings

> 🎉 Invalid readings now have their place, 📊
> Separate metrics, a more accurate space. 🚀
> Flux meters adapt, with methods anew, 🌟
> A clearer picture of data comes into view. 🖼️✨

Use the `community/mitchellh-go-json` remote plugin (1749)

- Remove the deprecated remote plugin, move to newly added official
community plugin
- Ref: https://github.com/bufbuild/plugins/issues/489
- Break up apreture buf gen file because `buf generate` now complains if
there are more than 10 remote plugins
- langs for languages remote plugins
- grpc-ecosystem for grpc-ecosystem related remote plugins

Load Shaper component and Feature Rollout scenario (1738)

* New components
* `SignalGenerator`
* `FlowRegulator`
* `LoadShaper`
* New Feature Rollout blueprints
* A scenario and a tutorial

![Screenshot from 2023-04-13
21-50-54](https://user-images.githubusercontent.com/3925292/231948041-a98642cf-2928-4886-a3af-95f5332f1b2f.png)

**New Feature:**
- Added FlowRegulator and LoadShaper components for improved flow
control
- Introduced new feature rollout policies based on average latency, EMA
latency, percentile latency, and PromQL

**Documentation:**
- Updated and added documentation for new components and policies

**Refactor:**
- Improved error handling and function naming in various files

> 🎉 A new dawn for flow control we see, 🌅<br>
> With FlowRegulator and LoadShaper, so free! 🚀<br>
> Feature rollouts now smarter than ever, 🧠<br>
> Latency-based policies, oh so clever! 💡

---------

Co-authored-by: Harjot Gill <harjot.gillfluxninja.com>

add support for vale linting (1718)

Co-authored-by: Sudhanshu Prajapati <spsupraj2dgmail.com>
Co-authored-by: Michał Krasnoborski <michal.krasnoborskifluxninja.com>
Co-authored-by: Daria Bialobrzeska <88393714+DariaKunoichiusers.noreply.github.com>
Co-authored-by: Hasit Mistry <hasitfluxninja.com>

Use new official remote plugins (1739)

**Chore:**
- Updated `protobuf-java-util` dependency version to 3.22.2
- Updated `com.google.protobuf` plugin version to 0.9.2

> 🎉 A protobuf update, so swift and true,
> With versions refreshed, our code feels anew.
> Embrace the improvements, let's celebrate,
> For better compatibility, we now create! 🚀

Improve RequestToInput performance in Check call (1712)

Replaced usage of envoy's RequestToInput with our own which creates ast.Value for Classifier to use directly.

Shrink aperture-operator more by splitting some packages (1722)

Continuation to 1700. Operator shrank from 68M to 57M.

* Extracted config package out of discovery/static, jobs,
extensions/sentry & prometheus to avoid depending on pkg/status.
* stopped depending on batchprocessor in otelcollector/config.
* Removed accidental dependency on gomega.

scheduler: replace audit logic with timeout context (1735)

Refactor:

Replace the scheduler package with a new one
Use timeout context for scheduling requests in RunLimiter
Add Return method to the TokenManager interface and implement it in token bucket classes

Export metrics with pod count (1655)

**New Feature:**
- Added Prometheus metrics and improved error handling to the Kubernetes
Control Point Discovery module
- Introduced a `podNotifier` struct for leader change notifications

**Chore:**
- Updated Python version in CircleCI workflows
- Updated pre-commit hook versions and added codespell hook

> 🎉 A new dawn rises, metrics now gleam,
> 🌟 With pod notifiers, our code's a dream.
> 🐍 Python updated, hooks refined,
> 🚀 This PR's changes, truly shine!

Java SDK: replace envoy authz with checkHttp (1704)

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v1.3.0...releases/aperture-controller/v1.4.0-rc.2

Changelog

List of aperture PRs merged since 1.3.0 release. For the full list of changes, see [list of changes][changes]

Improve RequestToInput performance in Check call (1712)

Replaced usage of envoy's RequestToInput with our own which creates ast.Value for Classifier to use directly.

Shrink aperture-operator more by splitting some packages (1722)

Continuation to 1700. Operator shrank from 68M to 57M.

* Extracted config package out of discovery/static, jobs,
extensions/sentry & prometheus to avoid depending on pkg/status.
* stopped depending on batchprocessor in otelcollector/config.
* Removed accidental dependency on gomega.

scheduler: replace audit logic with timeout context (1735)

Refactor:

Replace the scheduler package with a new one
Use timeout context for scheduling requests in RunLimiter
Add Return method to the TokenManager interface and implement it in token bucket classes

Export metrics with pod count (1655)

**New Feature:**
- Added Prometheus metrics and improved error handling to the Kubernetes
Control Point Discovery module
- Introduced a `podNotifier` struct for leader change notifications

**Chore:**
- Updated Python version in CircleCI workflows
- Updated pre-commit hook versions and added codespell hook

> 🎉 A new dawn rises, metrics now gleam,
> 🌟 With pod notifiers, our code's a dream.
> 🐍 Python updated, hooks refined,
> 🚀 This PR's changes, truly shine!

Java SDK: replace envoy authz with checkHttp (1704)


[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v1.3.0...releases/aperture-controller/v1.4.0-rc.1