Aperture-py

Changelog

List of aperture PRs merged since 0.7.0 release. For the full list of changes, see [list of changes][changes]

Revamp workload and flux meter metrics and labels (843)

Description of change
* New label `attribute_found` in FluxMeter to denote if the attribute on
which the flux meter is based was found in the access log/span
* Removed label `decision_type` on summary `workload_latency_ms` since
it is now emitted only if response was received.
* New counter `workload_requests_total` to measure the workload
decisions count since the summary does not take into account the
scenarios where response is not received e.g. rejects or connection
resets.
* A new column `response_received` on OLAP Flow events to denote the
case when response is not received.

Ignore negative workload latency (839)

Issue
* Workload latency in case of Envoy is calculated as:

workload_latency = response_latency - aperture_latency

* Workload Latency can become negative in case of connection reset
* If the connection is aborted by Client or Server Envoy immediately
terminates the connection for the other endpoint.
* In the Access Log, status code is set as 0 and `response_latency` is
set as zero.
* If Authz call to Aperture Agent had succeeded for this request, then
aperture_latency is greater than zero.
* This would lead the `workload_latency` to be computed as negative.
![Screenshot from 2022-10-28
19-24-44](https://user-images.githubusercontent.com/3925292/198775984-661e8293-e2eb-4357-a616-8552946572a0.png)

Fix
* Ignore negative workload latency I.E. don't populate the workload
latency column
* Publish Prometheus metrics for flux-meter or workload latency only if
the metric column is found

TickInfo in LoadDecision (836)

Description of change
* Put `TickInfo` in LoadDecision` to re-trigger fill-rate evaluation at
Agent.

Re-structure protos (831)


Fix telemetry labels propagation (835)

Description of change
This fixes regression introduced in
https://github.com/fluxninja/aperture/pull/828.

Dynamic Telemetry Flow Labels were added before labels filtering, which
led them to be incorrectly filtered out.

Fix telemetry labels propagation (835)

Description of change
This fixes regression introduced in
https://github.com/fluxninja/aperture/pull/828.

Dynamic Telemetry Flow Labels were added before labels filtering, which
led them to be incorrectly filtered out.

Bump OTEL to 0.63.0 (834)

Description of change
Bumps OTEL and FN OTEL to 0.63.0. This removes Istio 1.15 compat hack as
it is included in the upstream OTEL.

Response status in telemetry (828)

Description of change
This introduces `aperture.response_status` column in telemetry. It
mirrors the implementation of `response_status` label for metrics.
This also extends above logic to include `1xx`, `2xx`, and `3xx` codes
as OK instead of only `2xx` codes.

Besides this, some cleanup is done:
1. Above logic is moved from `FluxMeter` to OTEL package. **This changes
FluxMeter interface!**
2. A log of logic is moved from `metricsprocessor` to
`metricsprocessor/internal` for better visibility and easier separation
of functions which are called directly in metricsprocessor and helpers,
3. The above made creating UT much easier, so this PR also includes
some.

Ref: https://github.com/fluxninja/cloud/issues/6788

Dry run mode for Load Actuator (826)

Description of change
* Dry run mode for Load Actuator. No traffic can get dropped due to this
Load Actuator in this mode. Useful for observing the behavior of Load
Actuator without any disruptions.
* Load Actuator has a new Pass through mode
* Default to Pass through mode in case multiplier is invalid and also
when there is no decision available at the Agent including
initialization

Rollup based on metrics (821)

Closes: GH-515

docs: playground doc updates (819)

Description of change
- Moved demo_app to playground
- Added more details to playground documentation
- Bump istio and other tools

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v0.7.0...releases/aperture-controller/v0.8.0

Changelog

List of aperture PRs merged since 0.8.0-rc.3 release. For the full list of changes, see [list of changes][changes]

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v0.8.0-rc.3...releases/aperture-controller/v0.8.0-rc.4

Changelog

List of aperture PRs merged since 0.8.0-rc.2 release. For the full list of changes, see [list of changes][changes]

Revamp workload and flux meter metrics and labels (843)

Description of change
* New label `attribute_found` in FluxMeter to denote if the attribute on
which the flux meter is based was found in the access log/span
* Removed label `decision_type` on summary `workload_latency_ms` since
it is now emitted only if response was received.
* New counter `workload_requests_total` to measure the workload
decisions count since the summary does not take into account the
scenarios where response is not received e.g. rejects or connection
resets.
* A new column `response_received` on OLAP Flow events to denote the
case when response is not received.

Skip NaN auto-tokens (840)


Ignore negative workload latency (839)

Issue
* Workload latency in case of Envoy is calculated as:

workload_latency = response_latency - aperture_latency

* Workload Latency can become negative in case of connection reset
* If the connection is aborted by Client or Server Envoy immediately
terminates the connection for the other endpoint.
* In the Access Log, status code is set as 0 and `response_latency` is
set as zero.
* If Authz call to Aperture Agent had succeeded for this request, then
aperture_latency is greater than zero.
* This would lead the `workload_latency` to be computed as negative.
![Screenshot from 2022-10-28
19-24-44](https://user-images.githubusercontent.com/3925292/198775984-661e8293-e2eb-4357-a616-8552946572a0.png)

Fix
* Ignore negative workload latency I.E. don't populate the workload
latency column
* Publish Prometheus metrics for flux-meter or workload latency only if
the metric column is found

Checklist

- [X] Tested in playground or other setup

TickInfo in LoadDecision (836)

Description of change
* Put `TickInfo` in LoadDecision` to re-trigger fill-rate evaluation at
Agent.

Re-structure protos (831)


Fix telemetry labels propagation (835)

Description of change
This fixes regression introduced in
https://github.com/fluxninja/aperture/pull/828.

Dynamic Telemetry Flow Labels were added before labels filtering, which
led them to be incorrectly filtered out.

Checklist

- [x] Tested in playground or other setup
- [x] Breaking changes

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v0.8.0-rc.2...releases/aperture-controller/v0.8.0-rc.3

Changelog

List of aperture PRs merged since 0.8.0-rc.1 release. For the full list of changes, see [list of changes][changes]

Fix telemetry labels propagation (835)

Description of change
This fixes regression introduced in
https://github.com/fluxninja/aperture/pull/828.

Dynamic Telemetry Flow Labels were added before labels filtering, which
led them to be incorrectly filtered out.

Checklist

- [x] Tested in playground or other setup
- [x] Breaking changes

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v0.8.0-rc.1...releases/aperture-controller/v0.8.0-rc.2

Changelog

List of aperture PRs merged since 0.7.0 release. For the full list of changes, see [list of changes][changes]

Bump OTEL to 0.63.0 (834)

Description of change
Bumps OTEL and FN OTEL to 0.63.0. This removes Istio 1.15 compat hack as
it is included in the upstream OTEL.

Checklist

- [x] Tested in playground or other setup
- [x] Breaking changes

Response status in telemetry (828)

Description of change
This introduces `aperture.response_status` column in telemetry. It
mirrors the implementation of `response_status` label for metrics.
This also extends above logic to include `1xx`, `2xx`, and `3xx` codes
as OK instead of only `2xx` codes.

Besides this, some cleanup is done:
1. Above logic is moved from `FluxMeter` to OTEL package. **This changes
FluxMeter interface!**
2. A log of logic is moved from `metricsprocessor` to
`metricsprocessor/internal` for better visibility and easier separation
of functions which are called directly in metricsprocessor and helpers,
3. The above made creating UT much easier, so this PR also includes
some.

Ref: https://github.com/fluxninja/cloud/issues/6788

Checklist

- [x] Tested in playground or other setup
- [x] Tests and/or benchmarks are included
- [x] Breaking changes

Dry run mode for Load Actuator (826)

Description of change
* Dry run mode for Load Actuator. No traffic can get dropped due to this
Load Actuator in this mode. Useful for observing the behavior of Load
Actuator without any disruptions.
* Load Actuator has a new Pass through mode
* Default to Pass through mode in case multiplier is invalid and also
when there is no decision available at the Agent including
initialization

platform: panic handler minor fix (825)

Description of change
minor fix

Checklist

- [X] Tested in playground or other setup

docs: FluxMeter --> Flux Meter (822)

Description of change
Use the term "Flux Meter" instead of "FluxMeter" in the documentation.

Checklist

- [X] Documentation is changed or added

Rollup based on metrics (821)

Closes: GH-515

docs: playground doc updates (819)

Description of change
- Moved demo_app to playground
- Added more details to playground documentation
- Bump istio and other tools

Checklist

- [X] Tested in playground or other setup
- [X] Documentation is changed or added

Fix nil pointer error in runDockerContainer (818)

Description of change

Checklist

- [ ] Tested in playground or other setup
- [ ] Documentation is changed or added
- [ ] Tests and/or benchmarks are included
- [ ] Breaking changes

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg"
height="34" align="absmiddle"
alt="Reviewable"/>](https://reviewable.io/reviews/fluxninja/aperture/818)
<!-- Reviewable:end -->

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v0.7.0...releases/aperture-controller/v0.8.0-rc.1

Changelog

List of aperture PRs merged since 0.6.0 release. For the full list of changes, see [list of changes][changes]

K8s discovery (809)

Description of change
Fixes: 806

Checklist

- [x] Tested in playground or other setup

Co-authored-by: Harjot Gill <harjot.gillfluxninja.com>
Co-authored-by: Krzysztof Kwapisiewicz <krzysztof.kwapisiewiczfluxninja.com>

Docs: Typo and elaborate more about scheduler output signals (801)


api: fix generator on macOS (804)

Fixes pre-commit issue on main

Fix Concurrency Multiplier defaults (802)


Add send_batch_max_size option (792)

Description of change
This ensures that batches are not larger that the given value and will
be split into smaller units.

Checklist

- [x] Tested in playground or other setup
- [x] Documentation is changed or added
- [x] Tests and/or benchmarks are included

Closes: https://github.com/fluxninja/cloud/issues/6644

Update protoc-gen-go-deepcopy and copy go stubs to sdks/aperture-go (789)

This change
- updates protoc-gen-go-deepcopy - previously the generated files were importing github.com/golang/protobuf/proto (now deprecated) instead of google.golang.org/protobuf/proto, I opened a PR on istio's repo and fixed it, updated our buf plugin with v0.2.0 tag as well.
- adds flowcontrol stubs to aperture-go in sdks directory and its mirror - this is so we do not have to depend on buf registry and match it to what is happening in aperture-java.

rollupprocessor: Apply per-batch attribute cardinality limit (782)


Add tutorial for signal processing (784)

Description of change

Checklist

- [X] Tested in playground or other setup
- [X] Documentation is changed or added

Scheduler: Load Multiplier (783)

Description of change
* `LoadActuator` instead of `LoadShedActuator`
* `LoadActuator` takes `LoadMultiplier` instead of `LoadShedFactor`.
* Refactor PromQL results to include the Tick

Move SDKs into aperture repo (751)

Description of change
- Move both sdks under `sdks` directory
- Add jobs for validating both sdks
- Add jobs for mirroring both sdks to their respective repositories

[changes]: https://github.com/fluxninja/aperture/compare/releases/aperture-controller/v0.6.0...releases/aperture-controller/v0.7.0