Metaflow

Latest version: v2.12.0

Safety actively analyzes 634065 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 21

2.12.0

**Full Changelog**: https://github.com/Netflix/metaflow/compare/2.11.16...2.12.0

2.11.16

What's Changed
* [runtime limits] make runtime limits configurable by valayDave in https://github.com/Netflix/metaflow/pull/1834
* [Ready for Review] fix bug where client can not access foreach stack by darinyu in https://github.com/Netflix/metaflow/pull/1766
* [Ready for Review] add attempt_ok flag so that UI will not show up unknown node by darinyu in https://github.com/Netflix/metaflow/pull/1830
* pluggable azure credentials provider by oavdeev in https://github.com/Netflix/metaflow/pull/1756
* Secret Backend Support for Azure Key Vault by iamsgarg-ob in https://github.com/Netflix/metaflow/pull/1839
* reducing the dep version to 4.7.0 (47) by iamsgarg-ob in https://github.com/Netflix/metaflow/pull/1840
* [parallel on Kubernetes] support for Jobsets by valayDave in https://github.com/Netflix/metaflow/pull/1804
* Pluggable GCP auth by oavdeev in https://github.com/Netflix/metaflow/pull/1841
* GCP secret manager support by oavdeev in https://github.com/Netflix/metaflow/pull/1842
* [jobsets] py3.5 compatibility fixes. by valayDave in https://github.com/Netflix/metaflow/pull/1844
* Support Python 3.5 for tests by savingoyal in https://github.com/Netflix/metaflow/pull/1843
* py3.5 compatibility fixes [azure/gcp/jobsets] by valayDave in https://github.com/Netflix/metaflow/pull/1848
* [version bump] for release by valayDave in https://github.com/Netflix/metaflow/pull/1847
* [OB-625] adding metaflow/cron annotation to argo workflows by iamsgarg-ob in https://github.com/Netflix/metaflow/pull/1852

New Contributors
* iamsgarg-ob made their first contribution in https://github.com/Netflix/metaflow/pull/1839

**Full Changelog**: https://github.com/Netflix/metaflow/compare/2.11.15...2.11.16

2.11.15

Features

Displaying task attempt logs
When running a task with the `retry` decorator, previously we were able to only view the logs of the latest attempt of the task.
With this release it is now possible to target a specific attempt with the `--attempt` option for the logs command


python example.py logs 123/retry_step/456 --attempt 2


Scrubbing log contents
This release introduces a new command for scrubbing log contents of tasks in case they contain sensitive information that needs to be redacted.

Simplest use case is scrubbing the latest task logs. By default both `stdout` and `stderr` are scrubbed

python example.py logs scrub 123/example/456


There are also options to target only a specific log stream

python example.py logs scrub 123/example/456 --stderr
python example.py logs scrub 123/example/456 --stdout


when using the`retry` decorator, tasks can have multiple attempts with separate logs that require scrubbing. By default only the latest attempt is scrubbed. There are options to make scrubbing multiple attempts easier

scrub specific attempt
python example.py logs scrub 123/retry_step/456 --attempt 1

scrub all attempts
python example.py logs scrub 123/retry_step/456 --all

scrub specified attempt and all prior to it (this would scrub attempts 0,1,2,3)
python example.py logs scrub 123/retry_step/456 --all --attempt 3


The command also accepts only specifying a step for scrubbing. This is useful for steps with multiple tasks, like a foreach split.

python example.py logs scrub 123/foreach_step

all the above options also apply when targeting a step for scrubbing.

*Note*: Log scrubbing for running tasks is not recommended, and is actively protected against. There can be occasions where a task has failed in such a way that it still counts as not completed. For such a case you can supply the `--include-not-done` option to try and scrub it as well.

What's Changed
* feature: scrub logs by saikonen in https://github.com/Netflix/metaflow/pull/1802
* Fix issue 1805 by narayanacharya6 in https://github.com/Netflix/metaflow/pull/1807
* bug fix: unused arg for batch step --help by mae5357 in https://github.com/Netflix/metaflow/pull/1817
* bump version to 2.11.15 by saikonen in https://github.com/Netflix/metaflow/pull/1829

New Contributors
* narayanacharya6 made their first contribution in https://github.com/Netflix/metaflow/pull/1807
* mae5357 made their first contribution in https://github.com/Netflix/metaflow/pull/1817

**Full Changelog**: https://github.com/Netflix/metaflow/compare/2.11.14...2.11.15

2.11.14

What's Changed
* Increase Azure Blobstore connection_timeout. by shrinandj in https://github.com/Netflix/metaflow/pull/1827
* bump version to 2.11.14 by shrinandj in https://github.com/Netflix/metaflow/pull/1828


**Full Changelog**: https://github.com/Netflix/metaflow/compare/2.11.13...2.11.14

2.11.13

Features

Configurable default Kubernetes resources
This release introduces configuration options for setting default values for `cpu / memory / disk` when running on Kubernetes. These can be set either with environment variables

METAFLOW_KUBERNETES_CPU=
METAFLOW_KUBERNETES_MEMORY=
METAFLOW_KUBERNETES_DISK=

or in a Metaflow profile
json
{
"KUBERNETES_CPU": "",
"KUBERNETES_MEMORY": "",
"KUBERNETES_DISK": "",
}


These values will be overruled by specifying a value through the `kubernetes` or `resources` decorators.

Improvements

Support for wider foreach flows with Argo Workflows
This release changes the way task ids are generated on Argo Workflows in order to solve an issue where extremely wide foreach splits could not execute correctly due to hard limits on input parameters size on Argo Workflows.

What's Changed
* fix: Deterministic foreach task id's for Argo Workflows by saikonen in https://github.com/Netflix/metaflow/pull/1704
* Make Kubernetes default resources configurable by martinhausio in https://github.com/Netflix/metaflow/pull/1800
* Add METAFLOW_ESCAPE_HATCH_WARNING for s3op by default. by romain-intel in https://github.com/Netflix/metaflow/pull/1825
* bump version to 2.11.13 by saikonen in https://github.com/Netflix/metaflow/pull/1826

New Contributors
* martinhausio made their first contribution in https://github.com/Netflix/metaflow/pull/1800

**Full Changelog**: https://github.com/Netflix/metaflow/compare/2.11.12...2.11.13

2.11.12

What's Changed
* Fix: JSON Reference Path Error in AWS Step Functions Distributed Map by nidhinnru in https://github.com/Netflix/metaflow/pull/1822
* fix import of the new escape hatch flag by wangchy27 in https://github.com/Netflix/metaflow/pull/1823

New Contributors
* nidhinnru made their first contribution in https://github.com/Netflix/metaflow/pull/1822

**Full Changelog**: https://github.com/Netflix/metaflow/compare/2.11.11...2.11.12

Page 1 of 21

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.