CVE-2022-21699

Advisory

Flytekit 0.30.0 updates its dependency 'ipython' to v7.31.1 to include a security fix.

Affected package

Affected versions

Fixed versions

Vulnerability changelog

---
**Important Notes:**

* This release introduces a new type (see the section on Structured Dataset below). Naturally this means changes to the IDL, which means that tools that work with Flyte entities encode the new type will also need to be updated. For most readers of this note, that means `FlyteRemote` (which comes with `flytekit`).
* **Note:** This feature is turned off by default (will be on by default at the 1.0.0 release) because this new type needs a Propeller and Admin upgrade. Turn it on via the env var `FLYTE_SDK_USE_STRUCTURED_DATASET=TRUE`. See more below.
* This release also removes the legacy API (see more below). Because of this change, if you bump up your `flytekit` version, you'll also need to bump all your `flytekitplugins-` to the same version as well, otherwise you might see missing imports.
* Lastly, this release skips a few version numbers from 0.26 to 0.30 for future-proofing.

---

Notable Changes

New Type - Structured Dataset
To handle the new type, you will need to be on Propeller version v0.16.14 and Admin version v0.6.78 or later. Because of this backend upgrade restriction, we've decided to feature-gate this new type. To turn this feature on, you will need to set the environment variable:


FLYTE_SDK_USE_STRUCTURED_DATASET=TRUE


Note that we'll be turning on this feature by default at the 1.0.0 release of flytekit (and Flyte generally) in Q2.

If turned on, dataframe types (e.g. `pandas` or `pyspark` and others) will be converted into the new `StructuredDataset` [type](https://github.com/flyteorg/flyteidl/pull/227/files#diff-da0cd41ae99f181280058bc79cbc3f3565a22039fbefcbe5f191bc2780d72735R46) and [literal](https://github.com/flyteorg/flyteidl/pull/227/files#diff-e9c7f9528aac9670ee8251a31dd27a04b07bd1afbb195f053ad0182e39eb2119R66).

Please see the [PR](https://github.com/flyteorg/flytekit/pull/785) description for the full details, extension interfaces, and design, as well as the upcoming OSS meeting (Tuesday Feb. 8th) for a deeper discussion. The design doc also is [here](https://docs.google.com/document/d/1-t8C4vW_RhrJoC5Ju16fmqpyOvsCRb3XmulLLo5VX0E/edit).

Removal of legacy API

We've removed the old [legacy API](https://github.com/flyteorg/flytekit/pull/807). This helped remove over 20k lines of code, reduces complexity, and makes the ongoing maintenance load lighter. As part of this cleanup however, some functionality in `flyte-cli` and (`pyflyte`) will be going away. Most of this functionality though has been in `flytectl` for some time. Please let us know if you have any problems.

`flyte-cli`
* `launch-task`
* `execute-launch-plan`
* `watch-execution`
* `relaunch-execution`

These were all commands that relied on the old typing system that was removed.

`pyflyte`

* `register` and `fast-register`
* `launchplan` commands

These were commands that have been deprecated for some time because they communicated with the Flyte control plane (which has been taken over by `flytectl` and/or relied on the old typing system that was removed.)

---

Auto-generated Notes

What's Changed
* bump docsearch version by samhita-alla in https://github.com/flyteorg/flytekit/pull/805
* update docs for new navbar theme by cosmicBboy in https://github.com/flyteorg/flytekit/pull/806
* fix github security issue in requirments by evalsocket in https://github.com/flyteorg/flytekit/pull/810
* Add sphinx panels by cosmicBboy in https://github.com/flyteorg/flytekit/pull/815
* Schema overhaul by pingsutw in https://github.com/flyteorg/flytekit/pull/785
* Parent workflow serialization fails when calling a launch plan with fixed inputs by pingsutw in https://github.com/flyteorg/flytekit/pull/814
* Fix sagemaker plugin by eapolinario in https://github.com/flyteorg/flytekit/pull/817
* Remove legacy API by wild-endeavor in https://github.com/flyteorg/flytekit/pull/807
* Add Bigquery plugin by pingsutw in https://github.com/flyteorg/flytekit/pull/789
* Fixed flytekit-papermill ImportError by pingsutw in https://github.com/flyteorg/flytekit/pull/818
* Add support for string-format-like sytax for shell task by palchicz in https://github.com/flyteorg/flytekit/pull/792
* Expose configured RawOutputPrefix during execution by pingsutw in https://github.com/flyteorg/flytekit/pull/813
* Add secretsmanager back by wild-endeavor in https://github.com/flyteorg/flytekit/pull/820
* Add some tests by wild-endeavor in https://github.com/flyteorg/flytekit/pull/819
* Fix flaky test: test_spark_task by pingsutw in https://github.com/flyteorg/flytekit/pull/821
* [Snyk] Security upgrade ipython from 7.29.0 to 7.31.1 by snyk-bot in https://github.com/flyteorg/flytekit/pull/823
* [Snyk] Security upgrade ipython from 7.29.0 to 7.31.1 by snyk-bot in https://github.com/flyteorg/flytekit/pull/825
* [Snyk] Security upgrade ipython from 7.30.1 to 7.31.1 by snyk-bot in https://github.com/flyteorg/flytekit/pull/826
* [Snyk] Security upgrade ipython from 7.30.1 to 7.31.1 by snyk-bot in https://github.com/flyteorg/flytekit/pull/824
* Intratask checkpointing by kumare3 in https://github.com/flyteorg/flytekit/pull/771
* Support reading subset column by pingsutw in https://github.com/flyteorg/flytekit/pull/822
* Fix spark regression by pingsutw in https://github.com/flyteorg/flytekit/pull/830
* Update argument setting for in fast registered, dynamically generated, pod tasks by wild-endeavor in https://github.com/flyteorg/flytekit/pull/835
* `ctx` Context can be used within shell tasks - to access context vars and secrets by kumare3 in https://github.com/flyteorg/flytekit/pull/832
* Expose Checkpoint as a top-level interface in flytekit by kumare3 in https://github.com/flyteorg/flytekit/pull/839
* Parse duration field from flyteidl to `flytekit.models.execution.ExecutionClosure` by bstadlbauer in https://github.com/flyteorg/flytekit/pull/829
* Gate new Structured Dataset feature by wild-endeavor in https://github.com/flyteorg/flytekit/pull/831
* Fixing out of order for conditional outputs by kumare3 in https://github.com/flyteorg/flytekit/pull/843

New Contributors
* palchicz made their first contribution in https://github.com/flyteorg/flytekit/pull/792

**Full Changelog**: https://github.com/flyteorg/flytekit/compare/v0.26.0...v0.30.0

Resources

• https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-21699