Public API changes:
- For all of the run_* functions, deployment=None now implies Deployment.mirror_local. Previously, this would mean PreinstalledInterpreter(MEADOWRUN_INTERPRETER)
- Kubernetes(resuable_pods=True) is now the default (previously was False)
- Add Deployment.preinstalled_interpreter
- Remove LocalPipInterpreter(python_version=) parameter, just detect it from the interpreter
- In mirror_local, renamed additional_python_paths to additional_sys_paths, add a parameter include_sys_path_contents, and rename working_directory_globs to globs. globs can now specify files in sys.path (previously was limited to files in the current working directory)
- Add PipRequirementsString as a way to specify an interpreter
New features:
- Add support for using Google Cloud Storage with Kubernetes, means that GKE is supported in a first-class way
- Azure now uses incremental code upload
- Kubernetes has a pod_customization
- extra-index-url/index-url config in pip.conf will be synced automatically by mirror_local
- Include .so files in mirror_local by default
Bug fixes:
- Single use pods were not working
- Reusable pods weren't being killed correctly
- Add support for streaming logs with reusable pods
- Add a Resources(ephemeral_storage_gb) parameter that works only for Kubernetes for now
- Allow functions in run_map to call asyncio.run
- mirror_local can now be reused as long as its with the same host
- Provide a default None value for Kubernetes.storage_spec = None
- If a conda and pip environment were both active, mirror_local would always pick the conda environment even if the pip one was "higher priority"
- Don't fail if a file can't be read for zipping in mirror_local
- Allow using different versions of python with the same pip/poetry environment
-
**Full Changelog**: https://github.com/meadowdata/meadowrun/compare/v0.2.11...v0.2.13