Mlflow

Features:

- Enabled auto dependency inference in spark flavor in autologging (4759, harupy)

Bug fixes and documentation updates:

- Increased MLflow client HTTP request timeout from 10s to 120s (4764, jinzhang21)
- Fixed autologging compatibility bugs with TensorFlow and Keras version `2.6.0` (4766, dbczumar)

Small bug fixes and doc updates (4770, WeichenXu123)

- Avoid calling `importlib_metadata.packages_distributions` upon `mlflow.utils.requirements_utils` import (4741, dbczumar)
- Avoid depending on `importlib_metadata==4.7.0` (4740, dbczumar)

Features:

- Autologging for scikit-learn now records post training metrics when scikit-learn evaluation APIs, such as `sklearn.metrics.mean_squared_error`, are called (4491, 4628 4638, WeichenXu123)
- Autologging for PySpark ML now records post training metrics when model evaluation APIs, such as `Evaluator.evaluate()`, are called (4686, WeichenXu123)
- Add `pip_requirements` and `extra_pip_requirements` to `mlflow.*.log_model` and `mlflow.*.save_model` for directly specifying the pip requirements of the model to log / save (4519, 4577, 4602, harupy)
- Added `stdMetrics` entries to the training metrics recorded during PySpark CrossValidator autologging (4672, WeichenXu123)
- MLflow UI updates:
1. Improved scalability of the parallel coordinates plot for run performance comparison,
2. Added support for filtering runs based on their start time on the experiment page,
3. Added a dropdown for runs table column sorting on the experiment page,
4. Upgraded the AG Grid plugin, which is used for runs table loading on the experiment page, to version 25.0.0,
5. Fixed a bug on the experiment page that caused the metrics section of the runs table to collapse when selecting columns from other table sections (4712, dbczumar)
- Added support for distributed execution to autologging for PyTorch Lightning (4717, dbczumar)
- Expanded R support for Model Registry functionality (4527, bramrodenburg)
- Added model scoring server support for defining custom prediction response wrappers (4611, Ark-kun)
- `mlflow.*.log_model` and `mlflow.*.save_model` now automatically infer the pip requirements of the model to log / save based on the current software environment (4518, harupy)
- Introduced support for running Sagemaker Batch Transform jobs with MLflow Models (4410, 4589, YQ-Wang)

Bug fixes and documentation updates:

- Deprecate `requirements_file` argument for `mlflow.*.save_model` and `mlflow.*.log_model` (4620, harupy)
- set nextPageToken to null (4729, harupy)
- Fix a bug in MLflow UI where the pagination token for run search is not refreshed when switching experiments (4709, harupy)
- Fix a bug in the model scoring server that rejected requests specifying a valid `Content-Type` header with the charset parameter (4609, Ark-kun)
- Fixed a bug that caused SQLAlchemy backends to exhaust DB connections. (4663, arpitjasa-db)
- Improve docker build procedures to raise exceptions if docker builds fail (4610, Ark-kun)
- Disable autologging for scikit-learn `cross_val_*` APIs, which are incompatible with autologging (4590, WeichenXu123)
- Deprecate MLflow Models support for fast.ai V1 (4728, dbczumar)
- Deprecate the old Azure ML deployment APIs `mlflow.azureml.cli.build_image` and `mlflow.azureml.build_image` (4646, trangevi)
- Deprecate MLflow Models support for TensorFlow < 2.0 and Keras < 2.3 (4716, harupy)

Small bug fixes and doc updates (4730, 4722, 4725, 4723, 4703, 4710, 4679, 4694, 4707, 4708, 4706, 4705, 4625, 4701, 4700, 4662, 4699, 4682, 4691, 4684, 4683, 4675, 4666, 4648, 4653, 4651, 4641, 4649, 4627, 4637, 4632, 4634, 4621, 4619, 4622, 4460, 4608, 4605, 4599, 4600, 4581, 4583, 4565, 4575, 4564, 4580, 4572, 4570, 4574, 4576, 4568, 4559, 4537, 4542, harupy; 4698, 4573, Ark-kun; 4674, kvmakes; 4555, vagoston; 4644, zhengjxu; 4690, 4588, apurva-koti; 4545, 4631, 4734, WeichenXu123; 4633, 4292, shrinath-suresh; 4711, jinzhang21; 4688, murilommen; 4635, ryan-duve; 4724, 4719, 4640, 4639, 4629, 4612, 4613, 4586, dbczumar)

Features:

- Add support for plotting per-class feature importance computed on linear boosters in XGBoost autologging (4523, dbczumar)
- Add `mlflow_create_registered_model` and `mlflow_delete_registered_model` for R to create/delete registered models.
- Add support for setting tags while resuming a run (4497, dbczumar)
- MLflow UI updates (4490, sunishsheth2009)

- Add framework for internationalization support.
- Move metric columns before parameter and tag columns in the runs table.
- Change the display format of run start time to elapsed time (e.g. 3 minutes ago) from timestamp (e.g. 2021-07-14 14:02:10) in the runs table.

Bug fixes and documentation updates:

- Fix a bug causing MLflow UI to crash when sorting a column containing both `NaN` and empty values (3409, harupy)

Small bug fixes and doc updates (4541, 4534, 4533, 4517, 4508, 4513, 4512, 4509, 4503, 4486, 4493, 4469, harupy; 4458, KasirajanA; 4501, jimmyxu-db; 4521, 4515, jerrylian-db; 4359, shrinath-suresh; 4544, WeichenXu123; 4549, smurching; 4554, derkomai; 4506, tomasatdatabricks; 4551, 4516, 4494, dbczumar; 4511, keypointt)

Features:

- Autologging performance improvements for XGBoost, LightGBM, and scikit-learn (4416, 4473, dbczumar)
- Add new PaddlePaddle flavor to MLflow Models (4406, 4439, jinminhao)
- Introduce paginated ListExperiments API (3881, wamartin-aml)
- Include Runtime version for MLflow Models logged on Databricks (4421, stevenchen-db)
- MLflow Models now log dependencies in pip requirements.txt format, in addition to existing conda format (4409, 4422, stevenchen-db)
- Add support for limiting the number child runs created by autologging for scikit-learn hyperparameter search models (4382, mohamad-arabi)
- Improve artifact upload / download performance on Databricks (4260, dbczumar)
- Migrate all model dependencies from conda to "pip" section (4393, WeichenXu123)

Bug fixes and documentation updates:

- Fix an MLflow UI bug that caused git source URIs to be rendered improperly (4403, takabayashi)
- Fix a bug that prevented reloading of MLflow Models based on the TensorFlow SavedModel format (4223) (4319, saschaschramm)
- Fix a bug in the behavior of `KubernetesSubmittedRun.get_status()` for Kubernetes MLflow Project runs (3962) (4159, jcasse)
- Fix a bug in TLS verification for MLflow artifact operations on S3 (4047, PeterSulcs)
- Fix a bug causing the MLflow server to crash after deletion of the default experiment (4352, asaf400)
- Fix a bug causing `mlflow models serve` to crash on Windows 10 (4377, simonvanbernem)
- Fix a crash in runs search when ordering by metric values against the MSSQL backend store (2551) (4238, naor2013)
- Fix an autologging incompatibility issue with TensorFlow 2.5 (4371, dbczumar)
- Fix a bug in the `disable_for_unsupported_versions` autologging argument that caused library versions to be incorrectly compared (4303, WeichenXu123)

Small bug fixes and doc updates (4405, mohamad-arabi; 4455, 4461, 4459, 4464, 4453, 4444, 4449, 4301, 4424, 4418, 4417, 3759, 4398, 4389, 4386, 4385, 4384, 4380, 4373, 4378, 4372, 4369, 4348, 4364, 4363, 4349, 4350, 4174, 4285, 4341, harupy; 4446, kHarshit; 4471, AveshCSingh; 4435, 4440, 4368, 4360, WeichenXu123; 4431, apurva-koti; 4428, stevenchen-db; 4467, 4402, 4261, dbczumar)

Features:

- Add support for hyperparameter-tuning models to `mlflow.pyspark.ml.autolog()` (4270, WeichenXu123)

Bug fixes and documentation updates:

- Fix PyTorch Lightning callback definition for compatibility with PyTorch Lightning 1.3.0 (4333, dbczumar)
- Fix a bug in scikit-learn autologging that omitted artifacts for unsupervised models (4325, dbczumar)
- Support logging `datetime.date` objects as part of model input examples (4313, vperiyasamy)
- Implement HTTP request retries in the MLflow Java client for 500-level responses (4311, dbczumar)
- Include a community code of conduct (4310, dennyglee)

Small bug fixes and doc updates (4276, 4263, WeichenXu123; 4289, 4302, 3599, 4287, 4284, 4265, 4266, 4275, 4268, harupy; 4335, 4297, dbczumar; 4324, 4320, tleyden)