Cortex

**New features**

* Support configuring the log level for APIs ([docs](https://docs.cortex.dev/v/0.26/workloads/realtime/configuration)) https://github.com/cortexlabs/cortex/pull/1741 https://github.com/cortexlabs/cortex/issues/1484 ([RobertLucian](https://github.com/RobertLucian))
* Support creating a cluster in an existing AWS VPC ([docs](https://docs.cortex.dev/v/0.26/clusters/aws/install)) https://github.com/cortexlabs/cortex/pull/1759 https://github.com/cortexlabs/cortex/issues/1142 ([deliahu](https://github.com/deliahu))
* Support specifying the GCP network and subnet for the Cortex cluster ([docs](https://docs.cortex.dev/v/0.26/clusters/gcp/install)) https://github.com/cortexlabs/cortex/pull/1752 https://github.com/cortexlabs/cortex/issues/1738 ([deliahu](https://github.com/deliahu))
* Support configuring shared memory size (shm) for inter-process communication ([docs](https://docs.cortex.dev/v/0.26/workloads/realtime/configuration)) https://github.com/cortexlabs/cortex/pull/1756 https://github.com/cortexlabs/cortex/issues/1638 ([vishalbollu](https://github.com/vishalbollu))

**Breaking changes**

* The local provider has been removed. The best way to test your predictor implementation locally is to import it in a separate Python file and call your `__init__()` and `predict()` functions directly. The best way to test your API is to deploy it to a dev/test cluster.
* Built-in support for API Gateway has been removed. If you need to create an https endpoint with valid certs, some options are to set up a [custom domain](https://docs.cortex.dev/v/0.26/clusters/aws/index/custom-domain) or to [manually create an API Gateway](https://docs.cortex.dev/v/0.26/clusters/aws/index/https).
* Prediction monitoring has been removed. We are exploring how to build a more powerful and customizable solution for this.
* The `predict` CLI command has been deleted. `curl`, `requests`, etc. are the best tools for testing APIs.

**Bug fixes**

* For multi-model APIs, allow model names to share a prefix https://github.com/cortexlabs/cortex/pull/1745 https://github.com/cortexlabs/cortex/issues/1699 ([RobertLucian](https://github.com/RobertLucian))

**Docs**

* Misc docs improvements ([ospillinger](https://github.com/ospillinger))

**New features**

* Support server-side micro batching for the Python predictor ([docs](https://www.docs.cortex.dev/v/0.25/workloads/realtime/server-side-batching)) https://github.com/cortexlabs/cortex/pull/1653 https://github.com/cortexlabs/cortex/issues/1382 ([miguelvr](https://github.com/miguelvr))
* Add timeout configuration for batch jobs ([docs](https://www.docs.cortex.dev/v/0.25/workloads/batch/endpoints)) https://github.com/cortexlabs/cortex/pull/1712 https://github.com/cortexlabs/cortex/issues/1324 ([vishalbollu](https://github.com/vishalbollu))
* Support batch retries ([docs](https://www.docs.cortex.dev/v/0.25/workloads/batch/endpoints)) https://github.com/cortexlabs/cortex/pull/1713 https://github.com/cortexlabs/cortex/issues/1540 ([lapaniku](https://github.com/lapaniku), [vishalbollu](https://github.com/vishalbollu))
* Support sending failed batches to a dead-letter queue ([docs](https://www.docs.cortex.dev/v/0.25/workloads/batch/endpoints)) https://github.com/cortexlabs/cortex/pull/1713 https://github.com/cortexlabs/cortex/issues/1541 ([lapaniku](https://github.com/lapaniku), [vishalbollu](https://github.com/vishalbollu))
* Support installing the cortex Python client in predictors https://github.com/cortexlabs/cortex/pull/1709 https://github.com/cortexlabs/cortex/issues/1670 https://github.com/cortexlabs/cortex/issues/1206 ([RobertLucian](https://github.com/RobertLucian))

**Breaking changes**

* The `predictor.model_path` field of the realtime api configuration has been moved to `predictor.models.path`. In addition, for the Python predictor type, `predictor.models` has been renamed to `predictor.multi_model_reloading`. Here is the entire [API configuration schema](https://www.docs.cortex.dev/v/0.25/workloads/realtime/configuration).

**Bug fixes**

* Misc batch reliability improvements https://github.com/cortexlabs/cortex/pull/1705 https://github.com/cortexlabs/cortex/pull/1718 https://github.com/cortexlabs/cortex/pull/1729 ([vishalbollu](https://github.com/vishalbollu))

**Docs**

* Reorganize the [docs](https://www.docs.cortex.dev) structure https://github.com/cortexlabs/cortex/pull/1696 https://github.com/cortexlabs/cortex/pull/1701 https://github.com/cortexlabs/cortex/pull/1704 https://github.com/cortexlabs/cortex/pull/1719 https://github.com/cortexlabs/cortex/issues/1675 ([ospillinger](https://github.com/ospillinger))
* Add GCP to the [contributing guide](https://github.com/cortexlabs/cortex/blob/0.25/CONTRIBUTING.md) https://github.com/cortexlabs/cortex/pull/1720 https://github.com/cortexlabs/cortex/issues/1654 ([deliahu](https://github.com/deliahu))
* Add docs for [setting up kubectl on GCP](https://www.docs.cortex.dev/v/0.25/clusters/gcp/kubectl) https://github.com/cortexlabs/cortex/commit/759b4b144c25cc623e1b385b036f83825d122db7 ([deliahu](https://github.com/deliahu))

**Misc**

* Parse the request body as a string when content type `text/plain` is specified https://github.com/cortexlabs/cortex/pull/1714 ([deliahu](https://github.com/deliahu))
* Support paths to single ONNX files in API configuration https://github.com/cortexlabs/cortex/pull/1711 https://github.com/cortexlabs/cortex/issues/1686 ([RobertLucian](https://github.com/RobertLucian))
* Support deploying public S3 models on GCP, and public GCS models on AWS https://github.com/cortexlabs/cortex/pull/1694 https://github.com/cortexlabs/cortex/issues/1684 ([RobertLucian](https://github.com/RobertLucian))
* Pre-download docker images when creating GCP clusters https://github.com/cortexlabs/cortex/pull/1721 https://github.com/cortexlabs/cortex/issues/1658 ([deliahu](https://github.com/deliahu))
* Speed up the validation processes for multi-model APIs https://github.com/cortexlabs/cortex/pull/1690 https://github.com/cortexlabs/cortex/issues/1663 ([RobertLucian](https://github.com/RobertLucian))

**Bug fixes**

* Propagate the exit code from the predictor's initialization so that the API status is set to "error" when initialization fails https://github.com/cortexlabs/cortex/issues/1680 https://github.com/cortexlabs/cortex/pull/1691 ([RobertLucian](https://github.com/RobertLucian))

**New features**

* Add GCP support: our initial release supports all three predictor types (Python, TensorFlow, ONNX), on CPU or GPU, with live reloading, multi-model caching, and cluster autoscaling https://github.com/cortexlabs/cortex/pull/1655 https://github.com/cortexlabs/cortex/pull/1672 https://github.com/cortexlabs/cortex/pull/1667 https://github.com/cortexlabs/cortex/issues/1661 https://github.com/cortexlabs/cortex/issues/114 https://github.com/cortexlabs/cortex/issues/1600 https://github.com/cortexlabs/cortex/issues/1602 https://github.com/cortexlabs/cortex/issues/1616 https://github.com/cortexlabs/cortex/issues/1624 ([RobertLucian](https://github.com/RobertLucian), [deliahu](https://github.com/deliahu), [vishalbollu](https://github.com/vishalbollu))
* Add the patch command to the CLI and [Python client](https://docs.cortex.dev/v/0.24/workloads/python-client), which can be used to update an API using only the API configuration (without needing to provide the predictor's Python implementation) https://github.com/cortexlabs/cortex/pull/1651 https://github.com/cortexlabs/cortex/pull/1666 https://github.com/cortexlabs/cortex/issues/1329 ([vishalbollu](https://github.com/vishalbollu))
* Support deploying predictor Python classes from the Python client https://github.com/cortexlabs/cortex/pull/1587 https://github.com/cortexlabs/cortex/issues/1617 (see the [tutorial](https://docs.cortex.dev/v/0.24/tutorials/realtime) for an example) ([vishalbollu](https://github.com/vishalbollu))

**Breaking changes**

* The Python client's `deploy()` function has been renamed to `create_api()`, and some of the argument names have changed ([docs](https://docs.cortex.dev/v/0.24/workloads/python-client))

**Bug fixes**

* Enable CORS for APIs accessed via API Gateway or load balancer https://github.com/cortexlabs/cortex/pull/1649 https://github.com/cortexlabs/cortex/issues/1234 ([RobertLucian](https://github.com/RobertLucian), [deliahu](https://github.com/deliahu))
* Fix local TensorFlow models when live reloading is enabled https://github.com/cortexlabs/cortex/pull/1668 https://github.com/cortexlabs/cortex/issues/1554 ([RobertLucian](https://github.com/RobertLucian))
* Prevent TensorFlow multi-model caching from attempting to download local models from S3 https://github.com/cortexlabs/cortex/pull/1669 https://github.com/cortexlabs/cortex/issues/1598 ([RobertLucian](https://github.com/RobertLucian))

**Docs**

* Miscellaneous docs improvements ([vishalbollu](https://github.com/vishalbollu), [ospillinger](https://github.com/ospillinger))

**Misc**

* Improve Python client cross Python version compatibility https://github.com/cortexlabs/cortex/pull/1640 ([vishalbollu](https://github.com/vishalbollu))
* Reinstall TensorFlow and ONNX dependencies when the Python version is overridden https://github.com/cortexlabs/cortex/pull/1652 ([vishalbollu](https://github.com/vishalbollu))
* Terminate container when bootloader script fails https://github.com/cortexlabs/cortex/pull/1639 ([vishalbollu](https://github.com/vishalbollu))

**New features**

* Update Python client `deploy()` to accept a Python dictionary for API configuration (previously, only a file path was supported) ([docs](https://docs.cortex.dev/v/0.23/miscellaneous/python-client#deploy)) https://github.com/cortexlabs/cortex/pull/1587 ([vishalbollu](https://github.com/vishalbollu))
* Show API deployment history in `cortex get API_NAME` command https://github.com/cortexlabs/cortex/pull/1544 https://github.com/cortexlabs/cortex/issues/1496 ([deliahu](https://github.com/deliahu))
* Add `cortex export API_NAME` and `cortex export API_NAME API_ID` commands to export specific and historical API deployments https://github.com/cortexlabs/cortex/pull/1544 https://github.com/cortexlabs/cortex/issues/1497 ([deliahu](https://github.com/deliahu))
* Build and push `python-predictor-gpu-slim` image with different combinations of cuda and cudnn (`cuda10.0-cudnn7`, `cuda10.1-cudnn7`, `cuda10.1-cudnn8`, `cuda10.2-cudnn7`, `cuda10.2-cudnn8`, `cuda11.0-cudnn8`, `cuda11.1-cudnn8`) ([docs](https://docs.cortex.dev/v/0.23/advanced/system-packages#custom-docker-image)) https://github.com/cortexlabs/cortex/pull/1575 https://github.com/cortexlabs/cortex/issues/1574 ([deliahu](https://github.com/deliahu))

**Bug fixes**

* Allow local deployments of public S3 models without requiring AWS credentials https://github.com/cortexlabs/cortex/pull/1589 https://github.com/cortexlabs/cortex/issues/1588 ([RobertLucian](https://github.com/RobertLucian))

**Docs**

* Add guide for [avoiding Docker Hub rate limits](https://docs.cortex.dev/v/0.23/guides/docker-hub-rate-limiting) https://github.com/cortexlabs/cortex/pull/1576 ([RobertLucian](https://github.com/RobertLucian), [deliahu](https://github.com/deliahu))
* Add guide for [self-hosting Cortex's Docker images](https://docs.cortex.dev/v/0.23/guides/self-hosted-images) https://github.com/cortexlabs/cortex/pull/1579 ([RobertLucian](https://github.com/RobertLucian), [deliahu](https://github.com/deliahu))

**Misc**

* Remove API request maximum payload size limit https://github.com/cortexlabs/cortex/pull/1583 ([deliahu](https://github.com/deliahu))
* Switch to Quay docker container registry https://github.com/cortexlabs/cortex/pull/1578 ([deliahu](https://github.com/deliahu), [RobertLucian](https://github.com/RobertLucian))

**Bug fixes**

* Set the predictor's working directory to the root Cortex project directory https://github.com/cortexlabs/cortex/pull/1573 https://github.com/cortexlabs/cortex/issues/1572 ([deliahu](https://github.com/deliahu))
* Allow `max_instances` to be updated via `cortex cluster configure` https://github.com/cortexlabs/cortex/pull/1568 https://github.com/cortexlabs/cortex/issues/1567 ([deliahu](https://github.com/deliahu))
* Gracefully stop the serving container when a multi-processed cron throws exception https://github.com/cortexlabs/cortex/pull/1560 https://github.com/cortexlabs/cortex/issues/1552 ([RobertLucian](https://github.com/RobertLucian))

**Docs**

* Demonstrate how to make API requests with various payload types (binary, form fields, etc), and show how to access them in `predict()` https://github.com/cortexlabs/cortex/pull/1566 ([docs](https://docs.cortex.dev/v/0.22/deployments/realtime-api/predictors#api-requests))
* Misc docs improvements https://github.com/cortexlabs/cortex/pull/1551 https://github.com/cortexlabs/cortex/pull/1556 c3dab4045a61703cb1db1d5f95776614252f96c0 https://github.com/cortexlabs/cortex/pull/1557 ([deliahu](https://github.com/deliahu), [RobertLucian](https://github.com/RobertLucian))

**Misc**

* Build and upload the Python package/CLI to a public S3 bucket https://github.com/cortexlabs/cortex/pull/1562 ([vishalbollu](https://github.com/vishalbollu))