Bentoml

🍱 BentoML has just released `v1.0.6` featuring the [gRPC](https://grpc.io/) preview! Without changing a line of code, you can now serve your Bentos as a gRPC service. Similar to serving over HTTP, BentoML gRPC supports all the [ML frameworks](https://docs.bentoml.org/en/latest/frameworks/index.html), [observability features](https://docs.bentoml.org/en/latest/guides/tracing.html), [adaptive batching](https://docs.bentoml.org/en/latest/guides/batching.html), and more out-of-the-box, simply by calling the `serve-grpc` CLI command.

bash
> pip install "bentoml[grpc]"
> bentoml serve-grpc iris_classifier:latest --production


- Checkout our updated [tutorial](https://docs.bentoml.org/en/latest/tutorial.html) for a quick 10-minute crash course of BentoML gRPC.
- Review the standardized [Protobuf definition of service APIs and IO types](https://docs.bentoml.org/en/latest/guides/grpc.html#protobuf-definition), NDArray, DataFrame, File/Image, JSON, etc.
- Learn more about [multi-language client support (Python, Go, Java, Node.js, etc)](https://docs.bentoml.org/en/latest/guides/grpc.html#client-implementation) with working examples.
- Customize gRPC service by [mounting new servicers](https://docs.bentoml.org/en/latest/guides/grpc.html#mounting-servicer) and [interceptors](https://docs.bentoml.org/en/latest/guides/grpc.html#mounting-grpc-interceptors).

⚠️ gRPC is current under preview. The public APIs may undergo incompatible changes in the future patch releases until the official `v1.1.0` minor version release.

- Enhanced [access logging format](https://docs.bentoml.org/en/latest/guides/logging.html#logging-formatting) to output Trace and Span IDs in the more standard hex encoding by default.
- Added request total, duration, and in-progress metrics to Runners, in addition to API Servers.
- Added support for XGBoost SKLearn models.
- Added support for restricting image mime types in the [Image IO descriptor](https://docs.bentoml.org/en/latest/reference/api_io_descriptors.html#images).

🥂 We’d like to thank our community for their contribution and support.

- Shout out to benjamintanweihao for fixing a BentoML CLI bug.
- Shout out to lsh918 for mixing a PyTorch framework issue.
- Shout out to jeffthebear for enhancing the Pandas DataFrame OpenAPI schema.
- Shout out to jiewpeng for adding the support for customizing access logs with Trace and Span ID formats.

What's Changed
* fix: log runner errors explicitly by ssheng in https://github.com/bentoml/BentoML/pull/2952
* ci: temp fix for models test by sauyon in https://github.com/bentoml/BentoML/pull/2949
* fix: fix context parameter for multi-input IO descriptors by sauyon in https://github.com/bentoml/BentoML/pull/2948
* fix: use `torch.from_numpy()` instead of `torch.Tensor()` to keep data type by lsh918 in https://github.com/bentoml/BentoML/pull/2951
* docs: fix wrong name for example neural net by ssun-g in https://github.com/bentoml/BentoML/pull/2959
* docs: fix bentoml containerize command help message by aarnphm in https://github.com/bentoml/BentoML/pull/2957
* chore(cli): remove unused `--no-trunc` by benjamintanweihao in https://github.com/bentoml/BentoML/pull/2965
* fix: relax regex for setting environment variables by benjamintanweihao in https://github.com/bentoml/BentoML/pull/2964
* docs: update wrong paths for disabling logs by creativedutchmen in https://github.com/bentoml/BentoML/pull/2974
* feat: track serve update for start subcommands by ssheng in https://github.com/bentoml/BentoML/pull/2976
* feat: logging customization by jiewpeng in https://github.com/bentoml/BentoML/pull/2961
* chore(cli): using quotes instead of backslash by sauyon in https://github.com/bentoml/BentoML/pull/2981
* feat(cli): show full tracebacks in debug mode by sauyon in https://github.com/bentoml/BentoML/pull/2982
* feature(runner): add multiple output support by larme in https://github.com/bentoml/BentoML/pull/2912
* docs: add airflow integration page by parano in https://github.com/bentoml/BentoML/pull/2990
* chore(ci): fix the unit test of transformers by bojiang in https://github.com/bentoml/BentoML/pull/3003
* chore(ci): fix the issue caused by the change of check_task by bojiang in https://github.com/bentoml/BentoML/pull/3004
* fix(multipart): support multipart file inputs to non-file descriptors by sauyon in https://github.com/bentoml/BentoML/pull/3005
* feat(server): add runner metrics; refactoring batch size metrics by bojiang in https://github.com/bentoml/BentoML/pull/2977
* EXPERIMENTAL: gRPC support by aarnphm in https://github.com/bentoml/BentoML/pull/2808
* fix(runner): receive requests before cork by bojiang in https://github.com/bentoml/BentoML/pull/2996
* fix(server): service_name label of runner metrics by bojiang in https://github.com/bentoml/BentoML/pull/3008
* chore(misc): remove mentioned for team member from PR request by aarnphm in https://github.com/bentoml/BentoML/pull/3009
* feat(xgboost): support xgboost sklearn models by sauyon in https://github.com/bentoml/BentoML/pull/2997
* feat(io/image): allow restricting mime types by sauyon in https://github.com/bentoml/BentoML/pull/2999
* fix(grpc): docker message by aarnphm in https://github.com/bentoml/BentoML/pull/3012
* fix: broken legacy metrics by aarnphm in https://github.com/bentoml/BentoML/pull/3019
* fix(e2e): exception test for image IO by aarnphm in https://github.com/bentoml/BentoML/pull/3017
* revert(3017): filter write-only mime type for Image IO by bojiang in https://github.com/bentoml/BentoML/pull/3020
* chore: cleanup containerize utils by aarnphm in https://github.com/bentoml/BentoML/pull/3014
* feat(proto): add `serialized_bytes` to `pb.Part` by aarnphm in https://github.com/bentoml/BentoML/pull/3022
* docs: Update README.md by parano in https://github.com/bentoml/BentoML/pull/3023
* chore(grpc): vcs generated stubs by aarnphm in https://github.com/bentoml/BentoML/pull/3016
* feat(io/image): allow writeable mimes as output by sauyon in https://github.com/bentoml/BentoML/pull/3024
* docs: fix descriptor typo by darioarias in https://github.com/bentoml/BentoML/pull/3027
* fix(server): log localhost instead of 0.0.0.0 by sauyon in https://github.com/bentoml/BentoML/pull/3033
* fix(io): Pandas OpenAPI schema by jeffthebear in https://github.com/bentoml/BentoML/pull/3032
* chore(docker): support more cuda versions by larme in https://github.com/bentoml/BentoML/pull/3035
* docs: updates on blocks that failed to render by aarnphm in https://github.com/bentoml/BentoML/pull/3031
* chore: migrate to pyproject.toml by aarnphm in https://github.com/bentoml/BentoML/pull/3025
* docs: gRPC tutorial by aarnphm in https://github.com/bentoml/BentoML/pull/3013
* docs: gRPC advanced guides by aarnphm in https://github.com/bentoml/BentoML/pull/3034
* feat(configuration): override options with envvar by bojiang in https://github.com/bentoml/BentoML/pull/3018
* chore: update links by aarnphm in https://github.com/bentoml/BentoML/pull/3040
* fix(configuration): should validate config early by aarnphm in https://github.com/bentoml/BentoML/pull/3041
* qa(bentos): update latest options by aarnphm in https://github.com/bentoml/BentoML/pull/3042
* qa: ignore tools from distribution by aarnphm in https://github.com/bentoml/BentoML/pull/3045
* dependencies: ignore broken pypi combination by aarnphm in https://github.com/bentoml/BentoML/pull/3043
* feat: gRPC tracking by aarnphm in https://github.com/bentoml/BentoML/pull/3015
* configuration: migrate schema to `api_server` by ssheng in https://github.com/bentoml/BentoML/pull/3046
* qa: cleanup MLflow by aarnphm in https://github.com/bentoml/BentoML/pull/2945

New Contributors
* lsh918 made their first contribution in https://github.com/bentoml/BentoML/pull/2951
* ssun-g made their first contribution in https://github.com/bentoml/BentoML/pull/2959
* benjamintanweihao made their first contribution in https://github.com/bentoml/BentoML/pull/2965
* creativedutchmen made their first contribution in https://github.com/bentoml/BentoML/pull/2974
* darioarias made their first contribution in https://github.com/bentoml/BentoML/pull/3027
* jeffthebear made their first contribution in https://github.com/bentoml/BentoML/pull/3032

**Full Changelog**: https://github.com/bentoml/BentoML/compare/v1.0.5...v1.0.6

🍱 BentoML `v1.0.5` is released as a quick fix to a Yatai incompatibility introduced in `v1.0.4`.
- The incompatibility manifests in the following error message when deploying a bento on Yatai. Upgrading BentoML to `v1.0.5` will resolve the issue.
bash
Error while finding module specification for 'bentoml._internal.server.cli.api_server' (ModuleNotFoundError: No module named 'bentoml._internal.server.cli')

- The incompatibility resides in all Yatai versions prior to `v1.0.0-alpha.*`. Alternatively, upgrading Yatai to `v1.0.0-alpha.*` will also restore the compatibility with bentos built in `v1.0.4`.

- Added support for explicit GPU mapping for runners. In addition to specifying the number of GPU devices allocated to a runner, we can map a list of device IDs directly to a runner through [configuration](https://docs.bentoml.org/en/latest/concepts/runner.html#runner-configuration).

yaml
runners:
iris_clf_1:
resources:
nvidia.com/gpu: [2, 4] Map device 2 and 4 to iris_clf_1 runner
iris_clf_2:
resources:
nvidia.com/gpu: [1, 3] Map device 1 and 3 to iris_clf_2 runner


- Added SSL support for API server through both CLI and configuration.

yaml
--ssl-certfile TEXT SSL certificate file
--ssl-keyfile TEXT SSL key file
--ssl-keyfile-password TEXT SSL keyfile password
--ssl-version INTEGER SSL version to use (see stdlib 'ssl' module)
--ssl-cert-reqs INTEGER Whether client certificate is required (see stdlib 'ssl' module)
--ssl-ca-certs TEXT CA certificates file
--ssl-ciphers TEXT Ciphers to use (see stdlib 'ssl' module)


- Added adaptive batching size histogram metrics, `BENTOML_{runner}_{method}_adaptive_batch_size_bucket`, for observability of batching mechanism details.


- Added support OpenTelemetry [OTLP exporter for tracing](https://docs.bentoml.org/en/latest/guides/tracing.html) and configures the OpenTelemetry resource automatically if user has not explicitly configured it through environment variables. Upgraded OpenTelemetry python packages to version `0.33b0`.


- Added support for saving `external_modules` alongside with models in the `save_model` API. Saving external Python modules is useful for models with external dependencies, such as tokenizers, preprocessors, and configurations.
- Enhanced Swagger UI to include additional documentation and helper links.


💡 We continue to update the documentation on every release to help our users unlock the full power of BentoML.

- Checkout the [adaptive batching](https://docs.bentoml.org/en/latest/guides/batching.html) documentation on how to leverage batching to improve inference latency and efficiency.
- Checkout the [runner configuration](https://docs.bentoml.org/en/latest/concepts/runner.html#runner-configuration) documentation on how to customize resource allocation for runners at run time.

🙌 We continue to receive great engagement and support from the BentoML community.

- Shout out to sptowey for their contribution on adding SSL support.
- Shout out to dbuades for their contribution on adding the OTLP exporter.
- Shout out to tweeklab for their contribution on fixing a bug on `import_model` in the MLflow framework.

What's Changed
* refactor: cli to `bentoml_cli` by sauyon in https://github.com/bentoml/BentoML/pull/2880
* chore: remove typing-extensions dependency by sauyon in https://github.com/bentoml/BentoML/pull/2879
* fix: remove chmod install scripts by aarnphm in https://github.com/bentoml/BentoML/pull/2830
* fix: relative imports to lazy by aarnphm in https://github.com/bentoml/BentoML/pull/2882
* fix(cli): click utilities imports by aarnphm in https://github.com/bentoml/BentoML/pull/2883
* docs: add custom model runner example by parano in https://github.com/bentoml/BentoML/pull/2885
* qa: analytics unit tests by aarnphm in https://github.com/bentoml/BentoML/pull/2878
* chore: script for releasing quickstart bento by parano in https://github.com/bentoml/BentoML/pull/2892
* fix: pushing models from Bento instead of local modelstore by parano in https://github.com/bentoml/BentoML/pull/2887
* fix(containerize): supports passing multiple tags by aarnphm in https://github.com/bentoml/BentoML/pull/2872
* feat: explicit GPU runner mappings by jjmachan in https://github.com/bentoml/BentoML/pull/2862
* fix: setuptools doesn't include `bentoml_cli` by bojiang in https://github.com/bentoml/BentoML/pull/2898
* feat: Add SSL support for http api servers via bentoml serve by sptowey in https://github.com/bentoml/BentoML/pull/2886
* patch: ssl styling and default value check by aarnphm in https://github.com/bentoml/BentoML/pull/2899
* fix(scheduling): raise an error for invalid resources by bojiang in https://github.com/bentoml/BentoML/pull/2894
* chore(templates): cleanup debian dependency logic by aarnphm in https://github.com/bentoml/BentoML/pull/2904
* fix(ci): unittest failed by bojiang in https://github.com/bentoml/BentoML/pull/2908
* chore(cli): add figlet for CLI by aarnphm in https://github.com/bentoml/BentoML/pull/2909
* feat: codespace by aarnphm in https://github.com/bentoml/BentoML/pull/2907
* feat: use yatai proxy to upload/download bentos/models by yetone in https://github.com/bentoml/BentoML/pull/2832
* fix(scheduling): numpy worker environs are not taking effect by bojiang in https://github.com/bentoml/BentoML/pull/2893
* feat: Adaptive batching size histogram metrics by ssheng in https://github.com/bentoml/BentoML/pull/2902
* chore(swagger): include help links by parano in https://github.com/bentoml/BentoML/pull/2927
* feat(tracing): add support for otlp exporter by dbuades in https://github.com/bentoml/BentoML/pull/2918
* chore: Lock OpenTelemetry versions and add tracing metadata by ssheng in https://github.com/bentoml/BentoML/pull/2928
* revert: unminify CSS by aarnphm in https://github.com/bentoml/BentoML/pull/2931
* fix: importing mlflow:/ urls with no extra path info by tweeklab in https://github.com/bentoml/BentoML/pull/2930
* fix(yatai): make presigned_urls_deprecated optional by bojiang in https://github.com/bentoml/BentoML/pull/2933
* feat: add timeout option for bentoml runner config by jjmachan in https://github.com/bentoml/BentoML/pull/2890
* perf(cli): speed up by aarnphm in https://github.com/bentoml/BentoML/pull/2934
* chore: remove multipart IO descriptor warning by ssheng in https://github.com/bentoml/BentoML/pull/2936
* fix(json): revert eager check by aarnphm in https://github.com/bentoml/BentoML/pull/2926
* chore: remove `--config` flag to load the bentoml runtime config by jjmachan in https://github.com/bentoml/BentoML/pull/2939
* chore: update README messaging by ssheng in https://github.com/bentoml/BentoML/pull/2937
* fix: use a temporary file for file uploads by sauyon in https://github.com/bentoml/BentoML/pull/2929
* feat(cli): add CLI command to serve a runner by bojiang in https://github.com/bentoml/BentoML/pull/2920
* docs: Runner configuration for batching and resource allocation by ssheng in https://github.com/bentoml/BentoML/pull/2941
* bug: handle bad image file by parano in https://github.com/bentoml/BentoML/pull/2942
* chore(docs): earlier check for buildx by aarnphm in https://github.com/bentoml/BentoML/pull/2940
* fix(cli): helper message default values by ssheng in https://github.com/bentoml/BentoML/pull/2943
* feat(sdk): add external_modules option to save_model by bojiang in https://github.com/bentoml/BentoML/pull/2895
* fix(cli): component name regression by ssheng in https://github.com/bentoml/BentoML/pull/2944

New Contributors
* sptowey made their first contribution in https://github.com/bentoml/BentoML/pull/2886
* dbuades made their first contribution in https://github.com/bentoml/BentoML/pull/2918
* tweeklab made their first contribution in https://github.com/bentoml/BentoML/pull/2930

**Full Changelog**: https://github.com/bentoml/BentoML/compare/v1.0.3...v1.0.4

🍱 BentoML v1.0.3 release has brought a list of performance and feature improvement.

- Improved Runner IO performance by enhancing the underlying serialization and deserialization, especially in models with large input and output sizes. Our image input benchmark showed a 100% throughput improvement.
- v1.0.2 🐌

- v1.0.3 💨


- Added support for specifying URLs to exclude from tracing.
- Added support custom components for OpenAPI generation.

🙌 We continue to receive great engagement and support from the BentoML community.

- Shout out to [Ben Kessler](https://github.com/udevnl) for helping benchmarking performance.
- Shout out to [Jiew Peng Lim](https://github.com/jiewpeng) for adding the support for configuring URLs to exclude from tracing.
- Shout out to [Susana Bouchardet](https://github.com/sbouchardet) for add the support for JSON IO Descriptor to return empty response body.
- Thanks to [Keming](https://github.com/kemingy) and [mplk](https://github.com/mplk) for contributing their first PRs in BentoML.

What's Changed
* chore(deps): bump actions/setup-node from 2 to 3 by dependabot in https://github.com/bentoml/BentoML/pull/2846
* fix: extend --cache-from consumption to python tuple by anwang2009 in https://github.com/bentoml/BentoML/pull/2847
* feat: add support for excluding urls from tracing by jiewpeng in https://github.com/bentoml/BentoML/pull/2843
* docs: update notice about buildkit by aarnphm in https://github.com/bentoml/BentoML/pull/2837
* chore: add CODEOWNERS by aarnphm in https://github.com/bentoml/BentoML/pull/2842
* doc(frameworks): tensorflow by bojiang in https://github.com/bentoml/BentoML/pull/2718
* feat: add support for specifying urls to exclude from tracing as a list by jiewpeng in https://github.com/bentoml/BentoML/pull/2851
* fix(configuration): merging global runner config to runner specific config by jjmachan in https://github.com/bentoml/BentoML/pull/2849
* fix: Setting status code and cookies by ssheng in https://github.com/bentoml/BentoML/pull/2854
* chore: README typo by kemingy in https://github.com/bentoml/BentoML/pull/2859
* chore: gallery links to `bentoml/examples` by aarnphm in https://github.com/bentoml/BentoML/pull/2858
* fix(runner): use pickle instead for multi payload parameters by aarnphm in https://github.com/bentoml/BentoML/pull/2857
* doc(framework): pytorch guide by bojiang in https://github.com/bentoml/BentoML/pull/2735
* docs: add missing `output` to Runner docs by mplk in https://github.com/bentoml/BentoML/pull/2868
* chore: fix push and load interop by aarnphm in https://github.com/bentoml/BentoML/pull/2863
* fix: Usage stats by ssheng in https://github.com/bentoml/BentoML/pull/2876
* fix: `JSON(IODescriptor[JSONType]).to_http_response` returns empty body when the response is `None`. by sbouchardet in https://github.com/bentoml/BentoML/pull/2874
* chore: Address comments in the 2874 by ssheng in https://github.com/bentoml/BentoML/pull/2877
* fix: debugger breaks on circus process by aarnphm in https://github.com/bentoml/BentoML/pull/2875
* feat: support custom components for OpenAPI generation by aarnphm in https://github.com/bentoml/BentoML/pull/2845

New Contributors
* anwang2009 made their first contribution in https://github.com/bentoml/BentoML/pull/2847
* jiewpeng made their first contribution in https://github.com/bentoml/BentoML/pull/2843
* kemingy made their first contribution in https://github.com/bentoml/BentoML/pull/2859
* mplk made their first contribution in https://github.com/bentoml/BentoML/pull/2868
* sbouchardet made their first contribution in https://github.com/bentoml/BentoML/pull/2874

**Full Changelog**: https://github.com/bentoml/BentoML/compare/v1.0.2...v1.0.3

🍱 We have just released BentoML v1.0.2 with a number of features and bug fixes requested by the community.

- Added support for custom model versions, e.g. `bentoml.tensorflow.save_model("model_name:1.2.4", model)`.
- Fixed PyTorch Runner payload serialization issue due to tensor not on CPU.

text
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first


- Fixed Transformers GPU device assignment due to kwargs handling.
- Fixed excessive Runner thread spawning issue under high load.
- Fixed PyTorch Runner inference error due to saving tensor during inference mode.

text
RuntimeError: Inference tensors cannot be saved for backward. To work around you can make a clone to get a normal tensor and use it in autograd.


- Fixed Keras Runner error when the input has only a single element.
- Deprecated the `validate_json` option in JSON IO descriptor and recommended specifying validation logic natively in the [Pydantic model](https://github.com/bentoml/BentoML/blob/main/examples/pydantic_validation/service.py#L30).

🎨 We added an [examples](https://github.com/bentoml/BentoML/tree/main/examples) directory and in it you will find interesting sample projects demonstrating various applications of BentoML. We welcome your contribution if you have a project idea and would like to share with the community.

💡 We continue to update the documentation on every release to help our users unlock the full power of BentoML.

- Did you know BentoML service supports [mounting and calling runners from custom FastAPI and Flask apps](https://docs.bentoml.org/en/latest/guides/server.html)?
- Did you know IO descriptor supports [input and output validation](https://docs.bentoml.org/en/latest/concepts/service.html#schema-and-validation) of schema, shape, and data types?

What's Changed
* chore: remove all `--pre` from documentation by aarnphm in https://github.com/bentoml/BentoML/pull/2738
* chore(framework): onnx guide minor improvements by larme in https://github.com/bentoml/BentoML/pull/2744
* fix(framework): fix how pytorch DataContainer convert GPU tensor by larme in https://github.com/bentoml/BentoML/pull/2739
* doc: add missing variable by robsonpeixoto in https://github.com/bentoml/BentoML/pull/2752
* chore(deps): `cattrs>=22.1.0` in setup.cfg by sugatoray in https://github.com/bentoml/BentoML/pull/2758
* fix(transformers): kwargs and migrate to framework tests by ssheng in https://github.com/bentoml/BentoML/pull/2761
* chore: add type hint for run and async_run by aarnphm in https://github.com/bentoml/BentoML/pull/2760
* docs: fix typo in SECURITY.md by parano in https://github.com/bentoml/BentoML/pull/2766
* chore: use pypa/build as PEP517 backend by aarnphm in https://github.com/bentoml/BentoML/pull/2680
* chore(e2e): capture log output by aarnphm in https://github.com/bentoml/BentoML/pull/2767
* chore: more robust prometheus directory ensuring by bojiang in https://github.com/bentoml/BentoML/pull/2526
* doc(framework): add scikit-learn section to ONNX documentation by larme in https://github.com/bentoml/BentoML/pull/2764
* chore: clean up dependencies by sauyon in https://github.com/bentoml/BentoML/pull/2769
* docs: misc docs reorganize and cleanups by parano in https://github.com/bentoml/BentoML/pull/2768
* fix(io descriptors): finish removing init_http_response by sauyon in https://github.com/bentoml/BentoML/pull/2774
* chore: fix typo by aarnphm in https://github.com/bentoml/BentoML/pull/2776
* feat(model): allow custom model versions by sauyon in https://github.com/bentoml/BentoML/pull/2775
* chore: add watchfiles as bentoml dependency by aarnphm in https://github.com/bentoml/BentoML/pull/2777
* doc(framework): keras guide by larme in https://github.com/bentoml/BentoML/pull/2741
* docs: Update service schema and validation by ssheng in https://github.com/bentoml/BentoML/pull/2778
* doc(frameworks): fix pip package syntax by larme in https://github.com/bentoml/BentoML/pull/2782
* fix(runner): thread limiter doesn't take effect by bojiang in https://github.com/bentoml/BentoML/pull/2781
* feat: add additional env var configuring num of threads in Runner by parano in https://github.com/bentoml/BentoML/pull/2786
* fix(templates): sharing variables at template level by aarnphm in https://github.com/bentoml/BentoML/pull/2796
* bug: fix JSON io_descriptor validate_json option by parano in https://github.com/bentoml/BentoML/pull/2803
* chore: improve error message when failed importing user service code by parano in https://github.com/bentoml/BentoML/pull/2806
* chore: automatic cache action version update and remove stale bot by aarnphm in https://github.com/bentoml/BentoML/pull/2798
* chore(deps): bump actions/checkout from 2 to 3 by dependabot in https://github.com/bentoml/BentoML/pull/2810
* chore(deps): bump codecov/codecov-action from 2 to 3 by dependabot in https://github.com/bentoml/BentoML/pull/2811
* chore(deps): bump github/codeql-action from 1 to 2 by dependabot in https://github.com/bentoml/BentoML/pull/2813
* chore(deps): bump actions/cache from 2 to 3 by dependabot in https://github.com/bentoml/BentoML/pull/2812
* chore(deps): bump actions/setup-python from 2 to 4 by dependabot in https://github.com/bentoml/BentoML/pull/2814
* fix(datacontainer): pytorch to_payload should disable gradient by aarnphm in https://github.com/bentoml/BentoML/pull/2821
* fix(framework): fix keras single input edge case by larme in https://github.com/bentoml/BentoML/pull/2822
* fix(framework): keras GPU handling by larme in https://github.com/bentoml/BentoML/pull/2824
* docs: update custom bentoserver guide by parano in https://github.com/bentoml/BentoML/pull/2809
* fix(runner): bind limiter to runner_ref instead by bojiang in https://github.com/bentoml/BentoML/pull/2826
* fix(pytorch): inference_mode context is thead local by bojiang in https://github.com/bentoml/BentoML/pull/2828
* fix: address multiple tags for containerize by aarnphm in https://github.com/bentoml/BentoML/pull/2797
* chore: Add gallery projects under examples by ssheng in https://github.com/bentoml/BentoML/pull/2833
* chore: running formatter on examples folder by aarnphm in https://github.com/bentoml/BentoML/pull/2834
* docs: update security auth middleware by g0nz4rth in https://github.com/bentoml/BentoML/pull/2835
* fix(io_descriptor): DataFrame columns check by alizia in https://github.com/bentoml/BentoML/pull/2836
* fix: examples directory structure by ssheng in https://github.com/bentoml/BentoML/pull/2839
* revert: "fix: address multiple tags for containerize (2797)" by ssheng in https://github.com/bentoml/BentoML/pull/2840

New Contributors
* robsonpeixoto made their first contribution in https://github.com/bentoml/BentoML/pull/2752
* sugatoray made their first contribution in https://github.com/bentoml/BentoML/pull/2758
* g0nz4rth made their first contribution in https://github.com/bentoml/BentoML/pull/2835
* alizia made their first contribution in https://github.com/bentoml/BentoML/pull/2836

**Full Changelog**: https://github.com/bentoml/BentoML/compare/v1.0.0...v1.0.1

🍱 The wait is over. [BentoML](https://github.com/bentoml/BentoML) has officially released v1.0.0. We are excited to share with you the notable features improvements.

- Introduced BentoML Runner, an abstraction for parallel model inference. It allows the compute intensive model inference step to scale separately from the transformation and business logic. The Runner is easily instantiated and invoked, but behind the scenes, BentoML is optimizing for micro-batching and fanning out inference if needed. Here’s a simple example of instantiating a Runner. Learn more about [using runners](https://docs.bentoml.org/en/latest/concepts/runner.html).
- Redesigned how models are saved, moved, and loaded with BentoML. We introduced new primitives which allow users to call a save_model() method which saves the model in the most optimal way based the recommended practices of the ML framework. The model is then stored in a flexible local repository where users can use “import” and “export” functionality to push and pull “finalized” models from remote locations like S3. Bentos can be built locally or remotely with these models. Once built, [Yatai](https://github.com/bentoml/Yatai) or [bentoctl](https://github.com/bentoml/bentoctl) can easily deploy to the cloud service of your choice. Learn more about [preparing models](https://docs.bentoml.org/en/latest/concepts/model.html) and [building bentos](https://docs.bentoml.org/en/latest/concepts/bento.html).
- Enhanced micro-batching capability with the new runner abstraction, batching is even more powerful. When incoming data is spread to different transformation processes, the runner will fan in inferences when inference is invoked. Multiple inputs will be batched into a single inference call. Most ML frameworks implement some form of vectorization which improves performance for multiple inputs at once. Our adaptive batching not only batches inputs as they are received, but also regresses the time of the last several groups of inputs in order to optimize the batch size and latency windows.
- Improved reproducibility of the model by recording and locking the dependent library versions. We use the versions to package the correct dependencies so that the environment in which the model runs in production is identical to the environment it was trained in. All direct and transitive dependencies are recorded and deployed with the model when running in production. In our 1.0 version we now support Conda as well as several different ways to customize your pip packages when “building your Bento”. Learn more about [building bentos](https://docs.bentoml.org/en/latest/concepts/bento.html).
- Simplified Docker image creation during containerization to generate the right image for you depending on the features that you’ve decided to implement in your service. For example, if your runner specifies that it can run on a GPU, we will automatically choose the right Nvidia docker image as a base when containerizing your service. If needed, we also provide the flexibility to customize your docker image as well. Learn more about [containerization](https://docs.bentoml.org/en/latest/guides/containerization.html).
- Improved input and output validation with native type validation rules. Numpy and Pandas DataFrame can specify a static shape or even dynamically infer schema by providing sample data. The Pydantic schema that is produced per endpoint also integrates with our Swagger UI so that each endpoint is better documented for sharing. Learn more about [service APIs and IO Descriptors](https://docs.bentoml.org/en/latest/concepts/service.html#service-apis).

⚠️ BentoML v1.0.0 is backward incompatible with v0.13.1. If you wish to stay on the v0.13.1 LTS version, please lock the dependency with `bentoml==0.13.1`. We have also prepared a [migration guide](https://docs.bentoml.org/en/latest/guides/migration.html) from v0.13.1 to v1.0.0 to help with your project migration. We are committed to supporting the v0.13-LTS versions with critical bug fixes and security patches.

🎉 After years of seeing hundreds of model serving use cases, we are proud to present the official release of BentoML 1.0. We could not have done it without the growth and support of our community.