Initial public release of ExecuTorch. See https://pytorch.org/executorch for documentation.
**Important: This is a preview release**
This is a preview version of ExecuTorch and should be used for testing and evaluation purposes only. It is not yet recommended for use in production settings. We welcome any feedback, suggestions, and bug reports from the community to help us improve the technology. Please use the [PyTorch Forums](https://discuss.pytorch.org/) for discussion and feedback about ExecuTorch using the tag **#executorch**, and our [GitHub repository](https://github.com/pytorch/executorch/issues) for bug reporting.
stable-2023-09-19
**New models enabled (e2e tested via portable lib):**
* Emformer RNN-T Transcriber, Predictor, Joiner (as three modules)
**Quantization:**
* Enabled quantization for incpetion_v4 and deeplab_v3 in examples with XNNPACKQuantizer
**API changes:**
* **Runtime API**
* Many runtime APIs changed to improve ergonomics and to better match the style guide. Most of these changes are non-breaking (unless indicated as breaking), since the old APIs are available but marked as deprecated. We recommend that users migrate off of the deprecated APIs before the next release.
* For an example of how these API changes affected common use cases, see the edits made to ``examples/executor_runner/executor_runner.cpp`` under the "Files changed" tab of [https://github.com/pytorch/executorch/compare/stable-2023-09-12...78f884f2b9702a819907824c7ba76d76216dd901](https://github.com/pytorch/executorch/compare/stable-2023-09-12...78f884f2b9702a819907824c7ba76d76216dd901)
* Breaking behavioral change: ``MethodMeta``
* ``MethodMeta::num_non_const_buffers`` and ``MethodMeta::non_const_buffer_size`` no longer require adjusting by 1 to skip over the reserved zero index. This will require that users of ``MethodMeta`` remove adjustments while counting and iterating over non-const buffers.
* Details about the change, including migration to adapt to the new behavior: [https://github.com/pytorch/executorch/commit/5762802b47bca6ee9e85e50f21615554b815828c](https://github.com/pytorch/executorch/commit/5762802b47bca6ee9e85e50f21615554b815828c)
* Also note that these methods have been renamed to ``num_memory_planned_buffers`` and ``memory_planned_buffer_size`` (see note below)
* Note that the deprecated ``Program::num_non_const_buffers`` and ``Program::get_non_const_buffer_size`` methods did not change behavior re: skipping index zero. But they are deprecated, and will be removed in a future release, so we recommend that users migrate to the ``MethodMeta`` API and behavior.
* ``MethodMeta`` method names changed from `non_const to memory_planned`
* ``MethodMeta::num_non_const_buffers()`` is now ``MethodMeta::num_memory_planned_buffers()``
* ``MethodMeta::non_const_buffer_size(N)`` is now ``MethodMeta::memory_planned_buffer_size(N)``
* Changed in [https://github.com/pytorch/executorch/commit/6944c4556bcf22d4b51de9b3cc2876fe1b6ee91e](https://github.com/pytorch/executorch/commit/6944c4556bcf22d4b51de9b3cc2876fe1b6ee91e)
* The old names are available but deprecated, and will be removed in a future release
* Breaking code-compatibility change: ``Method``'s constructor and ``init()`` method are now private
* Users should not have used these methods; `Method` instances should only be created by ``Program::load_method()``
* Changed in [https://github.com/pytorch/executorch/commit/4f3e5e65d1881326b9729b8cbbadf0cb7739c649](https://github.com/pytorch/executorch/commit/4f3e5e65d1881326b9729b8cbbadf0cb7739c649)
* ``MemoryManager`` constructor no longer requires ``const_allocator`` or ``kernel_temporary_allocator``
* A new constructor lets users avoid creating zero-sized allocators that they don't use
* It also renames the parameters for the remaining allocators to make their uses more clear
* Changed in [https://github.com/pytorch/executorch/commit/6944c4556bcf22d4b51de9b3cc2876fe1b6ee91e](https://github.com/pytorch/executorch/commit/6944c4556bcf22d4b51de9b3cc2876fe1b6ee91e)
* Example migration to the new constructor: [https://github.com/pytorch/executorch/commit/fedc04c490ed41049a8303b7cf536cb3e41334b1](https://github.com/pytorch/executorch/commit/fedc04c490ed41049a8303b7cf536cb3e41334b1)
* The old constructor is available but deprecated, and will be removed in a future release
* Breaking code-compatibility change: ``MemoryManager`` is now final and cannot be subclassed
* Changed in [https://github.com/pytorch/executorch/commit/6944c4556bcf22d4b51de9b3cc2876fe1b6ee91e](https://github.com/pytorch/executorch/commit/6944c4556bcf22d4b51de9b3cc2876fe1b6ee91e)
* ``HierarchicalAllocator``'s constructor now takes an array of ``Span<uint8_t>`` instead of an array of ``MemoryAllocator``
* Changed in [https://github.com/pytorch/executorch/commit/58c8c924fc90a4128947ed23a5b46952624faf4f](https://github.com/pytorch/executorch/commit/58c8c924fc90a4128947ed23a5b46952624faf4f)
* Example migration to the new API: [https://github.com/pytorch/executorch/commit/0bce2cb0a6aec9b01f3b7d99373711f95a4d598c](https://github.com/pytorch/executorch/commit/0bce2cb0a6aec9b01f3b7d99373711f95a4d598c)
* The old constructor is still available but deprecated, and will be removed in a future release
* Breaking code-compatibility change: ``HierarchicalAllocator`` is now ``final`` and cannot be subclassed
* Changed in [https://github.com/pytorch/executorch/commit/58c8c924fc90a4128947ed23a5b46952624faf4f](https://github.com/pytorch/executorch/commit/58c8c924fc90a4128947ed23a5b46952624faf4f)
* ``Program::Load()`` renamed to ``Program::load()``
* Changed in [https://github.com/pytorch/executorch/commit/8a5f3e89c97fb59b60cccef3345a90e90ed2a474](https://github.com/pytorch/executorch/commit/8a5f3e89c97fb59b60cccef3345a90e90ed2a474)
* The old name is still available but deprecated, and will be removed in a future release
* ``FileDataLoader::From()`` renamed to ``FileDataLoader::from()``
* Changed in [https://github.com/pytorch/executorch/commit/e2dd0bec0e80ad6e1c97fad93012c60e30b7766d](https://github.com/pytorch/executorch/commit/e2dd0bec0e80ad6e1c97fad93012c60e30b7766d)
* The old name is still available but deprecated, and will be removed in a future release
* ``MmapDataLoader::From()`` renamed to ``MmapDataLoader::from()``
* Changed in [https://github.com/pytorch/executorch/commit/395e51acd56491255891677982686a3d14bdb8e4](https://github.com/pytorch/executorch/commit/395e51acd56491255891677982686a3d14bdb8e4)
* The old name is still available but deprecated, and will be removed in a future release
* **Delegate API**
* File rename: ``runtime/backend/backend_registry.cpp`` -> ``runtime/backend/interface.cpp``
* Partition API update: Partitioner.partition function takes `ExportedProgram` instead of `torch.nn.GraphModule`. With this change we access the parameters and buffer in partition function.
* How to rebase: access graphmodule by `exported_program.graph_module`
* **SDK**
* BundledProgram updates APIs to enable user bundling test cases on specific method by using method name instead of method id in the past
* AOT: class ``BundledConfig (method_names: List[str], inputs: List[List[Any]], expected_outputs: List[List[Any]])``. `method_names` is the new added attribute.
* Runtime: Replace the original method_idx with method_name
* API for load bundled test input to ET program:
``
__ET_NODISCARD Error LoadBundledInput(
Method& method,
serialized_bundled_program* bundled_program_ptr,
MemoryAllocator* memory_allocator,
const char* method_name,
size_t testset_idx);
``
* API for verify result with bundled expected output:
``
__ET_NODISCARD Error VerifyResultWithBundledExpectedOutput(
Method& method,
serialized_bundled_program* bundled_program_ptr,
MemoryAllocator* memory_allocator,
const char* method_name,
size_t testset_idx,
double rtol = 1e-5,
double atol = 1e-8);
``
* Details and examples can be found https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/bundled_program.md
**Bug Fixes:**
* When exporting with enable_aot=True, all constant tensors will be lifted as inputs to the graph (in addition to the parameters and buffers).
* Kwargs are now consistently placed in the call_spec of the exported program.
stable-2023-09-12
**New models enabled (e2e tested via portable lib):**
* MobileBert
**Export API**
* Two stage export API
* We are in the process of moving away from `exir.capture()`: Please refer to this issue [https://github.com/pytorch/executorch/issues/290](https://github.com/pytorch/executorch/issues/290) for more details. Also look at the updated doc at [https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/exporting_to_executorch.md](https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/exporting_to_executorch.md)
* `exir.serialize`
* The `exir.serialize` module was renamed to `exir._serialize` and is now private
* `transform()`
* For perform passes on the same dialect, use [transform()](https://github.com/pytorch/executorch/blob/stable/exir/program/_program.py#L364-L369)
**Runtime API**
* Method
* Added `set_output_data_ptr()`, which is a simpler and safer way to set the output buffers if they were not memory-planned
* `Program::load_method()` now accepts an optional `EventTracer` parameter for non-global profiling and event data collection
**Delegation API**
* `backend.init()` and `backend.execute()` API changes.
* `BackendInitContext` is a new added argument for `backend.init` and `BackendExecutionContext` is the new added argument for `backend.execute()`.
* How to rebase on these apis changes?
* For backend.init, if `runtime_allocator` is not used, just mark context is not used with `__ET_UNUSED`. Otherwise, `runtime_allocator` can be accessed from the context.
* For backend.execute, nothing has been added to `context` yet, just mark it with `__ET_UNUSED` directly. We’ll add event tracer for profiling via `context` soon.
* `backend.preprocess()` API changes
* Updated backend.preprocess:
* `def preprocess( edge_program: ExportedProgram, compile_specs: List[CompileSpec], ) -> PreprocessResult`
* How to rebase on this API changes?
* Wrap the result like `PreprocessResult(processed_bytes=bytes)`
* Partitioner.partition API changes
* Updated Partition class definition. Move partition_tags from class attribute to be part of the `ParititionResult`.
* `def partition(self, graph_module: GraphModule) -> PartitionResult`
* How to rebase on this API change?
* Wrap both `partition_tags` and the `tagged_graph` together as `PartitionResult`
* Example Quantizer and Delegate e2e demo
* Added an [example](https://github.com/pytorch/executorch/blob/stable/backends/example/README.md) to show to add a quantizer and have it working with delegate to fully delegated a quantized MobileNetV2 model to the example backend.
**XnnpackDelegate**
* In an effort to align better with the rest of the Executorch AoT stack, XnnpackDelegate added preliminary support to also handle graphs exported with the canonical capture config (i.e. CaptureConfig.enable_aot=True and CaptureConfig._unlift=False)
**SDK**
* DelegateMappingBuilder to generate debug handle mapping AOT for delegates
* [https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/profiling_and_debugging_delegates.md](https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/profiling_and_debugging_delegates.md)
* BundledProgram enabled for usage with examples (more API changes to come in subsequent releases to improve usability, these will be breaking API changes)** **
* Documentation:
* [https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/bundled_program.md](https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/bundled_program.md)
* Example code pointers:
* [https://github.com/pytorch/executorch/tree/stable/examples/bundled_executor_runner](https://github.com/pytorch/executorch/tree/stable/examples/bundled_executor_runner)
* [https://github.com/pytorch/executorch/blob/stable/examples/export/export_bundled_program.py](https://github.com/pytorch/executorch/blob/stable/examples/export/export_bundled_program.py)
**Misc**
* [Linter](https://github.com/pytorch/executorch/blob/stable/CONTRIBUTING.md) enabled
* `pytest` enabled. Rerun `pip install .` to install `pytest` and other deps
* gtest enabled via buck, for example, run gtest for `runtime/core`
* `/tmp/buck2 test runtime/core/test/…`
* Index operator [rewrite](https://github.com/pytorch/executorch/pull/278):
* Fixed bug related to null indices.
* Implemented full [Numpy’s advanced indexing](https://numpy.org/doc/stable/user/basics.indexing.html#advanced-indexing) functionality (now it is possible to use multidimensional indices, and masks that only index a subspace).
* Build/CMake
* CMake release build mode with size optimization flags. We have an example in `examples/selective_build/test_selective_build.sh`
stable-2023-08-29
New models enabled (e2e tested via portable lib):
- Wav2Letter
- Inception V3 and Inception V4
- Resnet18 and Resnet50
Quantization:
- Enabled E2E MobileNet V2:
- Model can be quantized and run with portable + quantized op (for quantize/dequantize ops) lib.
Follow, https://github.com/pytorch/executorch/blob/main/examples/README.md#quantization, to run a quantized model via portable lib.
- MobileNet V3:
- Needs bumping up the pytorch nightly version (dev20230828) in order to enable MobileNet V3 quantization. However, this breaks ViT export, hence this cut will skip MobileNet V3 quantization until we resolve ViT export breakage.
Delegation:
- API update:
- [breaking changes] delegate AOT APIs are moved from `executorch/backends/` to `executorch/exir/backend`. To address the breakage: Update `from executorch.backends.backend_details` to `from executorch.exir.backend.backend_details`, and `from executorch.backends.backend_api` to `from executorch.exir.backend.backend_api`
- XNNPACK:
- XNNPACK delegated models can run on Mac/Linux in OSS
- XNNPACK lowering workflow examples have been added for MobileNet V2 (with quantization and delegation) and MobileNet V3 (with delegation)
- Showcase preliminary [XNNPACK perf stats](https://github.com/pytorch/executorch/tree/main/examples/backend#xnnpack-performance-gain) on Linux x86 & Mac M1
Selective build:
- Added buck2 examples to demonstrate 3 APIs to do selective build on any executorch runtime build
- Run [test_selective_build.sh](https://github.com/pytorch/executorch/blob/main/examples/selective_build/test_selective_build.sh)
-
stable-2023-08-15
- New models in example folder:
- Torchvision ViT. Run the example from `executorch` dir:
- `python3 -m examples.export.export_example --model_name="vit"`
- `buck2 run //examples/executor_runner:executor_runner -- --model_path vit.pte`
- Quantization workflow example added and validated to work with MV2:
- `python3 -m examples.quantization.example --model_name mv2`
- CMake build:
- [executor_runner](https://github.com/pytorch/executorch/blob/main/examples/executor_runner/executor_runner.cpp) can be built via cmake. See [cmake_build_system.md](https://github.com/pytorch/executorch/blob/main/docs/website/docs/tutorials/cmake_build_system.md).
- Custom ops:
- Add examples to register custom ops into EXIR and Executorch runtime.
- Note: buck2 in [test_custom_ops.sh](https://github.com/pytorch/executorch/blob/main/examples/custom_ops/test_custom_ops.sh) should point to installed buck2 if it is not accessible in system’s PATH
stable-2023-08-01
Initial release to early users.