Coremltools

Release Notes

Bug Fixes

* Add support for converting Softplus layer in coremltools.
* Fix in gelu and layer norm fusion pass.
* Simplified build & CI setup.
* Fixed critical numpy

This release includes new op conversion supports, bug fixes, and improved graph optimization passes.

Install/upgrade to the latest `coremltools` with `pip install --upgrade coremltools`.

More details can be found in [neural-network-guide.md](https://github.com/apple/coremltools/blob/master/docs/NeuralNetworkGuide.md).

Changes:

- Add support for TensorFlow 2.x file format (.h5, SavedModel, and concrete functions).
- Add support for several new ops, such as `AddV2`, `FusedBatchNormV3`.
- Bug fixes in the Tensorflow converter's op fusion graph pass.

Known Issues:

- `tf.keras` model conversion supported only with TensorFlow 2
- Currently, there are issues while invoking the TensorFlow 2.x model conversion in Python 2.x.
- Currently, there are issues while converting `tf.keras` graphs that contain recurrent layers.

We are happy to announce the official release of coremltools 3 which aligns with Core ML 3. It includes a new version of the .mlmodel specification (version 4) which brings with it support for:

* Updatable models - Neural Network and KNN
* More dynamic and expressive neural networks - approx. 100 more layers added compared to Core ML 2
* Dynamic control flows
* Nearest neighbor classifiers
* Recommenders
* Linked models
* Sound analysis preprocessing
* Runtime adjustable parameters for on-device update

This version of coremltools also includes a new converter path for TensorFlow models. The [tfcoreml converter](https://github.com/tf-coreml/tf-coreml) has been updated to include this new path to convert to specification 4 which can handle control flow and cyclic tensor flow graphs.

Control flow example can be found [here](https://github.com/apple/coremltools/blob/master/examples/Neural_network_control_flow_power_iteration.ipynb).

Updatable Models

Core ML 3 supports an on-device update of models. Version 4 of the `.mlmodel` specification can encapsulate all the necessary parameters for a model update. Nearest neighbor, neural networks and pipeline models can all be made updatable.
Updatable neural networks support the training of convolution and fully connected layer weights (with back-propagation through many other layers types). Categorical cross-entropy and mean squared error losses are available along with stochastic gradient descent and Adam optimizers.
See examples of how to convert and create [updatable models](https://github.com/apple/coremltools/tree/master/examples/updatable_models).
See the [MLUpdateTask API reference](https://developer.apple.com/documentation/coreml/mlupdatetask) for how to update a model from within an app.

Neural Networks

* Support for new layers in Core ML 3 added to the `NeuralNetworkBuilder`
* Exact rank mapping of multi dimensional array inputs
* Control Flow related layers (branch, loop, range, etc.)
* Element-wise unary layers (ceil, floor, sin, cos, gelu, etc.)
* Element-wise binary layers with broadcasting (addBroadcastable, multiplyBroadcastable, etc)
* Tensor manipulation layers (gather, scatter, tile, reverse, etc.)
* Shape manipulation layers (squeeze, expandDims, getShape, etc.)
* Tensor creation layers (fillDynamic, randomNormal, etc.)
* Reduction layers (reduceMean, reduceMax, etc.)
* Masking / Selection Layers (whereNonZero, lowerTriangular, etc.)
* Normalization layers (layerNormalization)
* For a full list of supported layers in Core ML 3, check out Core ML [specification documentation](https://apple.github.io/coremltools/coremlspecification/sections/NeuralNetwork.html) or [NeuralNetwork.proto](https://github.com/apple/coremltools/blob/c6e7d15e3aef676a60247fea235da58aedbfcfd7/mlmodel/format/NeuralNetwork.proto#L535).
* Support conversion of recurrent networks from [TensorFlow](https://github.com/tf-coreml/tf-coreml/releases)

* Converting a Keras model that uses mean squared error for the loss function will not create a valid model. A workaround is to set respect_trainable to False (the default) when converting and then manually add the loss function.

Core ML 3 Developer Beta 1

* The default number of epochs encoded in model is not respected and may run for 0 epochs, immediately returning without training.
* Workaround: Explicitly supply epochs via MLModelConfiguration updateParameters using MLParameterKey.epochs even if you want to use the default value encoded in the model.
* Loss returned by the ADAM optimizer is not correct
* Some updatable pipeline models containing a static neural network sub-model can intermittently fail to update with the error: “Attempting to hash an MLFeatureValue that is not an image or multi array”. This error will surface in task.error as part of MLUpdateContext passed to the provided completion handler.
* Workaround: Retry model update by creating a new update task with the same training data.
* Some of the new neural network layers may result in an error when the model is run on a non-CPU compute device.
* Workaround: restrict computation to CPU with MLModelConfiguration computeUnits
* Enumerated shape flexibility, when used with Neural network inputs with 'exact_rank' mapping (i.e. rank 5 disabled), may result in an error during prediction.
* Workaround: use range shape flexibility