Fastestimator

Release Note

Backend:
* Upgraded tensorFlow backend to 2.9.1
* upgraded PyTorch backend to 1.10.2
* Both backends now require CUDA > 11

NumpyOp:
* Re-designed `ReadMat` Op so that it can read arbiturary keys in .mat files and create different keys.

TensorOp:
* `Resize3D` can achieve data resizing while preserving the gradient for 3D data
* Added a new `Dice` Op that can be used in Loss and Trace. It supports a wide range of formulations. [[code snippet](https://github.com/fastestimator-util/fastestimator-misc/blob/22602865abd6a1c9ac099a0aeaa2eedc1dde66f1/examples/1.5/dice_loss/unet_torch.py#L103-L107)]
* Added a new L1 Loss Op that supports L1 loss and its variants (Smoothed L1, Huber Loss) [[code snippet](https://github.com/fastestimator-util/fastestimator-misc/blob/22602865abd6a1c9ac099a0aeaa2eedc1dde66f1/examples/1.5/l1_loss/retinanet.py#L567-L570)]
* Added support of focal loss for classification and segmentation problems [[code snippet](https://github.com/fastestimator-util/fastestimator-misc/blob/22602865abd6a1c9ac099a0aeaa2eedc1dde66f1/examples/1.5/retinanet/retinanet_tf.py#L576-L578)]

Search API:
* Redesigned Search API for general purpose hyper-parameter tuning or variety robustness testing

Pipeline:
* Reduced the key pruning warning frequency

Network:
* improved training speed on multi-gpu
* supported modifying layer's retrainable property through trace for fine-tuning purpose
* supported PyTorch training with partial gradient available
* Supported TensorFlow sub-classing model definition

Estimator:
* Added Evaluation log log during eval phase to show eval progress

Visualization
* Switched the visualization engine to plotly, now FE supports more advanced interactive visualization
* Added Search visualization API to work with the re-designed search


Tutorial
* Added new tutorial about debugging in FE to help people debug FE easier
* Updated the search tutorial to leverage new visualization API
* Added a new model robustness tutorial to showcase model robustness testing with Search API and Visualization API

Others:
* Significantly reduced the time needed to import FE through lazy import
* Added dependency warning check to warn inconsistent backend versions

This is a patch release that fixes outdated dependency: sklearn and pycocotool for FE 1.4.X

This release includes a hot-fix for a multiprocessing issue with mac book

History API
* Fixed a critical bug about database access

Backend
* Fixed a bug caused by assertion during relaxed shape static graph execution
* Hardware-agnostic pytorch model saving & loading

Dataset
* New dataset - skl digits

Network
* New Permute TensorOp
* New Normalize TensorOp

Architecture & Apphubs
* Momentum inconsistency between TensorFlow and PyTorch

Estimator
* New per-instance CSV metric reporting and its per-instance interface for custom Trace
* BleuScore Trace as NLP metric

Plotting
* Fixed several bugs in log plotting

XAI
* New Eigen Cam capability: allowing user to specify percentage of variation

Dataset
* ExtendDataset: extending dataset length for better performance

Pipeline
* BatchOp: more flexible batching behavior and support batched `NumpyOp`
* FilterOp: removing sample elements based on criteria

Network
* l2 regularization as TensorOp: allow for tf model to use equivalent weight decay as torch

Estimator
* new `train_steps_per_epoch` and `eval_steps_per_epoch` argument. Now user can truly set arbitrary epoch length with them.

Apphub
* Instance segmentation - SOLOv2 TensorFlow & PyTorch implementation

Tutorial
* Custom Data Loaders
* Finetuning

Others
This release also includes many bug fixes throughout the API.

This release is a patch of FE1.3.X releases about sklearn and pycocotool dependency.