Rainbowneko

First Stable Version
+ update LR schedule
+ all gradient accumulation steps as one global step. (N accumulation steps means batch_size*N, it is one step actually)
+ remake ckpt manager
* support model save/load
* support different source and format
+ onnx export/load support
+ remake model loader
+ update config of `MetricGroup` and `Handler`

+ workflow for model inference
+ skip compile `lambda` in python config file
+ update model config parser

+ Direct use module in `_base_` for python config

+ lora modules
+ accelerate config manager
+ documents update

+ remake dataflow (Dataset:[DataSource, DataHandler, Bucket])
+ Now can control the transfer of data between modules in the configuration file.
+ Add semi-supervised configuration file.
+ Add `CharbonnierLoss`, `MSSSIMLoss`
+ Supporting split cfg into multiple functions with `neko_cfg`.
+ bug fix

dataflow control
python
Dataset -> Model -> Loss

For example in semi-supervised task
Dataset
dataset_U=BaseDataset(
source=...,
handler=HandlerGroup(handlers=dict(
strong=HandlerChain(handlers=..., key_map_out=('image -> image_strong',)),
weak=HandlerChain(handlers=..., key_map_out=('image -> image_weak',)),
)),
...
)

Model
model=dict(
wrapper=DistillationWrapper(_partial_=True, _replace_=True,
key_map_in_student=dict(
dataset_S=('image -> 0',),
dataset_U=('image_strong -> 0',),
),
key_map_in_teacher=dict(
dataset_S={},
dataset_U=('image_weak -> 0',),
),
key_map_out=('pred_student -> pred_student', 'pred_teacher -> pred_teacher'),
...
)
),

Loss
loss=dict(_replace_=True,
dataset_S=LossContainer(CrossEntropyLoss(), key_map=('pred.pred_student -> 0', 'inputs.label -> 1')),
dataset_U=LossContainer(PseudoLabelLoss(), key_map=('pred.pred_student -> pred', 'pred.pred_teacher -> pred_label')),
),