Rknn-toolkit2

- RKNN-Toolkit2 support ARM64 architecture
- RKNN-Toolkit-Lite2 support installation via pip
- Add support for W4A16 symmetric quantization (RK3576)
- Operator optimization, such as LayerNorm, LSTM, Transpose, MatMul, etc.

- Support installation via pip
- Optimize transformer model performance
- Support Python 3.12
- Operator optimization, such as softmax, hardmax, MatMul, etc.

- Support RV1103B (Beta)
- Support RK2118 (Beta)
- Support Flash Attention (Only RK3562 and RK3576)
- Improve MatMul API
- Improve support for int32 and int64
- Support more operators and operator fusion

- Support RK3576 (Beta)
- Support RK2118 (Beta)
- Support SDPA (Scaled Dot Product Attention) to improve transformer performance
- Improve custom operators support
- Improve MatMul API
- Improve support for Reshape,Transpose,BatchLayernorm,Softmax,Deconv,Matmul,ScatterND etc.
- Support pytorch 2.1
- Improve support for QAT models of pytorch and onnx
- Optimize automatic generation of C++ code

- Support ONNX model of OPSET 12~19
- Support custom operators (including CPU and GPU)
- Improve support for dynamic weight convolution, Layernorm, RoiAlign, Softmax, ReduceL2, Gelu, GLU, etc.
- Added support for python3.7/3.9/3.11
- Add rknn_convert function
- Improve transformer support
- Improve MatMul API, such as increasing the K limit length, RK3588 adding int4 * int4 -> int16 support, etc.
- Reduce RV1106 rknn_init initialization time, memory consumption, etc.
- RV1106 adds int16 support for some operators
- Fixed the problem that the convolution operator of RV1106 platform may make random errors in some cases.
- Improve user manual
- Reconstruct the rknn model zoo and add support for multiple models such as detection, segmentation, OCR, and license plate recognition.

- Improve dynamic shape support
- Improve matmul api support
- Add GPU back-end implementations for some operators such as matmul
- Improve transformer support
- Reduce rknn_init memory usage
- Optimize rknn_init time-consuming