Trtutils

Added

- core.init_cuda
- Use to start CUDA if not using a TRTEngine and only the core setup.
- benchmark_engines and jetson.benchmark_engines
- Can benchmark TRTEngines in concurrently running mode
- Example yolo CLI program
- Currently only support video file input and display.
- General CLI fixes for trtexec
- Experimental non-pagelocked memory addressing for TRTengines
- Unstable, should be used with caution. Will be refined in the future
- Does not provide performance improvement, simply for testing speedup
of pagelocked memory utilization. As such, low-priority
- Basic internal profiling setup for YOLO objects.
- No current public access, but accessible through: (_pre)(_infer)(_post)_profile attributes
- Only stores last timestamp tuple
- No end2end method support yet

Fixed

- yolo.CUDAPreprocessor using the wrong block size during resize call
- Various fixes and extensions to ParallelTRTEngines

Added

- CUDA-based resize kernels
- Peform linear or letterbox resizing
- core.create_kernel_args nad core.Kernel

Added

- CUDA-based preprocessing for YOLO:
- Introduced `CUDAPreprocessor` and `CPUPreprocessor`.
- Additional parameters in YOLO constructor and methods:
- `conf_thres`
- `extra_nms`, `agnostic_nms`
- `resize_method`, `preprocessing_unit`.
- Runtime CUDA kernel generation with NVRTC:
- Final transform (transpose from HWC to BCHW) reduced from 50ms to 5ms for 1280x1280, achieving a 10x speedup.

Improved

- Multi-threading safety:
- `ParallelYOLO` enforces serial deserialization of engine files.
- `CUDAProcessor` now serializes initialization.
- Core CUDA/NVRTC calls use mutexes.

---

Added
- 'impls.yolo.YOLO':
- Added `input_range` parameter for specifying the input range.
- `YOLOX` uses `[0:255]`, all others use `[0:1]`.

---

Added
- Variations of `impls.yolo.YOLO`: YOLO7, YOLO8, YOLO9, YOLO10, and YOLOX.

Changed
- `impls.yolo.YOLO`:
- Version inference is now automatic.
- Postprocessing determined from outputs.

---

Improved
- Outputs from `impls.yolo.YOLO` now use standard Python types:
- Improved compatibility with JIT compilers like `numba`.

---