Hakkero-dataloader

What's Changed
* fix(tokenization): add st_token_ignore let user to ignore some tokens by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/17


**Full Changelog**: https://github.com/ericxsun/hakkero-dataloader/compare/v1.2.18...v1.2.19

What's Changed
* style: add autoflake by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/14
* feat: enhance check by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/15
* fix(tokenize): assistant's content starts with '\n' leads to a wierd bug by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/16


**Full Changelog**: https://github.com/ericxsun/hakkero-dataloader/compare/v1.2.16...v1.2.18

What's Changed
* fix: default epoch=1 in config by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/13


**Full Changelog**: https://github.com/ericxsun/hakkero-dataloader/compare/v1.2.15...v1.2.16

What's Changed
* docs: update readme by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/11
* feat: shuf while build index by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/12


**Full Changelog**: https://github.com/ericxsun/hakkero-dataloader/compare/v1.2.14...v1.2.15

What's Changed
* docs: update README by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/9
* feat: check while build index by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/10


**Full Changelog**: https://github.com/ericxsun/hakkero-dataloader/compare/v1.2.13...v1.2.14

What's Changed
* feat: add install make by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/7
* feat: support mllm data by liangzz1991 in https://github.com/ericxsun/hakkero-dataloader/pull/6
* refactor: refactor strategy by ericxsun in https://github.com/ericxsun/hakkero-dataloader/pull/8

New Contributors
* liangzz1991 made their first contribution in https://github.com/ericxsun/hakkero-dataloader/pull/6

**Full Changelog**: https://github.com/ericxsun/hakkero-dataloader/compare/v1.2.12...v1.2.13