Kanama

KANama is now a PyPi Package!

you can install it via `pip install kanama`

Changes and Updates:
- The `KANamav5` MoE model is now renamed to `KANaMoEv1`.
- You can now save and load a model based on the config.json file via calling if `from model.handler import save_pretrained, from_pretrained`. The `save_pretrained` has the path and the model as the input, the `from_pretrained` function only takes in the math of the model and config.json file.
- `quick_inference` has also been moved to `model.handler`.



Example:

python
from model.handler import save_pretrained, from_pretrained, quick_inference

load the model from a path
model = from_pretrained('path/to/model/')

get a tokenizer and tokenize your prompt

save the same model to a path
save_pretrained('path/to/new/model/', model)

- Adding a MOE model (KANamav5)
- Adding a better trainer and fixing batching function
- Adding more examples, with tiny-shakespear.txt and fineweb.jsonl datasets
- Adding loading bars to the SFTTrainer
- General fixes

To use it with python

python
from KANama.model.args import ModelArgs
from KANama.model.KANamav4 import KANamav4

vocab_size, pad_id must be set
ModelArgs.vocab_size = 30
ModelArgs.pad_id = 0

model = KANamav4(ModelArgs)
print(model)