[v0.1.4](https://github.com/togethercomputer/together/releases/tag/v0.1.4) - 2023-08-18
**Full Changelog**: https://github.com/togethercomputer/together/compare/v0.0.9...v0.1.4
Changes
- Step by step guidance and examples added to documentation for the completion, files, and finetune sections, including example jsonl dataset that can be downloaded locally for inspection, automated checking and uploading
- optional but very useful together.Files.check function to check the correctness of jsonl files. Gives useful feeback on which first line that is formatted incorrectly, will give the specific reason for being the line being incorrect and how to fix it with a helpful link
- model specific token option to check if model specific tokens are used in dataset
- together.Finetune.create will give useful feedback if user tries to upload anything other than a valid file-id for a file they already have uploaded successfully
Here is an example file check output:
{ 'file_present': 'File found',
'file_size': 'File size 0.2 GB',
'is_check_passed': False,
'key_value': 'Unexpected, value type for "text" key on line 6 of the input '
'file.The value type of the "text" key must be a '
'string.Expected format: {"text":"my sample string"}See '
'https://docs.together.ai/docs/fine-tuning for more '
'information.{"text": {"text":"<human>: Salutations!\\n<bot>: '
'Salutations to you as well! How can I help you today?"}}\n',
'min_samples': 'Processing /Users/carsonlam/Projects/data/bad.jsonl '
'resulted in only 10 samples. Our minimum is 100 samples. ',
'model_special_tokens': 'we are not yet checking end of sentence tokens '
'for this model',
'num_samples_w_eos_token': 0,
'text_field': 'No "text" field was found on line 7 of the the input '
'file.Expected format: {"text":"my sample string"}.see '
'https://docs.together.ai/docs/fine-tuning for more '
'information.{"ext": "<human>: Hiya!\\n<bot>: Hiya! How may '
'I assist you?"}\n'}