- **mysfire/processors**: Added new tokenizers processor, minor lightning db changes
Fix
- **mysfire/__init__.py**: fixed import bugs with new import system - **mysfire/**: Changed the way processors are imported - **mysfire/cloud_utils.py**: Changed prefix of GCS to gs:// from gcs://
0.4.4
Fix
- **mysfire/processors/nlp**: Added new transformers tokenizer, fixed bugs in parser
0.4.3
Fix
- **mysfire/processors/nlp**: Fix an issue where mysfire does not have tokenization processors
0.4.2
Fix
- **setup.cfg**: More fixes for versions not available in pypi
0.4.1
Fix
- **setup.cfg**: Fix pyarrow version bug
0.4.0
Fix
- **setup.cfg**: Fixed dependencies - **mysfire/cloud_utils.py**: Fixed issues with typing in cloud utils - **mysfire/vars_registry.py**: Made Dataset class pickleable with cloudpickle - **mysfire/dataset**: Fix small memory leak in _flatten_dicts
Feat
- **mysfire/cloud_utils.py**: Added rudimentary GCS connection - **mysfire/dataset.py**: Added light-weight validation of sample data to dataset - **mysfire/dataset.py**: Added ability to resample from dataset on processor exceptions
Refactor
- **mysfire/cloud_utils.py**: Refactored underlying S3 tools so that they are easier to swap with GCS