- `pad_batch` function more general - pad logic for custom fields using `add_padding_ops` - model output can include all the HuggingFace outputs, using `return_all=True` in `TransformerEmbedder`
1.5.1
Exposed `has_double_sep` and `has_starting_token` as `Tokenizer` properties
1.4.8
Fix `Tokenizer` regression
1.4.7
- Added properties to `Tokenizer`, exposing model special tokens - Expose `num_special_tokens` to get the number of special tokens the model needs
1.4.6
Exposed `save_pretrained` in `TransformerEmbedder`