Tensorflow-text

Bug fixes

* Force MacOS builds to build for OSX 10.9 so they can be installed to a wider range of MacOS versions.

Major Features and Improvements

* Add op for solving max-spanning-tree (MST) problems. The code here is intended for NLP applications, but attempts to remain agnostic to particular NLP tasks (such as dependency parsing).
* Add max_spanning_tree_gradient.
* Add support for 'preserve_unused_tokens' options in BertTokenizer.

Bug Fixes and Other Changes

* Documentation updates.
* Reorganize the BUILD file for keras layers.
* Update model server testing. The test script now generates a model that integrates into tf serving's testing infra.
* Remove unneeded heavy dependencies in regex_split library.
* Turn TF text's ConstrainedSequence implementations into standalone callable functions.
* Fix bug in ViterbiAnalysis computation triggered when not using transition_weights.
* Removing testing_utils run_tf_function which is enabled by default now.
* Update patch params to work with Bazel >=1.0.0
* Remove circular dependencies by removing submodule imports from ragged package.
* Prevent lack of ragged_ops.py being released in TF from breaking tf.Text

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Hyunwoo Cho

Minor Updates
---
* BertTokenizer to accept a string tensor for the vocab_lookup_table.

Bug Fixes
---
* Update ICU data name so as to not conflict with core TF in model server.

Major Updates
---
* Added SplitMergeTokenizer.
* Add support for token offsets to BertTokenizer.

Minor Updates
---
* Give BertTokenizer ability to read in a vocab file directly.
* Migrate from std::string to tensorflow::tstring.
* Many build script improvements.
* Update ToDense layer with ragged support attribute.

Bug Fixes
---
* Update SentencePiece to inherit from TokenizerWithOffsets.
* Fix ICU data linking issue.

Bug Fixes
---
* Missing regex character class used in BertTokenizer
* Mac build issue

Major Updates
---
* Added a regex_split op.
* Fixes a bug in case_fold_utf8 and normalize_utf8 ops where they were unable to locate the ICU data file.
* Fixed a problem with the BertTokenizer where it was using merge_dims which is unreleased for the corresponding version of TensorFlow.
* Updated the BertTokenizer to use regex_split to match the exact regex used by original BERT.