Tensorflow-ranking

This is the 0.4.2 release of TensorFlow Ranking. The main changes are the TFR-BERT module based on [the Orbit framework in tf-models](https://github.com/tensorflow/models/tree/master/orbit), which facilitates users to write customized training loops. The new components are:

**TFR-BERT in Orbit**

- [tfr.keras.task](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/task.py): This module contains the general boilerplate code to train TF-Ranking models in the Orbit framework. Particularly, there are:
- `RankingDataLoader`, which parses an ELWC formatted data record into tensors
- `RankingTask`, which specifies the behaviors of each training and evaluation step, as well as the training losses and evaluation metrics.
- In addition, there are config data classes like `RankingDataConfig` and `RankingTaskConfig` to store configurations for above classes.
- [tfr.keras.premade.tfrbert_task](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/premade/tfrbert_task.py): This module contains the TFR-BERT specification of the TF-Ranking Orbit task.
- `TFRBertDataLoader`, which subclasses the `RankingDataLoader` and further specifies the feature specs of a TFR-BERT model.
- `TFRBertScorer` and `TFRBertModelBuilder`, which defines a model builder that can create a TFR-BERT ranking model as a Keras model, based on tf-models’ implementation of BERT encoder.
- `TFRBertTask`, which is a subclass of `RankingTask`. It defines the `build_model` behavior. It also defines the `initialization` method which would load an pretrained BERT checkpoint to initialize the encoder. It also provides the function to output the prediction results along with query ids and document ids.
- In addition, there are config data classes like `TFRBertDataConfig`, `TFRBertModelConfig` and `TFRBertConfig` which stores configurations for above classes.
- [examples/keras/tfrbert_antique_train.py](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/tfrbert_antique_train.py): This file provides an example of training a TFR-BERT model on the Antique data set. There is also an [.yaml](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/tfrbert_antique_train_config.yaml) file where users can specify parameter configurations.


Dependencies: The following packages will be installed as required when installing `tensorflow-ranking`.
- `tf-models-official >= 2.5.0`
- `tensorflow-serving-api>= 2.0.0, < 3.0.0`
- `tensorflow==2.5.0`.

This release is one of the major releases for TF-Ranking. It provides full support to build and train a native Keras model for ranking problems. It includes necessary Keras layers for a ranking model, a module to construct a model in a flexible manner, and a pipeline to train a model with minimal boilerplate. To get started, please follow the example [here](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/keras_dnn_tfrecord.py). In addition, the new release adds `RaggedTensor` support in losses and metrics and we provide a handy [example](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/antique_ragged.py) to show how to use it in a ranking model.

The new components are listed below:

- **Keras Layers**:
- Use input packing for layer signatures for SavedModel compatibility.
- `create_tower` function to create a feedforward neural network with batch normalization and dropout.
- `GAMLayer`, a Keras layer which implements the neural generalized additive ranking model.
- Update build method of `DocumentInteractionAttention` layer to ensure SavedModel is restored correctly.

- **ModelBuilder to build `tf.keras.Model` using Functional API**:
- `AbstractModelBuilder` class for users to inherit.
- `ModelBuilder` class that wraps the boilerplate code to build `tf.keras.Model` for a ranking model.
- `InputCreator` abstract class to implement `create_inputs` in `ModelBuilder`.
- `FeatureSpecInputCreator` class to create inputs from `feature_spec`s.
- `TypeSpecInputCreator` class to create inputs from `type_spec`s.
- `Preprocessor` abstract class to implement `preprocess` in `ModelBuilder`.
- `PreprocessorWithSpec` class to do Keras preprocessing or feature transformations with functions specified in Specs.
- `Scorer` abstract class to implement `score` in `ModelBuilder`.
- `UnivariateScorer` class to implement univariate scoring functions.
- `DNNScorer` class to implement fully connected DNN univariate scoring.
- `GAMScorer` class to implement feature based GAM univariate scoring.

- **Pipeline to wrap the boilerplate codes for training**:
- `AbstractDatasetBuilder` abstract class to build and serve the dataset for training.
- `BaseDatasetBuilder` class to build training and validation datasets and signatures for SavedModel from `feature_spec`s.
- `SimpleDatasetBuilder` class to build datasets with a single label feature spec.
- `MultiLabelDatasetBuilder` class to build datasets for multi-task learning.
- `DatasetHparams` dataclass to specify all hyper-parameters used in `BaseDatasetBuilder` class.
- `AbstractPipeline` abstract class to train and validate the ranking `tf.keras.Model`.
- `ModelFitPipeline` class to train the ranking models using `model.fit()` compatible with distribution strategies.
- `SimplePipeline` class for single-task training.
- `MultiTaskPipeline` class for multi-task training.
- An example [client](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/keras_dnn_tfrecord.py) to showcase training a deep neural network model with a distribution strategy using `SimplePipeline`.
- `PipelineHparams` dataclass to specify all hyper-parameters used in `ModelFitPipeline` class.
- `strategy_utils` helper module to support `tf.distribute` strategies.

- **RaggedTensor support in losses and metrics**:
- Losses in `tfr.keras.losses` and metrics in `tfr.keras.metrics` support to act on `tf.RaggedTensor` inputs. To do so, set argument `ragged=True` when defining the loss and metric objects:
- E.g.: `loss = tf.keras.losses.SoftmaxLoss(name=’softmax_loss’, ragged=True)`
- Add this argument in `get` to get the losses and metrics support ragged tensors: `loss = tf.keras.losses.get(‘softmax_loss’, ragged=True)`
- An example [client](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/antique_ragged.py) to showcase training a deep neural network model using `model.fit()` with ragged inputs and outputs.

Dependencies: The following packages will be installed as required when installing `tensorflow-ranking`.

This is the 0.3.3 release of TensorFlow Ranking. It depends on `tf-models-official >= 2.4.0` and `tensorflow-serving-api>= 2.0.0, < 3.0.0`. It is compatible with `tensorflow==2.4.1`. All of these packages will be installed as required packages when installing `tensorflow-ranking`.

The main changes in this release contain the Document Interaction Network (DIN) layer and layers for training Keras models using Functional API. The new components are listed below:

- **Document Interaction Network**: See [paper](http://research.google/pubs/pub49364).
- Building Keras ranking models for DIN using Keras Preprocessing Layers.
- Native Keras training: An example [client](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/antique_kpl_din.py) to showcase such a model using `model.fit()`.
- Estimator based training: Another example [client](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/examples/tf_ranking_tfrecord.py#L74) to showcase training a DIN model as an Estimator.
- [`tfr.keras.layers.DocumentInteractionAttention`](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/python/keras/layers.py#L311): A keras layer to model cross-document interactions. Applies cross-document attention across valid examples identified using a mask.

- **Keras Layers**: for easy transformation of context and example features and related utilities.
- [`tfr.keras.layers.FlattenList`](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/python/keras/layers.py#L28): Flattens the `batch_size` dimension and the `list_size` dimension for the `example_features` and expands `list_size` times for the `context_features`.
- [`tfr.keras.layers.ConcatFeatures`](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/python/keras/layers.py#L217): Concatenates context features and example features in a listwise manner.
- [`tfr.keras.layers.RestoreList`](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/python/keras/layers.py#L129): Output layer to restore `batch_size` dimension and `list_size` dimension for the output shape of logits.

- **Others**
- [`tfr.keras.metrics.get(metric_key)`](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/python/keras/metrics.py#L59): Add a `get` metric factory for keras metrics.
- Masking support in tfr.data: Add support for parsing a boolean `mask` tensor which indicates number of valid examples via `mask_feature_name` argument in [`tfr.data._RankingDataParser`](https://github.com/tensorflow/ranking/blob/a197718f2d8eb71e02b5d164bcf72d08716cb6b2/tensorflow_ranking/python/data.py#L108) and all associated input data parsing and `serving_input_fn` builders.

In the latest release of [TensorFlow Ranking v0.3.2](https://github.com/tensorflow/ranking/releases/tag/v0.3.2), we introduce TFR-BERT extension to better support ranking models for text data based on BERT. [BERT](https://arxiv.org/abs/1810.04805) is a pre-trained language representation model which has achieved substantial improvement over numerous NLP tasks. We find that fine-tuning BERT with ranking losses further improve the ranking performance ([arXiv](https://arxiv.org/abs/2004.08476)). You can read detailed information about what is included in TFR-BERT extension [here](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/extension/README.md#tfr-bert-extension). There is also an example showing how to use TFR-BERT [here](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/extension/examples/tfrbert_example.py).

This is the 0.3.1 release of TensorFlow Ranking. It depends on `tensorflow-serving-api==2.1.0` and is fully compatible with `tensorflow==2.2.0`. Both will be installed as required packages when installing `tensorflow-ranking`.

The main changes in this release are canned [Neural RankGAM](https://arxiv.org/abs/2005.02553) estimator, canned DNN estimators, canned Neural RankGAM keras models and their examples. The new components are:

- **Neural RankGAM**
- [`make_gam_ranking_estimator`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/estimator.py#L703), which makes a canned estimator of a neural ranking generalized additive model. An [example](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/tf_ranking_canned_gam.py) client to showcase the usage of `make_gam_ranking_estimator`.

- [`make_dnn_ranking_estimator`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/estimator.py#L404), which makes a canned estimator of a feed-forward neural network for ranking. An [example](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/tf_ranking_canned_dnn.py) client to showcase the usage of `make_dnn_ranking_estimator`.

- [`GAMRankingNetwork`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/canned/gam.py#L70), which encapsulates Neural RankGAM models in a keras ranking network. An [example](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/keras/keras_m2e_gam_tfrecord.py) client to showcase the usage of `GAMRankingNetwork`.

- **Keras**

- Add serialization and deserialization support for feature columns via [`tfr.keras.feature.serialize_feature_columns`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/feature.py#L25) and [`tfr.keras.feature.deserialize_feature_columns`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/feature.py#L51). RankingNetwork’s `get_config` and `from_config` methods rely on this.

This is the 0.3.0 release of TensorFlow Ranking. It depends on `tensorflow-serving-api==2.1.0` and is fully compatible with `tensorflow==2.1.0`. Both will be installed as required packages when installing `tensorflow-ranking`.

The main changes in this release are related to the DNN Estimator Builder and Keras APIs.

A DNN Estimator Builder is available at [`tfr.estimator.make_dnn_ranking_estimator()`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/estimator.py#L401).

For Keras, we provide an [example](https://github.com/tensorflow/ranking/tree/master/tensorflow_ranking/examples/keras/keras_m2e_tfrecord.py) to showcase the use of Keras APIs to build ranking models , and a [documentation](https://github.com/tensorflow/ranking/tree/master/tensorflow_ranking/examples/keras/README.md) providing step-by-step user instructions outlining the Keras user journey.

The new Keras components are:
+ Losses: Ranking losses in Keras object oriented loss format, along with a base class and a factory method. The APIs are:
+ [`tfr.keras.losses.get(loss_key)`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/losses.py#L47)
+ Base class: [`tfr.keras.losses._RankingLoss`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/losses.py#L104)
+ Losses under [`tfr.keras.losses.*`](https://github.com/tensorflow/ranking/tree/master/tensorflow_ranking/python/keras/losses.py)

+ Metrics: Ranking metrics in Keras object oriented metric format, along with a base class and a default metrics getter method. The APIs are:
+ [`tfr.keras.metrics.get_default_metrics()`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/metrics.py#L31)
+ [`Base class: [`tfr.keras.metrics._RankingMetric`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/metrics.py#L53)
+ Metrics under [`tfr.keras.metrics.*`](https://github.com/tensorflow/ranking/tree/master/tensorflow_ranking/python/keras/metrics.py)

+ Feature Transformations: tfr.keras.feature.EncodeListwiseFeatures, to convert sparse ranking features to dense. The APIs are:
+ [`tfr.keras.feature.EncodeListwiseFeatures`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/feature.py#L129)

+ Ranking Network: Base classes for building Ranking Networks, which define scoring logic. The APIs are:
+ [`tfr.keras.network.RankingNetwork`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/network.py#L26)
+ [`tfr.keras.network.UnivariateRankingNetwork`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/network.py#L139)

+ Premade Networks: We support premade architectures users can access out-of-the-box. The APIs are:
+ [`tfr.keras.canned.DNNRankingNetwork`](https://github.com/tensorflow/ranking/tree/master/tensorflow_ranking/python/keras/canned/dnn.py)

+ Keras Model : Ranking models can be built using Keras Functional Model API. The APIs are:
+ [`tfr.keras.model.create_keras_model()`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/model.py#L23)

+ Integration with Estimators and RankingPipeline: Keras model can be converted to Estimator to use Estimator’s training utilities and is compatible with RankingPipeline. The APIs for conversion are:
+ [`tfr.keras.estimator.model_to_estimator()`](https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/python/keras/estimator.py#L21)