** Modalities refactor: Thanks to Dustin, all modalities are now an enum and just functions, making it easier to understand what's happening in the model. Thanks Dustin!
**[Model-Based Reinforcement Learning for Atari](https://arxiv.org/abs/1903.00374)** using T2T, please find a nice writeup in at https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/rl/README.md -- thanks a lot to all the authors! lukaszkaiser mbz piotrmilos blazejosinski Roy Campbell konradczechowski doomie Chelsea Finn koz4k Sergey Levine rsepassi George Tucker and henrykmichalewski !
**[TRAX](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/trax) = T2T + [JAX]**(https://github.com/google/jax) - please try out and give us feedback at #1478
New Models:
* Evolved Transformer, thanks stefan-it for adding the paper in 1426
* textCNN model by ybbaigo in 1421
Documentation and Logging:
* MultiProblem by cwbeitel in 1399
* ML Enginge logging in 1390 by lgeiger
Thanks again cwbeitel and lgeiger -- good docs and logging goes a long way for understandability.
Bugs fixed:
* t2t_decoder checkpoint fix in 1471 by wanqizhu
* xrange fix for py3 by in 1468 lgeiger
* Fixing COCO dataset in 1466 by hbrylkowski
* Fix math problems by artitw
* Decoding rev problems enzh by googlehjx on 1389
* And honourable mentions to qixiuai , 1440
Many many thanks wanqizhu lgeiger hbrylkowski artitw googlehjx and qixiuai for finding and fixing these and sorry for missing anyone else -- this is really really helpful.
Code Cleanups:
* Registry refactor and optimizer registry by jackd in 1410 and 1401
* Numerous very nice cleanup PRs ex: 1454 1451 1446 1444 1424 1411 1350 by lgeiger
Many thanks for the cleanups jackd and lgeiger -- and sorry if I missed anyone else.
v.1.12.0
Summary of changes:
PRs:
* A lot of code cleanup thanks a ton to lgeiger ! This goes a long way with regards to code maintainability and is much appreciated. Ex: PR 1361 , 1350 , 1344 , 1346 , 1345 , 1324
* Fixing LM decode, thanks mikeymezher - PR 1282
* More fast decoding by gcampax, thanks! - PR 999
* Avoid error on beam search - PR 1302 by aeloyq , thanks!
* Fix invalid list comprehension, unicode simplifications, py3 fixes 1343, 1318 , 1321, 1258 thanks cclauss !
* Fix is_generate_per_split hard to spot bug, thanks a lot to kngxscn in PR 1322
* Fix py3 compatibility issues in PR 1300 by ywkim , thanks a lot again!
* Separate train and test data in MRPC and fix broken link in PR 1281 and 1247 by ywkim - thanks for the hawk eyed change!
* Fix universal transformer decoding by artitw in PR 1257
* Fix babi generator by artitw in PR 1235
* Fix transformer moe in 1233 by twilightdema - thanks!
* Universal Transformer bugs corrected in 1213 by cfiken - thanks!
* Change beam decoder stopping condition, makes decode faster in 965 by mirkobronzi - many thanks!
* Bug fix, problem_0_steps variable by senarvi in 1273
* Fixing a typo, by hsm207 in PR 1329 , thanks a lot!
New Model and Problems:
* New problem and model by artitw in PR 1290 - thanks!
* New model for scalar regression in PR 1332 thanks to Kotober
* Text CNN for classification in PR 1271 by ybbaigo - thanks a lot!
* en-ro translation by lukaszkaiser !
* CoNLL2002 Named Entity Recognition problem added in 1253 by ybbaigo - thanks!
New Metrics:
* Pearson Correlation metrics in 1274 by luffy06 - thanks a lot!
* Custom evaluation metrics, this was one of the most asked features, thanks a lot ywkim in PR 1336
* Word Error Rate metric by stefan-falk in PR 1242 , many thanks!
* SARI score for paraphrasing added.
Enhancements:
* Fast decoding !! Huge thanks to aeloyq in 1295
* Fast GELU unit
* Relative dot product visualization PR 1303 thanks aeloyq !
* New MTF models and enhacements, thanks to Noam, Niki and the MTF team
* Custom eval hooks in PR 1284 by theorm - thanks a lot !
RL:
Lots of commits to Model Based Reinforcement Learning code by konradczechowski koz4k blazejosinski piotrmilos - thanks all !