Topicnet

New

* Datasets are now also available on HuggingFace: https://huggingface.co/TopicNet.

Fix

* Datasets downloading via dataset manager (that is, fix datasets site URL address; [issue](https://github.com/machine-intelligence-laboratory/TopicNet/issues/92), [pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/93)).
* Thetaless regularizer behavior for some "extreme modality cases" ([issue](https://github.com/machine-intelligence-laboratory/TopicNet/issues/79), [pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/85), related [pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98)).
* Dataset class internals for newer pandas, also "pandas + numpy" collab in some scores ([issue1](https://github.com/machine-intelligence-laboratory/TopicNet/issues/89), [issue2](https://github.com/machine-intelligence-laboratory/TopicNet/issues/94), [pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98)).
* Update some demo notebooks ([issue1](https://github.com/machine-intelligence-laboratory/TopicNet/issues/83), [pr1](https://github.com/machine-intelligence-laboratory/TopicNet/pull/88); [issue2](https://github.com/machine-intelligence-laboratory/TopicNet/issues/87), [pr2](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98)).
* Fix some ARTM-master-related-stuff, freeze "ARTM-compatible" protobuf version for easier installation (related [issue](https://github.com/machine-intelligence-laboratory/TopicNet/issues/90), [pr1](https://github.com/machine-intelligence-laboratory/TopicNet/pull/85), [pr2](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98)).
* Freeze all other package versions in order to facilitate the work of argeologists in the future ([pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98), and minor pr-s: [pr1](https://github.com/machine-intelligence-laboratory/TopicNet/pull/103), [pr2](https://github.com/machine-intelligence-laboratory/TopicNet/pull/104), [pr3](https://github.com/machine-intelligence-laboratory/TopicNet/pull/105), [pr4](https://github.com/machine-intelligence-laboratory/TopicNet/pull/106)).
* "Lick" Readme: add more links, fix some "formatting irregularities" ([issue1](https://github.com/machine-intelligence-laboratory/TopicNet/issues/95), [issue2](https://github.com/machine-intelligence-laboratory/TopicNet/issues/96), [pr1](https://github.com/machine-intelligence-laboratory/TopicNet/pull/85), [pr2](https://github.com/machine-intelligence-laboratory/TopicNet/pull/99)).
* Add long project description for proper setup ([pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/108)), which is now displayed on TopicNet's PyPI page: https://pypi.org/project/topicnet.

Change

* Dummy model now stores *all* score values (not only the last one; [pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98)).
* Frozen score now can be added to topic model (this allows loading model after incomplete score saving; [pr](https://github.com/machine-intelligence-laboratory/TopicNet/pull/98)).

<br/>

<img src="https://github.com/user-attachments/assets/1e0b5149-bfde-4713-b1c2-cd9aa5db9ec6" width="50%" />

Fixed

* Is the score out of control or not in [controller_cube.py](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/cubes/controller_cube.py#L203)
* Separate/multithread mode for cubes in [config_parser.py](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/config_parser.py#L583)
* TopicNet is finally Mac-installable using Pip 🎉

Changed

* From now on the library is compatible with Python **3.7** or higher (not Python 3.6)

New

* Ability to define a [score controller](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/cubes/controller_cube.py#L154) for any custom score
* Experiment's [restore_mode](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/experiment.py#L646): if something happens during the computation, one can resume the process and proceed from last completed cube
* Dataset's dictionary can be altered [using recipes](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/config_parser.py#L173)
* [Thetaless regularizer](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/models/thetaless_regularizer.py#L282) is now more user-friendly: only dataset is required as input, not `n_dw` matrix ([the notebook](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/demos/Topic-Thetaless-Regularizer.ipynb) is also updated)
* Score's [should_compute](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/models/base_score.py#L30): scores may be computed not necessarily on every iteration
* Score's [precomputed_data](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/cooking_machine/models/base_score.py#L147): scores may share some data between each other (eg. one score calculates something and makes the result available for other scores)

* Reworked top_tokens_viewer and top_documnets_viewer
* Added WNTM recipe
* Added dataset_cooc
* Reworked dataset
* Speed up get_possible_modalities
* Reworked write_vw
* Added new regularizer thetaless and demo of its usage and benefits

Various changes, as can be seen in release commit [description](https://github.com/machine-intelligence-laboratory/TopicNet/pull/43)

Fixed

Recipes `topicnet.cooking_machine.recipes` now included in the assembly on PyPi: setup.py updated, project rebuilt and uploaded.

New

* Added demo notebooks:
* [one](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/demos/20NG-GenSim%20vs%20TopicNet.ipynb): some comparison of TopicNet with Gensim library
* [two](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/demos/Making-Decorrelation-and-Topic-Selection-Friends.ipynb): example of analysis of the 20 Newsgroups dataset, more examples of how one can conduct topic modeling with the help of TopicNet and ARTM
* [three](https://github.com/machine-intelligence-laboratory/TopicNet/blob/master/topicnet/demos/20NG-PREPROCESSING.ipynb): more about 20 Newsgroups dataset analysis before actual topic modeling

Fixed

* Improved top tokens html display by TopTokensViewer
* Fixed TopicNet installation via pip: now all the necessary packages should be installed automatically.
So the command `pip install topicnet` should work just fine... for Linux :)

Future Plans

* Add new regularizers
* Add more abilities to control TopicModel's training process — with `model.fit()` function
* Make the library installable with `pip` also for Windows and Mac without BigARTM preinstalled