Neuralogic

Bug Fixes

- Fixed cycle breaking 44
- Wrong string representation of a negated relation

Changes

- Reintroduction of relation negation:
- `~R.rel` is a negation
- `-R.rel` applies reverse transformation function

Bug fixes

- Fixed incorrect compression (removal) of transposition neurons.

Bug fixes

- Fixed missing skip connection in Transformer encoder/decoder

Additions:

- Transformer modules - `Transformer`, `TransformerEncoder`, and `TransformerDecoder`.
- Attention modules - `Attention`, `MultiheadAttention`.
- `PositionalEncoding` module.
- Softmax as a (splittable) aggregation function.
- Concatenation as an aggregation function.
- Reshape transformation function.
- Slice transformation function.
- Learning rate decay as an optimizer argument.
- Batching support.
- Progress bar for tracking dataset-building progress.
- Special arithmetic predicates for addition (`add`), subtraction (`sub`), and modulo (`mod`).

Changes:

- Concatenation can now be parametrized with a dimension.
- Refactored MLP module - the current version is more compact.
- Changed the behavior of `Variable`/`V` factory - only the first letter is capitalized.
- Performance improvements.

Breaking changes

- Arguments of `build_dataset` methods, except for the dataset, are now forced to be keyword arguments.
- Changes in recurrent modules constructors (+ removal of custom next predicate in favor of the special next predicate).

Additions

- Building dataset can now display progress (spinner + number of built samples) - via the `progress` argument of `build_dataset` methods.
- New `norm` transformation function.
- New `prune_only_identities` settings (`chain_pruning` customization to prune identities only)
- Computation cycle breaking

Changes

- Logging is now enabled by default (to stdout with the severe level)
- Minor tweaks to user-friendliness

Bug fixes

- Fixed softmax combination
- Fixed settings function state while using SoftEntropy


Major part of changes were implemented in the [backend](https://github.com/GustikS/NeuraLogic)