Scistag

It's been a while since the last release due to several events real-life and a quite long vacation after the last release.

Anyways here it finally is: v0.9.0 which fixes some minor VisualLog formatting issues but most importantly makes the library compatible to Pydantic 2 which in combination with FastAPI is now the de-facto standard for type-safety and clean serialization and de-serialization.

On top of that the possibility to style your logs in more detail has been added to SciStag, see SciStag/Examples/c01_basics for several examples.

Mermaid graphs and tracking of external files in VisualLog

An image can say more than 1,000 words... and so can a graph or a sequence diagram.

The major change in this release is the integration of the support of **Mermaid** diagrams.<br>
[Mermaid](https://mermaid.js.org/) is an awesome diagramming and charting tool written in JavaScript which can parse intuitive, markdown like ASCII code and render different kinds of graphs and charts of this such as classical flowcharts, class diagrams, sequence diagrams etc.
For more information visit the official homepage: https://mermaid.js.org/

When using **Markdown** you can now use Mermaid code cells such as
\`\`\`mermaid
graph LR
A-->B
\`\`\`
to add a graph to your log.

To add a graph to your cell use `LogBuilder.chart.mmd("MARKDOWN SOURCE w/o fences")`.

As such graphs can become pretty large and complex also three further major features have been added in 0.8.2:
* Directly embedding external markdown files via `LogBuilder.md.embed(markdown filename)` which then can contain a Mermaid graph and/or just classical markdown for a long text part.
* Embeddeing of Mermaid files (*.mmd / *.mermaid) via `LogBuilder.chart.embed(mmd filename)`
* Cells can now **watch** external source files via `add_dependency` and automatically rebuild the cell when the external file got modified.
This function is used by the new Markdown and Mermaid embedding features when the `watch` flag is set to True. This allows you to modify the graph or Markdown files in an external editor such as PyCharm and see the impact asap in VisualLog's live view.

Major enhancements to VisualLog

The biggest change of v0.8.1 is the implementation of different kinds of cache references and bindings. Until now one had to access data being streamed from one LogBuilder cell to another with a classical, dictionary-like item access such as `my_value = self["my_value"]`

This had two major flaws: A typo can happen easily while typing a variable's name and the IDE did not know which kind of type could be assumed leading to further potential errors due to missing code analysis.

Second there was currently no clean way to access the cache's data from another thread, e.g. for time consuming workloads which should not be executed in the main thread.

For the first case there is now the new system of CacheLinks which works as follows: In the LogBuilder's __init__ function (like for a common object) you assign the class attribute a so called CLRef via self.clref(default_value), optionally you can also set the parameter create to false if you do not want to create the variable initially such as follows:
`self.my_value: int = self.clref(0)`

This tells the IDE the type and will initialize the value in the cache (if create isn't set to False). When ever you now access the variable's name "magic" in the LogBuilder's class will forward all read and write accesses from and to the cache this you can use a cache's data entry like every common class attribute.

For helper classes where this is no possibility, e.g. other threads, you can create a so called CacheRef via self.cache.create_ref. It will provide a thread-safe link to the LogBuilder's cache and most of all the possibility to (ex)change cache element's asynchronously, e.g. input data to output data.

VisualLog

* Disable automatically catching stdout in cells by default. Will be added to the options soon.
* Enhanced Cache functionality:
* Changes can now be staged via set_async and lpush_async. The staged changes can then be applied in the main thread by async_fetch. This way corrupting the cache in the middle of a building process can be avoided as now the main thread has full control over WHEN to apply these updates
* Test enhancements for Cache and CacheRef
* Implemented the LSelect widget (a combo box)
* Added the class CacheRef which can read and write one specific cache value and decide whether it will be written asap or via the new async methods.
* Added basic support for pages and tabs to LogBuilder. It is now possible to assign a page and/or a tab to a cell and use LogBuilder.page, LogBuilder.tab or LogBuilder.set_page to select which cells shall be visible.
* Added support for cell groups to LogBuilder. It is now possible to assign one or multiple groups to a cell and to filter the visible cells via LogBuilder.visible_groups and LogBuilder.hidden_groups
* Optimized the logging of simple tables
* Cells can not verify requirements by value via keyName==assumedValue
* Added the possibility to bind LogBuilder virtual attributes to element's in the cache for easier usage and better code insight support. Via `self.my_attribute = self.clref(default_value)`` a LogBuilder subclass attribute is bound to the corresponding name in the cache.
* Added tests for the LSelect class
* data, stream and once cells are now static to don't create waste within the html output

ImageStag

* Bugfix: Disable lazy loading for PIL images as they caused evil side effects in multi-threaded environments

DataStag

* It is now possible to transfer dictionaries, pandas dataframes and pandas time series via a remote DataStag connection

Bugfixes
* DataStagVault: It was not possible to pop the last element of a list via connection.pop(index=-1) because negative values were invalid in general. Now negative values are supported as for common Python lists.
* Editing an LSlider's value in the edit field directly did not trigger any change event
* An (empty) Image could not initialized as grayscale image

Massive enhancement & refactoring of the VisualLog module

This update and pull request was a major and full rework of the whole VisualLog sub module with in consequence massive amounts of changes.

Major changes:
* To completely decouple the class VisualLog from user specific data to allow concurrent users session VisualLog was extensively reworked and split into multiple classes. In consequence a class PageSession (which stores a user's page / result data and is the input and output "gate" to the web) and a class LogBuilder (providing the writing functions) were introduced. All links from those two classes back to the VisualLog were removed.
* To minimize the amount of parameters passed to the VisualLog and LogBuilder's constuctors and run functions and to to maximize configuration possibilities. Basically all prior parameters and configuration flags were from from VisualLog and LogBuilder to the scistag.vislog.options sub module. A basic option set can be create via VisualLog.setup_options()
* To create real dynamic pages with a two way communication between Python and JavaScript, e.g. to receive all kinds of input events in the browser on the one hand but most of all to also be able to update and refresh single areas of the page without requiring a field update. In consequence the so called LogElements were introduced. They basically represent a named dom object inside the HTML element which a unique name and can be updated individually.
* To cleaner structure large pages. For this so called "cells" were introduced which define a section of the page. The decorator cell can be used on any function of a class derived of LogBuilder to specific a region of the log. A cell can be of one of the following types:
* cell: Displays a part of the log, without any special, visual decorations. A cell can request being updated in specific time intervals or when single values such as variables in the cache are changed.
* section: Defines a section, either with or without title which is visually separated with a horizontal line containing the section's name
* data: A cell which is not able to create visual objects and is purely ment to load and/or process data
* once: A cell which is executed not more than once, e.g. loading a neural network upon first use
* stream: A cell with defined input and output variables
* Two new major widgets were introduced:
* An LSlider widget which lets the user choose a numeric value either with floating point precision or as integer value
* An LFileUpload widget which lets the user upload one or multiple files
* It is now very easy possible to insert one LogBuilder's content into another LogBuilder via LogBuilder.create_backup and .insert_backup
* When creating regression tests you can now just specify "???" as a hash value on the line of assertion and LogBuilder.test will automatically replace the value with the new, correct hash, e.g .via assert_cp_diff.
* It is now possible to log colored images to a Linux and OS X terminal (most important feature of all)
* It is now possible to easily log emojis to a log via LogBuilder.emoji
* Added the possibility to align text and elements via LogBuilder.align
* Massively enhanced the possibilities of logging tables via LogBuilder.table
* VisualLogBuilder was renamed to LogBuilder
* Introduced the black code formatter - applied the black code formatter to all files
* Added a new major VisualLog component named Cell. A cell is a dynamic area within a log which can be updated individually at any point of time using JavaScript.
* Added the helper class Timer as VisualLog widget which can be configured to be called once after a delay and/or in certain intervals to update a cell
* Added the class PageSession which is now responsible for storing the whole page data for one browser session where as a straight forward, non-hosted log acts as a single session
* Added the class LogElement which defines a part of a log in form of a bytes string and can contain further sub elements. Upon rendering all LogElements are combined to the final result.
* Added the possibility to add a button to a VisualLog and to react to it's clicks
* Added the possibility to add a range slider to dynamically select integer or float values. Optionally a number edit field or value display can be shown
* Added the possibility to log all datatypes supported by LogBuilder.add to a table
* Text can now be aligned to the left, center or right via LogBuilder.align
* HTML elements can now be custom styles via LogBuilder.style.css
* Added a widget LComparison to easily compare two images (and other data types later)
* Added the extension Snippet which allows to record logged data instead of logging it directly to the log and to replay this data at some later * Added the possibility to add a background to an alpha-transparent image with a single line of code
* Added the possibility to load images from numpy arrays and to apply a color palette to them
* Fixed a bug which incorrectly displayed checkerboard backgrounds
* Adjusted all affected unit tests

SciStag 0.7.5 is a huge step towards the prior two early previews and so was the step in the version number.

Especially the **vislog** module and its classes VisualLog und VisualLogBuilder have massively grown in regards of functionality, visual appearance but especially also test coverage and cleanness of the interfaces themselves, becoming more and more mature.

Other very large improvements have been made to FileStag, especially FileSource and FileSink which really make it blazing fast now importing data from Azure or zip files, process it and store them, re-bundle and re-upload them.

SciStag v0.0.3 introduces massive enhancements for working with data in the cloud via FileStag and introduces the vislog (VisualLogStag) sub package which allows the generation of a HTML and Markdown documentation from Python.

For examples of the new vislog package see https://github.com/SciStag/SciStag/tree/main/scistag/examples/vislog