Deep-lynx

Various bugfixes and ability to save GraphQL queries to a `json` or `csv` file.

This release is considered a minor, non-breaking release of DeepLynx. The following features have been added and bugs have been corrected.

- Service Users can now be accessed as "External Applications" from inside DeepLynx's UI. They can be used to assign external applications api keys.
- Various bugfixes

This release is considered a minor, non-breaking release of DeepLynx. The following features have been added and bugs have been corrected.

- Timeseries data processing enabled - more information below and on the wiki https://gitlab.software.inl.gov/b650/Deep-Lynx/-/wikis/Querying-Timeseries-Data
- Ontology versioning enhanced and various versioning bugs corrected
- The server crash that happened on an invalid json payload when uploading data has been corrected


As of this release, DeepLynx now has the capability to store and query timeseries data without having to first store that information on the graph. Prior to this release, time series data was handled a in the following ways, each considered suboptimal:

- **Storing each timeseries entry as a node on the graph:** Considering the amount of time series data there is per sensor, this quickly overwhelms the graph and had the potential to drastically increase latency when querying or manipulating the graph. Storing as nodes also did not maintain order, order would have to be artificially enforced by choosing a property of the timeseries data to sort on.
- **Storing timeseries data as files:** In this solution users would typically create a node with metadata about the series of measurements contained in a file or multiple files. They would then upload those files to DeepLynx&39;s blob storage system and attach them to the relevant node. While this eliminated the problem of having an unwieldly number of nodes, hiding the files in blobs meant that DeepLynx lacked the capability to query parts of the data not contained or covered in the metadata stored earlier. This method also necessitated the user downloading the data and using a third-party program to display it.

To improve on the existing solutions, the following feature set was adopted as a target for DeepLynx timeseries data storage and querying capabilities.

- Timeseries data must follow the same data ingestion route as before
- Timeseries data must be mapped to a timeseries specific database table prior to storage
- Users must be able to query timeseries data quickly and without having to first download or leave DeepLynx
- Users must be able to perform simple filtering and ordering on their timeseries data
- Support for managing terabytes of timeseries data

We&39;re happy to note that the 0.3.3 release has met these goals.

Various bugfixes

Description
- modified the package.json file to clean up npm commands and clarify the build process
- modified the Dockerfile to correctly build and run DeepLynx in a reproducible manner
- modified the database migration functionality - migrate is no longer a separate step but gets run on each application startup, removed old migrate commands
- added a docker_compose.yml file - this allows us to quickly scaffold and connect a Timescaledb image and DeepLynx image in a reproducible manner. Containers are pulled from the registries, ensuring a stable build each time
- the encryption key is now automatically generated and saved if a user no longer provides their own key, modified the config to handle this automatic generation
- modified the readme to reflect all changes
- corrected tests
- corrected processing thread to avoid swamping the queue
- updated UI, and fixed various bugs

Motivation and Context
Users were struggling to build DeepLynx from source, and the Docker capability was ignored or not well known - as well as not handling the database migration correctly. In order to provide end users with a quick and easy way to get DeepLynx up and running we've added Docker Compose capability and greatly cleaned up the steps for building from source.

We were also running into various issues with the processing thread causing queue buildup. Our changes have made it so processing is quicker and does not swamp the processing queue.


[DeepLynxUAT.xlsx](https://github.com/idaholab/Deep-Lynx/files/8556055/DeepLynxUAT.xlsx)

This is a minor, non-breaking release of Deep Lynx. It lays the foundation for our new time series data as well as solving a few bugs and streaming the type mapping process.

Release Notes
- updated packages for both server and UI, `npm audit` ran
- modified the gui's client code to return the full error on failed api call instead of just the status
- updated the delete dialogs to set a timer of 5 seconds instead of 10 when warning users
- updated the transformation dialog to flow more easily and be more visually appealing
- added time series specific fields and operations to the transformation dialog - such as table mapping and node attachement parameters
- various changes to the TransformationT type to reflect changes in the backing code
- various translation updates
- ontology versioning disabled by default
- eslint cleanup of the ui
- added timescale container run command to package.json
- added migrations and new fields for transformation object, updated mapper and repository to match