Deep-lynx

This release of Deep Lynx is considered a minor, breaking release.

The purpose of the Deep Lynx query layer is to query and filter data ingested in previous steps. While the previous version of this query layer allowed for sufficient filtering of nodes based on Metatype properties, this release allows for querying of and filtering on nodes, edges, or graph-like node-edge structure.

These changes are considered breaking because an extra wrapper was placed around Metatype queries to account for possible overlap in naming conventions when querying on Metatype Relationships. While the previous version of this query layer allowed queries to begin by stating the Metatype name directly to be queried on, this query must now be wrapped within the metatypes{} object.

As mentioned earlier, edges can now be queried on using the relationships{} object, and graph-like data can be queried using the graph{} object. Additionally, when querying on Metatypes, users can now filter by relationships to other metatypes. Documentation has been updated to exhibit use cases for these newly supported behaviors.

These changes impact any code that relies on the new GraphQL query layer for data retrieval. Any code referencing the legacy query layer will not be affected. The Deep Lynx frontend does not yet reflect these changes.
This release also contains a few minor bug fixes and security updates.

This update to Deep Lynx is considered a minor, non-breaking release.

This update contains various security changes, such as package updates and changes to how the internal application handles exceptions and errors. This update contains no changes to endpoints or UI.

This release of Deep Lynx is considered a major, non-breaking release.

Deep Lynx is unique in its ability to store data in graph-like format and under a user-defined ontology. Data has been versioned in Deep Lynx since roughly version 0.0.5. Data ingested from sources have their changes tracked and theoretically a user could see what the data looked like at any given point in time.

However, while data was versioned the ontology it was stored under was not. This led to problems when users might have edited the ontology - removing or requiring new fields for example. These changes would not be tracked and if a user accessed data stored under an old version of a class, they might see deprecated properties, or be lacking required data. There was no way to reprocess data that had come in to fit the new ontology and the type mapping process had to be manually updated to handle changes.

To solve these problems and give users confidence in the accuracy of stored data, versioning was introduced to ontologies stored in Deep Lynx. Now when users query data they will always see the class and properties the way they existed when that data was stored. Changes to the ontology are now tracked and managed- and final approval of changes prior to application now falls to the container administrators, not all users.

https://gitlab.software.inl.gov/b650/Deep-Lynx/-/wikis/Ontology-Versioning

This is a minor release for Deep Lynx and contains various bugfixes and stability corrections. Please run `npm run migrate` as soon as you pull this release as database structure has changed.

Added the ability to back artifact storage with Postgres's Large Object storage capability. This is now the default for artifact storage over local filesystem, as it's a better respecter of keys and not overwriting data.

Also includes routes for managing changelists, ontology versions, and container alerts - though these have not been fully implemented on the GUI as of yet.

Deep Lynx has been modified to no longer process data and data sources in batches, but in an event-driven queue based manner. Using this methodology we hope to scale more efficiently to larger data sets and allow users to more easily process data. You can find a diagram attached detailing the changes.