Vdms

What's New

- **Filters to Identify Near-Neighbor Groups (FLINNG):**
- Integrated a new indexing and similarity search library (FLINNG) that addresses the computational challenges of high-dimensional similarity search by presenting an index with fast construction time, low memory requirement, and zero query-time distance computations
- Instructions are provided (https://github.com/IntelLabs/vdms/wiki/FLINNG-Library-and-Performance-Details)

- **Periodic backup (Auto-replication):**
- Users can automatically backup db folder of VDMS periodically and restore the database as needed
- Instructions are provided (https://github.com/IntelLabs/vdms/wiki/VDMS-Backup-and-Restoring-Options)

- **Kafka Connectors:**
- We are moving towards distributed VDMS and have provided Kafka connectors, configurations, instructions, and examples to use VDMS in a distributed manner leveraging Kafka
- Instructions are provided (https://github.com/IntelLabs/vdms/wiki/Run-Kafka-VDMS-Example)

- **Video Improvement**
- Addressed memory leak in video command

- **Independent C++ Client:**
- Ability to build client independently without building VDMS server

What's New

- Flexible Retention Policy (Auto-deletion):
- Users can automatically have node removed from VDMS
- Internal Database Retry:
- Retry query before asserting and exiting database
- Allow User-defined Operators in VCL:
- User can define a new operator in VCL and expose it in the VDMS API. Provided a sample custom operation
- Improve Descriptor Search:
- Fixed descriptor bug (insert multiple descriptors)
- Added the ability to search descriptors by query blob AND constraints
- JPEG Storage Quality:
- Image differences due to lossy compression of JPEG. Added disclaimer to user to use lossless format (PNG) for inference applications (https://github.com/IntelLabs/vdms/wiki/AddImage#inference-image-format )
- Increase Buffer Size:
- We increased the max buffer size for database transactions to 1GB and set a default size of 32MB (previous max)
- Removed Dockerfiles:
- Only provide base Dockerfile

This release includes an updated version of VDMS that compiles against updated versions of OpenCV, FAISS, and Protobuf.
Features Added:
1) Data deletion and expiration
2) VCL Image storage to Binary

What's new

* Enhanced metadata store (through PMGD):
- Fully ACID compliant transactions
- Support for persistence for contemporary systems with no PM through MSYNC

* Support for user-defined blobs:
- Insert and retrieve arbitrary blobs
- Add and query user-defined properties for blobs

* Support for querying of relationships and its properties (FindConnection Command)

* Support for Visual Descriptors (aka feature vectors):
- New commands for adding and indexing feature vectors.
- Native support for k-nearest neighbor search operation at query time.
- Support for Facebook's Faiss indexes
- Support for a Sparse Arrays based indexes (through TileDB 1.3.1)
- Support for Classification operation, based on user-defined labels.

* Enhanced support for operations over Images:
- New Operation: Flip (vertical, horizontal, or both)
- New Operation: Rotate (and optional resize adjust)
- TileDB image format updated to TileDB 1.3.1

* Support for Bounding Boxes:
- Support for insert/retrieve bounding boxes and links with images.

* Support for Storing and Accessing Videos (through OpenCV 3.4):
- Support for codecs: h.264, xvid, h.263.
- Support for containers: avi, mp4, mov.
- Support for inserting and querying videos with user-defined properties.
- Support for crop, resize, threshold, and interval operations at query time.

* Update Commands (Update for Entities, Connections, Images, Bounding Boxes and Videos)

* Full Support for Python3 (backward compatible with Python 2):
- Installation of Python Client through pip
- Simply run: pip install vdms

* Updated docker images (publicly available at docker.io/intellabs/vdms):
- Very easy to have a VDMS Server running on your system
- Simply run: docker run -it intellabs/vdms:2.0.0

* Bug fixes and performance enhancements

* Updated documentation:
- New VDMS Commands (at github.com/IntelLabs/vdms/wiki)
- PMGD API documentation (at github.com/IntelLabs/pmgd/wiki)

Compatible with PMGD v2.0.0 and VCL v1.0.0.

Compatible with [PMGD v1.0.0](https://github.com/IntelLabs/pmgd/releases) and [VCL v0.1.0](https://github.com/IntelLabs/vcl/releases)