Milvus

Milvus 2.2.9 has added JSON support, allowing for more flexible schemas within collections through dynamic schemas. The search efficiency has been improved through partition keys, which enable data separation for different data categories, such as multiple users, in a single collection. Additionally, database support has been integrated into Role-Based Access Control (RBAC), further fortifying multi-tenancy management and security. Support has also been extended to Alibaba Cloud OSS, and connection management has been refined, resulting in an improved user experience.

As always, this release includes bug fixes, enhancements, and performance improvements. Notably, disk usage has been significantly reduced, and performance has been improved, particularly for filtered searches.

We hope you enjoy the latest release!

Breaking changes
For user's deploy with minio standalone, please be aware of that Milvus standalone data is not compatible due to https://min.io/docs/minio/linux/operations/install-deploy-manage/migrate-fs-gateway.html. You have to manualy migrate the data to new Minio instance before upgrade.

New Features

- JSON support

- Introduced JSON data type ([23839](https://github.com/milvus-io/milvus/pull/23839)).
- Added support for expressions with JSON fields ([23804](https://github.com/milvus-io/milvus/pull/23804), [#24016](https://github.com/milvus-io/milvus/pull/24016)).
- Enabled JSON support for bulk insert operations ([24227](https://github.com/milvus-io/milvus/pull/24227)).
- Enhanced performance of filters using JSON fields ([24268](https://github.com/milvus-io/milvus/pull/24268), [#24282](https://github.com/milvus-io/milvus/pull/24282)).

- Dynamic schema

- Added dynamic schema support ([24062](https://github.com/milvus-io/milvus/pull/24062), [#24176](https://github.com/milvus-io/milvus/pull/24176), [#24205](https://github.com/milvus-io/milvus/pull/24205), [#24099](https://github.com/milvus-io/milvus/pull/24099)).
- Enabled dynamic fields in bulk insert operations ([24265](https://github.com/milvus-io/milvus/pull/24265)).

- Partition key

- Introduced partition key ([23994](https://github.com/milvus-io/milvus/pull/23994)).
- Added support for imports when partition key is enabled and backup is present ([24454](https://github.com/milvus-io/milvus/pull/24454)).
- Added unit tests for partition key ([24167](https://github.com/milvus-io/milvus/pull/24167)).
- Resolved issue with bulk insert not supporting partition key ([24328](https://github.com/milvus-io/milvus/pull/24328)).

- Database support in RBAC

- Added database support in Role-Based Access Control (RBAC) ([23742](https://github.com/milvus-io/milvus/pull/23742)).
- Resolved non-existent database error for FlushAll function ([24222](https://github.com/milvus-io/milvus/pull/24222)).
- Implemented default database value for RBAC requests ([24307](https://github.com/milvus-io/milvus/pull/24307)).
- Ensured backward compatibility with empty database name ([24317](https://github.com/milvus-io/milvus/pull/24317)).

- Connection management

- Implemented the connect API to manage connections ([24224](https://github.com/milvus-io/milvus/pull/24224)) ([#24293](https://github.com/milvus-io/milvus/pull/24293))
- Implemented checks if a database exists when Connect was called ([24399](https://github.com/milvus-io/milvus/pull/24399))

- Alibaba Cloud OSS support

- Added support for Aliyun OSS in chunk manager ([22663](https://github.com/milvus-io/milvus/pull/22663), [#22842](https://github.com/milvus-io/milvus/pull/22842), [#23956](https://github.com/milvus-io/milvus/pull/23956)).
- Enabled Alibaba Cloud OSS as object storage using access key (AK) or Identity and Access Management (IAM) ([23949](https://github.com/milvus-io/milvus/pull/23949)).

- Additional features

- Implemented AutoIndex ([24387](https://github.com/milvus-io/milvus/pull/24387), [#24443](https://github.com/milvus-io/milvus/pull/24443)).
- Added configurable policy for query node and user-level schedule policy ([23718](https://github.com/milvus-io/milvus/pull/23718)).
- Implemented rate limit based on growing segment size ([24157](https://github.com/milvus-io/milvus/pull/24157)).
- Added support for single quotes within string expressions ([24386](https://github.com/milvus-io/milvus/pull/24386), [#24406](https://github.com/milvus-io/milvus/pull/24406)).

Bug fixes

- Added temporary disk data cleaning upon the start of Milvus ([24400](https://github.com/milvus-io/milvus/pull/24400)).
- Fixed crash issue of bulk insert caused by an invalid Numpy array file ([24480](https://github.com/milvus-io/milvus/pull/24480)).
- Fixed an empty result set type for Int8~Int32 ([23851](https://github.com/milvus-io/milvus/pull/23851)).
- Fixed the panic that occurs while balancing releasing a collection ([24003](https://github.com/milvus-io/milvus/pull/24003)) ([#24070](https://github.com/milvus-io/milvus/pull/24070)).
- Fixed an error that occurs when a role removes a user that has already been deleted ([24049](https://github.com/milvus-io/milvus/pull/24049)).
- Fixed an issue where session stop/goingStop becomes stuck after a lost connection ([23771](https://github.com/milvus-io/milvus/pull/23771)).
- Fixed the panic caused by incorrect logic of getting unindexed segments ([24061](https://github.com/milvus-io/milvus/pull/24061)).
- Fixed the panic that occurs when a collection does not exist in quota effect ([24321](https://github.com/milvus-io/milvus/pull/24321)).
- Fixed an issue where refresh may be notified as finished early ([24438](https://github.com/milvus-io/milvus/pull/24438)) ([#24466](https://github.com/milvus-io/milvus/pull/24466)).

Enhancement

- Added an error response to return when an unimplemented request is received ([24546](https://github.com/milvus-io/milvus/pull/24546))
- Reduced disk usage for Milvus Lite and Standalone:

- Refine RocksDB option ([24394](https://github.com/milvus-io/milvus/pull/24394))
- Fix RocksMQ retention not triggering at DataCoord timetick channel ([24134](https://github.com/milvus-io/milvus/pull/24134))

- Optimized quota to avoid OOM on search
- Added consistency_level in search/query request ([24541](https://github.com/milvus-io/milvus/pull/24541))
- (pr24562) Supported search with default parameters ([24516](https://github.com/milvus-io/milvus/pull/24516))
- Put DataNode load statslog lazy if SkipBFStatsLog is true ([23779](https://github.com/milvus-io/milvus/pull/23779))
- Put QueryNode lazy load statslog if SkipBFLoad is true ([23904](https://github.com/milvus-io/milvus/pull/23904))
- Fixed concurrent map read/write in rate limiter ([23957](https://github.com/milvus-io/milvus/pull/23957))
- Improved load/release performance:

- Implemented more frequent CollectionObserver checks to trigger during load procedure ([23925](https://github.com/milvus-io/milvus/pull/23925))
- Implemented checks to trigger while waiting for collection/partition to be released ([24535](https://github.com/milvus-io/milvus/pull/24535))

- Optimized PrivilegeAll permission check ([23972](https://github.com/milvus-io/milvus/pull/23972))
- Fixed the "not shard leader" error when gracefully stopping ([24038](https://github.com/milvus-io/milvus/pull/24038))
- Checked the overflow for inserted integer ([24142](https://github.com/milvus-io/milvus/pull/24142)) ([#24172](https://github.com/milvus-io/milvus/pull/24172))
- Lowered the task merge cap to mitigate an insufficient memory error ([24233](https://github.com/milvus-io/milvus/pull/24233))
- Removed constraint that prevents creating an index after load ([24415](https://github.com/milvus-io/milvus/pull/24415))
- Removed index check to trigger compaction ([23657](https://github.com/milvus-io/milvus/pull/23657)) ([#23688](https://github.com/milvus-io/milvus/pull/23688))

Performance improvements

- [Enhancement]: Optimize the search performance with high filtering ratio ([23948](https://github.com/milvus-io/milvus/pull/23948)).
- Added SIMD support for several filtering expressions ([23715](https://github.com/milvus-io/milvus/pull/23715), [#23781](https://github.com/milvus-io/milvus/pull/23781)).
- Reduced data copying during insertion into growing segments ([24492](https://github.com/milvus-io/milvus/pull/24492)).

Release date: 2 May, 2023

| Milvus version | Python SDK version | Java SDK version | Go SDK version | Node.js SDK version |
| -------------- | ------------------ | ---------------- | -------------- | ------------------- |

In this update, we have focused on resolving various issues reported by our users, enhancing the software's overall stability and functionality. Additionally, we have implemented several optimizations, such as load balancing, search grouping, and memory usage improvements.

Bugfix

- Fixed a panic caused by not removing metadata of a dropped segment from the DataNode. ([23492](https://github.com/milvus-io/milvus/pull/23492))
- Fixed a bug that caused forever blocking due to the release of a non-loaded partition. ([23612](https://github.com/milvus-io/milvus/pull/23612))
- To prevent the query service from becoming unavailable, automatic balancing at the channel level has been disabled as a workaround. ([23632](https://github.com/milvus-io/milvus/pull/23632)) ([#23724](https://github.com/milvus-io/milvus/pull/23724))
- Cancel failed tasks in the scheduling queue promptly to prevent an increase in QueryCoord scheduling latency. ([23649](https://github.com/milvus-io/milvus/pull/23649))
- Fixed compatibility bug and recalculate segment rows to prevent service queries from being unavailable. ([23696](https://github.com/milvus-io/milvus/pull/23696))
- Fixed a bug in the superuser password validation logic. ([23729](https://github.com/milvus-io/milvus/pull/23729))
- Fixed the issue of shard detector rewatch failure, which was caused by returning a closed channel. ([23734](https://github.com/milvus-io/milvus/pull/23734))
- Fixed a loading failure caused by unhandled interrupts in the AWS SDK. ([23736](https://github.com/milvus-io/milvus/pull/23736))
- Fixed the "HasCollection" check in DataCoord. ([23709](https://github.com/milvus-io/milvus/pull/23709))
- Fixed the bug that assigned all available nodes to a single replica incorrectly. ([23626](https://github.com/milvus-io/milvus/pull/23626))

Enhancement

- Optimized the display of RootCoord histogram metrics. ([23567](https://github.com/milvus-io/milvus/pull/23567))
- Reduced peak memory consumption during collection loading. ([23138](https://github.com/milvus-io/milvus/pull/23138))
- Removed unnecessary handoff event-related metadata. ([23565](https://github.com/milvus-io/milvus/pull/23565))
- Added a plugin logic to QueryNode to support the dynamic loading of shared library files. ([23599](https://github.com/milvus-io/milvus/pull/23599))
- Supports load balancing with replica granularity. ([23629](https://github.com/milvus-io/milvus/pull/23629))
- Released a load-balancing strategy based on scores. ([23805](https://github.com/milvus-io/milvus/pull/23805))
- Added a coroutine pool to limit the concurrency of cgo calls triggered by "delete". ([23680](https://github.com/milvus-io/milvus/pull/23680))
- Improved the compaction algorithm to make the distribution of segment sizes tend towards the ideal value. ([23692](https://github.com/milvus-io/milvus/pull/23692))
- Changed the default shard number to 1. ([23593](https://github.com/milvus-io/milvus/pull/23593))
- Improved search grouping algorithm to enhance throughput. ([23721](https://github.com/milvus-io/milvus/pull/23721))
- Code refactoring: Separated the read, build, and load DiskANN parameters. ([23722](https://github.com/milvus-io/milvus/pull/23722))
- Updated etcd and Minio versions. ([23765](https://github.com/milvus-io/milvus/pull/23765))

**Upgrade to Milvus 2.2.6 as soon as possible!**

You are advised to refrain from using version 2.2.5 due to several critical issues that require immediate attention. Version 2.2.6 addresses these issues. One of the critical issues is the inability to recycle dirty binlog data. We highly recommend using version 2.2.6 version instead of version 2.2.5 to avoid any potential complications.

If you hit the issue where data on object storage cannot be recycled, upgrade your Milvus to v2.2.6 to fix these issues.

Bugfix

- Fixed the problem of DataCoord GC failure ([23298](https://github.com/milvus-io/milvus/pull/23298))
- Fixed the problem that index parameters passed when creating a collection will override those passed in subsequent create_index operations ([23242](https://github.com/milvus-io/milvus/pull/23242))
- Fix the problem that the message backlog occurs in RootCoord, which causes the delay of the whole system to increase ([23267](https://github.com/milvus-io/milvus/pull/23267))
- Fixed the accuracy of metric RootCoordInsertChannelTimeTick ([23284](https://github.com/milvus-io/milvus/pull/23284))
- Fixed the issue that the timestamp reported by the proxy may stop in some cases ([23291](https://github.com/milvus-io/milvus/pull/23291))
- Fixed the problem that the coordinator role may self-destruct by mistake during the restart process ([23344](https://github.com/milvus-io/milvus/pull/23344))
- Fixed the problem that the checkpoint is left behind due to the abnormal exit of the garbage collection goroutine caused by the etcd restart ([23401](https://github.com/milvus-io/milvus/pull/23401))

Enhancement

- Added slow logging performance for query/search when the latency is not less than 5 seconds ([23274](https://github.com/milvus-io/milvus/pull/23274))

Security

Fixed MinIO CVE-2023-28432 by upgrading MinIO to RELEASE.2023-03-20T20-16-18Z。

New Features

- **First/Random replica selection policy**

This policy allows for a random replica selection if the first replica chosen under the round-robin selection policy fails. This improves the throughput of database operations.

Bug fixes

- Fixed index data loss during the upgrade from Milvus 2.2.0 to 2.2.3.

- Fixed an issue to prevent DataCoord from calculating segment lines by stale log entries num (23069)
- Fixed DataCoord's meta that may be broken with DataNode of the prior version (23031)

- Fixed DataCoord Out-of-Memory (OOM) with large fresh pressure.

- Fixed an issue to make DataNode's tt interval configurable (22990)
- Fixed endless appending SIDs (22989)

- Fixed a concurrency issue in the LRU cache that was caused by concurrent queries with specified output fields.

- Fixed an issue to use single-flight to limit the readWithCache concurrent operation (23037)
- Fixed LRU cache concurrency (23041)
- Fixed query performance issue with a large number of segments (23028)
- Fixed shard leader cache
- Fixed GetShardLeader returns old leader (22887) (22903)
- Fixed an issue to deprecate the shard cache immediately if a query failed (22848)
- Fixed an issue to enable batch delete files on GCP of MinIO (23052) (23083)
- Fixed flush delta buffer if SegmentID equals 0 (23064)
- fixed unassigned from resource group (22800)
- Fixed load partition timeout logic still using createdAt (23022)
- Fixed unsub channel always removes QueryShard (22961)

Enhancements

- Added memory Protection by using the buffer size in memory synchronization policy (22797)
- Added dimension checks upon inserted records (22819) (22826)
- Added configuration item to disable BF load (22998)
- Aligned the maximum dimensions of the DisANN index and that of a collection (23027)
- Added checks whether all columns aligned with same num_rows (22968) (22981)
- Upgraded Knowhere to 1.3.11 (22975)
- Added the user RPC counter (22870)

In addition to these features, Milvus 2.2.4 also addresses several issues related to rolling upgrades, failure recovery, and load balancing. These bug fixes contribute to a more stable and reliable system.

We have also made several enhancements to make your Milvus cluster faster and consume less memory with reduced convergence time for failure recovery.

New Features

- Resource grouping

Milvus has implemented resource grouping for QueryNodes. A resource group is a collection of QueryNodes. Milvus supports grouping QueryNodes in the cluster into different resource groups, where access to physical resources in different resource groups is completely isolated. See [Manage Resource Group](resource_group.md) for more information.

- Collection renaming

The Collection-renaming API provides a way for users to change the name of a collection. Currently, PyMilvus supports this API, and SDKs for other programming languages are on the way. See [Rename a Collection](rename_collection.md) for details.

- Google Cloud Storage support

Milvus now supports Google Cloud Storage as the object storage.

- New option to the search and query APIs

If you are more concerned with performance rather than data freshness, enabling this option will skip search on all growing segments and offer better search performance under the scenario search with insertion. See [search(/api-reference/pymilvus/v2.2.3/Collection/search().md)]() and [query()]() for details.

Bugfix

- Fixed segment not found when forwarding delete to empty segment [22528](https://github.com/milvus-io/milvus/pull/22528)[#22551](https://github.com/milvus-io/milvus/pull/22551)
- Fixed possible broken channel checkpoint in v2.2.2 [22205](https://github.com/milvus-io/milvus/pull/22205) [#22227](https://github.com/milvus-io/milvus/pull/22227)
- Fixed entity number mismatch with some entities inserted [22306](https://github.com/milvus-io/milvus/pull/22306)
- Fixed DiskANN recovery failure after QueryNode reboots [22488](https://github.com/milvus-io/milvus/pull/22488) [#22514](https://github.com/milvus-io/milvus/pull/22514)
- Fixed search/release on same segment [22414](https://github.com/milvus-io/milvus/pull/22414)
- Fixed file system crash during bulk-loading files prefixed with a '.' [22215](https://github.com/milvus-io/milvus/pull/22215)
- Added tickle for DataCoord watch event [21193](https://github.com/milvus-io/milvus/pull/21193) [#22209](https://github.com/milvus-io/milvus/pull/22209)
- Fixed deadlock when releasing segments and removing nodes concurrently [22584](https://github.com/milvus-io/milvus/pull/22584)
- Added channel balancer on DataCoord [22324](https://github.com/milvus-io/milvus/pull/22324) [#22377](https://github.com/milvus-io/milvus/pull/22377)
- Fixed balance generated reduce task [22236](https://github.com/milvus-io/milvus/pull/22236) [#22326](https://github.com/milvus-io/milvus/pull/22326)
- Fixed QueryCoord panic caused by balancing [22486](https://github.com/milvus-io/milvus/pull/22486)
- Added scripts for rolling update Milvus's component installed with helm [22124](https://github.com/milvus-io/milvus/pull/22124)
- Added `NotFoundTSafer` and `NoReplicaAvailable` to retriable error code [22505](https://github.com/milvus-io/milvus/pull/22505)
- Fixed no retires upon gRPC error [22529](https://github.com/milvus-io/milvus/pull/22529)
- Fixed an issue for automatic component state update to healthy after start [22084](https://github.com/milvus-io/milvus/pull/22084)
- Added graceful-stop for sessions [22386](https://github.com/milvus-io/milvus/pull/22386)
- Added retry op for all servers [22274](https://github.com/milvus-io/milvus/pull/22274)
- Fixed metrics info panic when network error happens [22802](https://github.com/milvus-io/milvus/pull/22802)
- Fixed disordered minimum timestamp in proxy's pchan statistics [22756](https://github.com/milvus-io/milvus/pull/22756)
- Fixed an issue to ensure segment ID recovery upon failures to send time-tick [22771](https://github.com/milvus-io/milvus/pull/22771)
- Added segment info retrieval without the binlog path [22741](https://github.com/milvus-io/milvus/pull/22741)
- Added distribution.Peek for GetDataDistribution in case of blocked by release [22752](https://github.com/milvus-io/milvus/pull/22752)
- Fixed the segment not found error [22739](https://github.com/milvus-io/milvus/pull/22739)
- Reset delta position to vchannel in packSegmentLoadReq [22721](https://github.com/milvus-io/milvus/pull/22721)
- Added vector float data verification for bulkinsert and insert [22729](https://github.com/milvus-io/milvus/pull/22729)
- Upgraded Knowhere to 1.3.10 to fix bugs [22746](https://github.com/milvus-io/milvus/pull/22746)
- Fixed RootCoord double updates TSO [22715](https://github.com/milvus-io/milvus/pull/22715) [#22723](https://github.com/milvus-io/milvus/pull/22723)
- Fixed confused time-tick logs [22733](https://github.com/milvus-io/milvus/pull/22733) [#22734](https://github.com/milvus-io/milvus/pull/22734)
- Fixed session nil point [22696](https://github.com/milvus-io/milvus/pull/22696)
- Upgraded Knowhere to 1.3.10 [22614](https://github.com/milvus-io/milvus/pull/22614)
- Fixed incorrect sequence of timetick statistics on proxy[21855](https://github.com/milvus-io/milvus/pull/21855) [#22560](https://github.com/milvus-io/milvus/pull/22560)
- Enabled DataCoord to handle GetIndexedSegment error from IndexCoord [22673](https://github.com/milvus-io/milvus/pull/22673)
- Fixed an issue for Milvus writes flushed segment key only after the segment is flushed [22667](https://github.com/milvus-io/milvus/pull/22667)
- Marked cache deprecated instead of removing it [22675](https://github.com/milvus-io/milvus/pull/22675)
- Updated shard leader cache [22632](https://github.com/milvus-io/milvus/pull/22632)
- Fixed an issue for the replica observer to assign node [22635](https://github.com/milvus-io/milvus/pull/22635)
- Fixed the not found issue when retrieving collection creation timestamp [22629](https://github.com/milvus-io/milvus/pull/22629) [#22634](https://github.com/milvus-io/milvus/pull/22634)
- Fixed time-tick running backwards during DDLs [22617](https://github.com/milvus-io/milvus/pull/22617) [#22618](https://github.com/milvus-io/milvus/pull/22618)
- Fixed max collection name case [22601](https://github.com/milvus-io/milvus/pull/22601)
- Fixed DataNode tickle not run default [22622](https://github.com/milvus-io/milvus/pull/22622)-
- Fixed DataCoord panic while reading timestamp of an empty segment [22598](https://github.com/milvus-io/milvus/pull/22598)
- Added scripts to get etcd info [22589](https://github.com/milvus-io/milvus/pull/22589)
- Fixed concurrent loading timeout during DiskANN indexing [22548](https://github.com/milvus-io/milvus/pull/22548)
- Fixed an issue to ensure index file not finish early because of compaction [22509](https://github.com/milvus-io/milvus/pull/22509)
- Added `MultiQueryNodes` tag for resource group [22527](https://github.com/milvus-io/milvus/pull/22527) [#22544](https://github.com/milvus-io/milvus/pull/22544)

Enhancement

- **Performance**

- Improved query performance by avoiding counting all bits [21909](https://github.com/milvus-io/milvus/pull/21909) [#22285](https://github.com/milvus-io/milvus/pull/22285)
- Fixed dual copy of varchar fields while loading [22114](https://github.com/milvus-io/milvus/pull/22114) [#22291](https://github.com/milvus-io/milvus/pull/22291)
- Updated DataCoord compaction panic after DataNode update plan to ensure consistency [22143](https://github.com/milvus-io/milvus/pull/22143) [#22329](https://github.com/milvus-io/milvus/pull/22329)
- Improved search performance by avoiding allocating a zero-byte vector during searches [22219](https://github.com/milvus-io/milvus/pull/22219) [#22357](https://github.com/milvus-io/milvus/pull/22357)
- Upgraded Knowhere to 1.3.9 to accelerate IVF/BF [22368](https://github.com/milvus-io/milvus/pull/22368)
- Improved search task merge policy [22006](https://github.com/milvus-io/milvus/pull/22006) [#22287](https://github.com/milvus-io/milvus/pull/22287)
- Refined Read method of MinioChunkManager to reduce IO[22257](https://github.com/milvus-io/milvus/pull/22257)

- **Memory Usage**

- Saved index files by 16m to save memory usage while indexing [22369](https://github.com/milvus-io/milvus/pull/22369)
- Added memory usage too large sync policy [22241](https://github.com/milvus-io/milvus/pull/22241)

- **Others**

- Removed constraints that compaction happens only on indexed segment [22145](https://github.com/milvus-io/milvus/pull/22145)
- Changed RocksMQ page size to 256M to reduce RocksMQ disk usage [22433](https://github.com/milvus-io/milvus/pull/22433)
- Changed the etcd session timeout to 20s to improve recovery speed[22400](https://github.com/milvus-io/milvus/pull/22400)
- Added the RBAC for the GetLoadingProgress and GetLoadState API [22313](https://github.com/milvus-io/milvus/pull/22313)