Skip to content

milvus-2.1.1
Compare
Choose a tag to compare
@yanliang567 yanliang567 released this 15 Aug 01:20
· 6281 commits to master since this release
v2.1.1
bc62ca1
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Release Notes

Find out what’s new in Milvus! This page summarizes information about new features, improvements, known issues, and bug fixes in each release. You can find the release notes for each released version after v2.1.0 in this section. We suggest that you regularly visit this page to learn about updates.

v2.1.1

Release date: 12 August 2022

Milvus version Python SDK version Java SDK version Go SDK version Node.js SDK version
2.1.1 2.1.1 2.1.0-beta4 2.1.1 2.1.2

Milvus 2.1.1 is a minor bug fix version of Milvus 2.1.0. It fixed Query Node crash under high concurrency, garbage collector failure to parse segment ID issue, and a few other stability issues.

Bug fixes

  • #18383, #18432 Fixed Garbage collector parse segment ID panics with bad input.

  • #18418 Fixed metatable related error when etcd compaction error happens.

  • #18568 Closes Node/Segment detector when closing ShardCluster.

  • #18542 Adds CGO worker pool for Query Node to avoid OpenMP creating too many threads.

  • #18569 Releases collection resources when all partitions are released to avoid resource leakage.

Improvements

  • #18394 Removes watch delta channel task-related logic.

  • #18513 Uses chunkManager rather than minio.Client in Data Node garbage collection.

  • #18410 Limits the concurrency level for single load request

Featuress

  • #18570 Supports dynamic change log level through HTTP.

v2.1.0

Release date: 27 July 2022

Compatibility

Milvus version Python SDK version Java SDK version Go SDK version Node.js SDK version
2.1.0 2.1.0 2.1.0-beta4 2.1.0 2.1.2

Milvus 2.1.0 not only introduces many new features including support for VARCHAR data type, memory replicas, Kafka support, and RESTful API but also greatly improves the functionality, performance, and stability of Milvus.

Features

  • Support for VARCHAR data type

Milvus now supports variable-length string as a scalar data type. Like previous scalar types, VARCHAR can be specified as an output field or be used for attribute filtering. A MARISA-trie-based inverted index is also supported to accelerate prefix query and exact match.

  • In-memory replicas

In-memory replicas enable you to load data on multiple query nodes. Like read replicas in traditional databases, in-memory replicas can help increase throughput if you have a relatively small dataset but want to scale read throughput with more hardware resources. We will support hedged read in future releases to increase availability when applying in-memory replicas.

  • Embedded Milvus

Embedded Milvus enables you to pip install Milvus in one command, try quick demos and run short scripts in Python on your Macbook, including on the ones with M1 processor.

  • Kafka support (Beta)

Apache Kafka is the most widely used open-source distributed message store. In Milvus 2.1.0, you can simply use Kafka for message storage by modifying configurations.

  • RESTful API (Beta)

Milvus 2.1.0 now provides RESTful API for applications written in PHP or Ruby. GIN, one of the most popular Golang web frameworks, is adopted as the web server.

Performance

The Milvus core team conducted a full performance benchmarking and profiling, and fixed a few bottlenecks on load/search paths. Under some test cases, Milvus search performance is boosted about 3.2 times thanks to the search combination logic.

  • #16014 Enables ZSTD compression for pulsar.
  • #16514 #17273 Improves load performance.
  • #17005 Loads binlog for different fields in parallel.
  • #17022 Adds logic for search merging and a simple task scheduler for read tasks.
  • #17194 Simplifies the merge logic of searchTask.
  • #17287 Reduces default seal proportion.

Stability

To improve stability, especially during streaming data insertion, we fixed a few critical issues including:

  • Fixed out of memory issues.
  • Fixed message queue backlog full caused by message queue subscription leakge.
  • Fixed the issue of deleted entities can still be readable.
  • Fixed data being erroneously cleaned by compaction during load or index.

Other improvements

  • Security

Starting from Milvus 2.1.0, we support username, password, and TLS connection. We also enable safe connections to our dependencies such as S3, Kafka and etcd.

  • ANTLR parser

Milvus now adopts Go ANTLR as the plan parser to make adding new grammar such as arithmetic operations on numerical fields more flexible. The adoption of ANTLR also prepares for Milvus query language support in future releases.

  • Observability

We refined monitoring metrics by adding important metrics including search QPS and latency to the new dashboard. Please notify us if any metrics critical to your production environment are not listed.

  • Deployment

For users who don't have a K8s environment but still want to deploy a cluster, Milvus now supports Ansible deployment. See Install Milvus Cluster for more information.

Known issues

  1. Partition is not a fully released feature so we recommend user not to rely on it. #17648 When a partition is dropped, the data and index cannot be cleaned.
  2. When building index after load, the collection need to released and reloaded. #17809 When an index is created on a loaded collection, the segment already loaded will not be notified to load the index.
Assets 6
Loading