Starrocks

Release date: May 25, 2022
Bug Fixes
The following bugs are fixed:
- Some graphical user interface (GUI) tools automatically configure the set_sql_limit variable. As a result, the SQL statement ORDER BY LIMIT is ignored, and consequently an incorrect number of rows are returned for queries. [5966](https://github.com/StarRocks/starrocks/issues/5966)
- If a colocation group (CG) contains a large number of tables and data is frequently loaded into the tables, the CG may not be able to stay in the stable state. In this case, the JOIN statement does not support Colocate Join operations. StarRocks has been optimized to wait for a little longer during data loading. This way, the integrity of the tablet replicas to which data is loaded can be maximized.
- If a few replicas fail to be loaded due to reasons such as heavy loads or high network latencies, cloning on these replicas is triggered. In this case, deadlocks may occur, which may cause a situation in which the loads on processes are low but a large number of requests time out. [5646](https://github.com/StarRocks/starrocks/issues/5646) [#6290](https://github.com/StarRocks/starrocks/issues/6290)
- After the schema of a table that uses the Primary Key model is changed, a "duplicate key xxx" error may occur when data is loaded into that table. [5878](https://github.com/StarRocks/starrocks/issues/5878)
- If the DROP SCHEMA statement is executed on a database, the database is forcibly deleted and cannot be restored. [6201](https://github.com/StarRocks/starrocks/issues/6201)


**Full Changelog**: https://github.com/StarRocks/starrocks/compare/2.0.5...2.0.6

Thanks to
Astralidea, Linkerist, chaoyli, decster, dirtysalt, gengjun-git, sevev, stdpain

Release date: May 13, 2022
Upgrade recommendation: Some critical bugs related to the correctness of stored data or data queries have been fixed in this version. It is recommended that you upgrade your StarRocks cluster in time.

Bug Fixes
The following bugs are fixed:
- [Critical Bug] Data may be lost as a result of BE failures. This bug is fixed by introducing a mechanism that is used to publish a specific version to multiple BEs at a time. [3140](https://github.com/StarRocks/starrocks/issues/3140)
- [Critical Bug] If tablets are migrated in specific data ingestion phases, data continues to be written to the original disk on which the tablets are stored. As a result, data is lost, and queries cannot be run properly. [5160](https://github.com/StarRocks/starrocks/issues/5160)
- [Critical Bug] When you run queries after you perform multiple DELETE operations, you may obtain incorrect query results if optimization on low-cardinality columns is performed for the queries. [5712](https://github.com/StarRocks/starrocks/issues/5712)
- [Critical Bug] If a query contains a JOIN clause that is used to combine a column with DOUBLE values and a column with VARCHAR values, the query result may be incorrect. [5809](https://github.com/StarRocks/starrocks/pull/5809)
- In certain circumstances, when you load data into your StarRocks cluster, some replicas of specific versions are marked as valid by the FEs before taking effect. At this time, if you query data of the specific versions, StarRocks cannot find the data and reports errors. [5153](https://github.com/StarRocks/starrocks/issues/5153)
- If a parameter in the SPLIT function is set to NULL, the BEs of your StarRocks cluster may stop running. [4092](https://github.com/StarRocks/starrocks/issues/4092)
- After your cluster is upgraded from Apache Doris 0.13 to StarRocks 1.19.x and keeps running for a period of time, a further upgrade to StarRocks 2.0.1 may fail. [5309](https://github.com/StarRocks/starrocks/issues/5309)

Thanks to:
ABingHuang, Astralidea, HangyuanLiu, Pslydhh, Seaven, Youngwb, adzfolc, decster, gengjun-git, kangkaisen, mergify, miomiocat, mofeiatwork, rickif, satanson, sevev, stdpain

Release date: April 18, 2022

Bug Fixes
The following bugs are fixed:
- After deleting columns, adding new partitions, and cloning tablets, the columns' unique ids in old and new tablets may not be the same, which may cause BE to stop working because the system uses a shared tablet schema. [4514](https://github.com/StarRocks/starrocks/issues/4514)
- When data is loading to a StarRocks external table, if the configured FE of the target StarRocks cluster is not a Leader, it will cause the FE to stop working. [4573](https://github.com/StarRocks/starrocks/issues/4573)
- Query results may be incorrect, when a Duplicate Key table performs schema change and creates materialized view at the same time. [4839](https://github.com/StarRocks/starrocks/issues/4839)
- The problem of possible data loss due to BE failure (solved by using Batch publish version). [3140](https://github.com/StarRocks/starrocks/issues/3140)

Release date: March 14, 2022

BugFix

The following bugs are fixed:

- Query fails when BE nodes are in suspended animation.
- Query fails when there is no appropriate execution plan for single-tablet table joins. [3854](https://github.com/StarRocks/starrocks/issues/3854)
- A deadlock problem may occur when an FE node collects information to build a global dictionary for low-cardinality optimization. [3839](https://github.com/StarRocks/starrocks/issues/3839)

**Improvement**
- Memory usage is optimized. Users can specify the label_keep_max_num parameter to control the maximum number of loading jobs to retain within a period of time. This prevents full GC caused by high memory usage of FE during frequent data loading. [2410](https://github.com/StarRocks/starrocks/issues/2410)

**BugFix**
The following bugs are fixed:
- BE nodes fail when the column decoder encounters an exception. [3510](https://github.com/StarRocks/starrocks/issues/3510)
- Auto __op mapping does not take effect when jsonpaths is specified in the command used for loading JSON data. [3405](https://github.com/StarRocks/starrocks/issues/3405)
- BE nodes fail because the source data changes during data loading using Broker Load. [3481](https://github.com/StarRocks/starrocks/issues/3481)
- Some SQL statements report errors after materialized views are created. [3053](https://github.com/StarRocks/starrocks/issues/3053)
- Query may fail if an SQL clause contains a predicate that supports global dictionary for low-cardinality optimization and a predicate that does not. [3421](https://github.com/StarRocks/starrocks/issues/3421)

Release date: Jan 21, 2022

Improvement
* Add implicit_cast in hive external table (2829)
* Use read/write lock avoid cpu cost too much when collect metrics(2901)
* optimize some statistics.

Bugfix
* Fix Global dictionary when replica is inconsistent. (2700) (2765)
* Add exec_mem_limit for stream load (2693)
* Fix OOM when loading into primary key model. (2743)(2777)
* Fix BE hang when query external mysql table (2881)

2.0.0-GA
Release date: Jan 4, 2022

New Feature
- External Table
- [Experimental Function]Support for Hive external table on S3
- DecimalV3 support for external table 425
- Implement complex expressions to be pushed down to the storage layer for computation, thus gaining performance gains
- Primary Key is officially released, which supports Stream Load, Broker Load, Routine Load, and also provides a second-level synchronization tool for MySQL data based on Flink-cdc

Improvement
- Arithmetic operators optimization
- Optimize the performance of dictionary with low cardinality 791
- Optimize the scan performance of int for single table [273](https://github.com/StarRocks/starrocks/issues/273)
- Optimize the performance of count(distinct int) with high cardinality [139](https://github.com/StarRocks/starrocks/pull/139) [#250](https://github.com/StarRocks/starrocks/pull/250) [#544](https://github.com/StarRocks/starrocks/pull/544)[#570](https://github.com/StarRocks/starrocks/pull/570)
- Execution level optimization and refinement Group by 2 int / limit / case when / not equal
- Optimize Group by 2 int / limit / case when / not equal in implementation-level
- Memory management optimization
- Refactor the memory statistics and control framework to accurately count memory usage and completely solve OOM
- Optimize metadata memory usage
- Solve the problem of large memory release stuck in execution threads for a long time
- Add process graceful exit mechanism and support memory leak check 1093

Bugfix
- Fix the problem that the Hive external table is timeout to get metadata in a large amount.
- Fix the problem of unclear error message of materialized view creation.
- Fix the implementation of like in vectorization engine 722
- Repair the error of parsing the predicate is in alter table 725
- Fix the problem that the curdate function can not format the date.