Starrocks

Release date: March 19, 2022

Bug Fixes

The following bugs are fixed:
- The problem of possible data loss due to BE failure (solved by using Batch publish version). [3140](https://github.com/StarRocks/starrocks/issues/3140)
- Some queries may cause memory limit exceeded errors due to inappropriate execution plans.
- The checksum between replicas may be inconsistent in different compaction processes. [3438](https://github.com/StarRocks/starrocks/issues/3438)
- Query may fail in some situation when JSON reorder projection is not processed correctly. [4056](https://github.com/StarRocks/starrocks/pull/4056)

Release date: March 14, 2022

BugFix

The following bugs are fixed:

- In a rolling upgrade from version 1.19 to 2.1, BE nodes stop working because of unmatched chunk sizes beween two versions. [3834](https://github.com/StarRocks/starrocks/issues/3834)
- Loading tasks may fail while StarRocks is updating from version 2.0 to 2.1. [3828](https://github.com/StarRocks/starrocks/issues/3828)
- Query fails when there is no appropriate execution plan for single-tablet table joins. [3854](https://github.com/StarRocks/starrocks/issues/3854)
- A deadlock problem may occur when an FE node collects information to build a global dictionary for low-cardinality optimization. [3839]( https://github.com/StarRocks/starrocks/issues/3839)
- Query fails when BE nodes are in suspended animation due to deadlock.
- BI tools cannot connect to StarRocks when the show variables command fails.[3708](https://github.com/StarRocks/starrocks/issues/3708)

New Features
- [Preview] StarRocks now supports Iceberg external tables.
- [Preview] The pipeline engine is now available. It is a new execution engine designed for multicore scheduling. The query parallelism can be adaptively adjusted without the need to set the parallel_fragment_exec_instance_num parameter. This also improves performance in high concurrency scenarios.
- The CTAS (Create Table As Select) function is supported, making ETL and table creation easier.
- SQL fingerprint is supported. SQL fingerprint is generated in audit.log, which facilitates the location of slow queries.

Improvements
- Compaction is optimized. A flat table can contain up to 10,000 columns.
- The performance of first-time scan and page cache is optimized. Random I/O is reduced to improve first-time scan performance. The improvement is more noticeable if first-time scan occurs on SATA disks. StarRocks' page cache can store original data, which eliminates the need for bitshuffle encoding and unnecessary decoding. This improves the cache hit rate and query efficiency.
- Schema change is supported in the primary key model. You can add, delete, and modify bitmap indexes by using `Alter table`.
- [Preview] The size of a string can be up to 1 MB.
- JSON load performance is optimized. You can load more than 100 MB JSON data in a single file.
- Bitmap index performance is optimized.
- The performance of StarRocks Hive external tables is optimized. Data in the CSV format can be read.
- DEFAULT CURRENT_TIMESTAMP is supported in the create table statement. [1193](https://github.com/StarRocks/starrocks/issues/1193)
- StarRocks supports the loading of CSV files with multiple delimiters.

BugFix
The following bugs are fixed:
- Auto __op mapping does not take effect if jsonpaths is specified in the command used for loading JSON data. [3405](https://github.com/StarRocks/starrocks/issues/3405)
- BE nodes fail because the source data changes during data loading using Broker Load. [3481](https://github.com/StarRocks/starrocks/issues/3481)
- Some SQL statements report errors after materialized views are created. [2975](https://github.com/StarRocks/starrocks/issues/2975)
- The routine load does not work due to quoted jsonpaths. [2488](https://github.com/StarRocks/starrocks/issues/2488)
- Query concurrency decreases sharply when the number of columns to query exceeds 200.

Behavior Changes
- The API for disabling a Colocation Group is changed from DELETE /api/colocate/group_stable to POST /api/colocate/group_unstable.

Others
- flink-connector-starrocks is now available for Flink to read StarRocks data in batches. This improves data read efficiency compared to the JDBC connector.

Release date: August 6, 2022

Bug Fixes
The following bugs are fixed:
- For a Broker Load job, if the broker is heavily loaded, internal heartbeats may time out, causing data loss. [[8282](https://github.com/StarRocks/starrocks/issues/8282)]
- For a Broker Load job, if the destination StarRocks table does not have the column specified by the COLUMNS FROM PATH AS parameter, the BEs stop running. [[5346](https://github.com/StarRocks/starrocks/issues/5346)]
- Some queries are forwarded to the Leader FE, causing the /api/query_detail action to return incorrect execution information about SQL statements such as SHOW FRONTENDS. [[9185](https://github.com/StarRocks/starrocks/issues/9185)]
- When multiple Broker Load jobs are created to load the same HDFS data file, if one job encounters exceptions, the other jobs may not be able to properly read data either and consequently fail. [[9506](https://github.com/StarRocks/starrocks/issues/9506)]

**Full Changelog**: https://github.com/StarRocks/starrocks/compare/2.0.8...2.0.9


Thanks to
dulong41, gengjun-git, liuyehcf, padmejin, trueeyu,

Release date: July 15, 2022

Bug Fixes
The following bugs are fixed:
- Switching Leader FE node repetitively may cause all load jobs hang and fail. [7350](https://github.com/StarRocks/starrocks/issues/7350)
- BE crashes when the memory usage estimation of MemTable exceeds 4GB, because, during a data skew in load, some fields may occupy a large amount of memory resources. [7161](https://github.com/StarRocks/starrocks/issues/7161)
- After restarting FEs, the schemas of materialized views changed due to incorrect parsing of uppercase and lowercase letters. [7362](https://github.com/StarRocks/starrocks/issues/7362)
- When you load JSON data from Kafka into StarRocks by using Routine Load, if there are blank rows in the JSON data, the data after the blank rows will be lost. [8534](https://github.com/StarRocks/starrocks/issues/8534)


**Full Changelog**: https://github.com/StarRocks/starrocks/compare/2.0.7...2.0.8

Thanks to
Astralidea, HangyuanLiu, Seaven, gengjun-git, imay, mergify, padmejin, rickif, stdpain, trueeyu, xiaoyong-z

Release date: June 13, 2022

Bug Fixes
The following bugs are fixed:
- If the number of duplicate values in a column of a table that is being compacted exceeds 0x40000000, the compaction is suspended. [6513](https://github.com/StarRocks/starrocks/issues/6513)
- After an FE restarts, it encounters high I/O and abnormally increasing disk usage due to a few issues in BDB JE v7.3.8 and shows no sign of restoring to normal. The FE is restored to normal after it rolls back to BDB JE v7.3.7. [6634](https://github.com/StarRocks/starrocks/issues/6634)


**Full Changelog**: https://github.com/StarRocks/starrocks/compare/2.0.6...2.0.7

Thanks to
Astralidea, RowenWoo, gengjun-git, meegoo, mergify, xiaoyong-z