Starrocks

Release date: December 18, 2023

New Features
- Added the [now(p)](https://docs.starrocks.io/zh/docs/3.2/sql-reference/sql-functions/date-time-functions/now/) function to return the current date and time with the specified fractional seconds precision (accurate to the microsecond). If p is not specified, this function returns only date and time accurate to the second. https://github.com/StarRocks/starrocks/pull/36676
- Added a new metric max_tablet_rowset_num for setting the maximum allowed number of rowsets. This metric helps detect possible compaction issues and thus reduces the occurrences of the error "too many versions". https://github.com/StarRocks/starrocks/pull/36539
Supports obtaining heap profiles by using a command line tool, making troubleshooting easier.https://github.com/StarRocks/starrocks/pull/35322
- Supports creating asynchronous materialized views with common table expressions (CTEs). https://github.com/StarRocks/starrocks/pull/36142
- Added the following bitmap functions: [subdivide_bitmap](https://docs.starrocks.io/zh/docs/sql-reference/sql-functions/bitmap-functions/subdivide_bitmap/), [bitmap_from_binary](https://docs.starrocks.io/zh/docs/sql-reference/sql-functions/bitmap-functions/bitmap_from_binary/), and [bitmap_to_binary](https://docs.starrocks.io/zh/docs/sql-reference/sql-functions/bitmap-functions/bitmap_to_binary/). https://github.com/StarRocks/starrocks/pull/35817 https://github.com/StarRocks/starrocks/pull/35621
Parameter Changes
- The FE dynamic parameter enable_new_publish_mechanism is changed to a static parameter. You must restart the FE after you modify the parameter settings. https://github.com/StarRocks/starrocks/pull/35338
- The default retention period of trash files is changed to 1 day from the original 3 days. https://github.com/StarRocks/starrocks/pull/37113
- A new [FE configuration item](https://docs.starrocks.io/zh/docs/administration/Configuration/#%E9%85%8D%E7%BD%AE-fe-%E5%8A%A8%E6%80%81%E5%8F%82%E6%95%B0) routine_load_unstable_threshold_second is added. https://github.com/StarRocks/starrocks/pull/36222
- A new BE configuration item enable_stream_load_verbose_log is added. The default value is false. With this parameter set to true, StarRocks can record the HTTP requests and responses for Stream Load jobs, making troubleshooting easier. https://github.com/StarRocks/starrocks/pull/36113
- A new BE configuration item enable_lazy_delta_column_compaction is added. The default value is true, indicating that StarRocks does not perform frequent compaction operations on delta columns. https://github.com/StarRocks/starrocks/pull/36654
Improvements
- A new value option GROUP_CONCAT_LEGACY is added to the session variable [sql_mode](https://docs.starrocks.io/zh/docs/reference/System_variable/#sql_mode) to provide compatibility with the implementation logic of the [group_concat](https://docs.starrocks.io/zh/docs/sql-reference/sql-functions/string-functions/group_concat/) function in versions earlier than v2.5. https://github.com/StarRocks/starrocks/pull/36150
- The Primary Key table size returned by the [SHOW DATA](https://docs.starrocks.io/zh/docs/sql-reference/sql-statements/data-manipulation/SHOW_DATA/) statement includes the sizes of .cols files (these are files related to partial column updates and generated columns) and persistent index files. https://github.com/StarRocks/starrocks/pull/34898
- Queries on MySQL external tables and the external tables within JDBC catalogs support including keywords in the WHERE clause. https://github.com/StarRocks/starrocks/pull/35917
- Plugin loading failures will no longer cause an error or cause an FE start failure. Instead, the FE can properly start, and the error status of the plug-in can be queried using [SHOW PLUGINS](https://docs.starrocks.io/docs/sql-reference/sql-statements/Administration/SHOW_PLUGINS/). https://github.com/StarRocks/starrocks/pull/36566
- Dynamic partitioning supports random distribution. https://github.com/StarRocks/starrocks/pull/35513
- The result returned by the [SHOW ROUTINE LOAD](https://docs.starrocks.io/zh/docs/sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD/) statement now includes the timestamps of consumption messages from each partition. https://github.com/StarRocks/starrocks/pull/36222
- The result returned by the [SHOW ROUTINE LOAD](https://docs.starrocks.io/zh/docs/sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD/) statement provides a new field OtherMsg, which shows information about the last failed task. https://github.com/StarRocks/starrocks/pull/35806
- The authentication information aws.s3.access_key and aws.s3.access_secret for AWS S3 in Broker Load jobs are hidden in audit logs. https://github.com/StarRocks/starrocks/pull/36571
- The be_tablets view in the information_schema database provides a new field INDEX_DISK, which records the disk usage (measured in bytes) of persistent indexes https://github.com/StarRocks/starrocks/pull/35615
Bug Fixes
Fixed the following issues:

- The BEs crash if users create persistent indexes in the event of data corruption. https://github.com/StarRocks/starrocks/pull/30841
- If users create an asynchronous materialized view that contains nested queries, the error "resolve partition column failed" is reported. https://github.com/StarRocks/starrocks/issues/26078
- If users create an asynchronous materialized view on a base table whose data is corrupted, the error "Unexpected exception: null" is reported. https://github.com/StarRocks/starrocks/pull/30038
- If users run a query that contains a window function, the SQL error "[1064] [42000]: Row count of const column reach limit: 4294967296" is reported. https://github.com/StarRocks/starrocks/pull/33561
- The FE performance plunges after the FE configuration item enable_collect_query_detail_info is set to true. https://github.com/StarRocks/starrocks/pull/35945
- In the StarRocks shared-data mode, the error "Reduce your request rate" may be reported when users attempt to delete files from object storage. https://github.com/StarRocks/starrocks/pull/35566
- Deadlocks may occur when users refresh materialized views. https://github.com/StarRocks/starrocks/pull/35736
- After the DISTINCT window operator pushdown feature is enabled, errors are reported if SELECT DISTINCT operations are performed on the complex expressions of the columns computed by window functions. https://github.com/StarRocks/starrocks/pull/36357
- The BEs crash if the source data file is in ORC format and contains nested arrays. https://github.com/StarRocks/starrocks/pull/36127
Some S3-compatible object storage returns duplicate files, causing the BEs to crash. https://github.com/StarRocks/starrocks/pull/36103

Release date: November 28, 2023

New features
- The CN nodes of a StarRocks shared-data cluster now support data export. https://github.com/StarRocks/starrocks/pull/34018
Improvements
- The [COLUMNS](https://github.com/StarRocks/starrocks/blob/main/docs/en/reference/information_schema/columns.md) view in the system database INFORMATION_SCHEMA can display ARRAY, MAP, and STRUCT columns. https://github.com/StarRocks/starrocks/pull/33431
- Supports queries against Parquet, ORC, and CSV formatted files that are compressed by using LZO and stored in [Hive](https://github.com/StarRocks/starrocks/blob/main/docs/en/data_source/catalog/hive_catalog.md). https://github.com/StarRocks/starrocks/pull/30923 https://github.com/StarRocks/starrocks/pull/30721
- Supports updates onto the specified partitions of an automatically partitioned table. If the specified partitions do not exist, an error is returned. https://github.com/StarRocks/starrocks/pull/34777
- Supports automatic refresh of materialized views when Swap, Drop, or Schema Change operations are performed on the tables and views (including the other tables and materialized views associated with these views) on which these materialized views are created. https://github.com/StarRocks/starrocks/pull/32829
- Optimized the performance of some Bitmap-related operations, including:
- Optimized nested loop joins. https://github.com/StarRocks/starrocks/pull/34804 https://github.com/StarRocks/starrocks/pull/35003
- Optimized the bitmap_xor function. https://github.com/StarRocks/starrocks/pull/34069
- Supports Copy on Write to optimize Bitmap performance and reduce memory consumption. https://github.com/StarRocks/starrocks/pull/34047

Bug Fixes
Fixed the following issues:
- If a filtering condition is specified in a Broker Load job, BEs may crash during the data loading in certain circumstances. https://github.com/StarRocks/starrocks/pull/29832
- An unknown error is reported when SHOW GRANTS is executed. https://github.com/StarRocks/starrocks/pull/30100
- When data is loaded into a table that uses expression-based automatic partitioning, the error "Error: The row create partition failed since Runtime error: failed to analyse partition value" may be thrown. https://github.com/StarRocks/starrocks/pull/33513
- The error "get_applied_rowsets failed, tablet updates is in error state: tablet:18849 actual row size changed after compaction" is returned for queries. https://github.com/StarRocks/starrocks/pull/33246
- In a StarRocks shared-nothing cluster, queries against Iceberg or Hive tables may cause BEs to crash. https://github.com/StarRocks/starrocks/pull/34682
- In a StarRocks shared-nothing cluster, if multiple partitions are automatically created during data loading, the data loaded may occasionally be written to unmatched partitions. https://github.com/StarRocks/starrocks/pull/34731
- Long-time, frequent data loading into a Primary Key table with persistent index enabled may cause BEs to crash. https://github.com/StarRocks/starrocks/pull/33220
- The error "Exception: java.lang.IllegalStateException: null" is returned for queries. https://github.com/StarRocks/starrocks/pull/33535
- When show proc '/current_queries'; is being executed and meanwhile a query begins to be executed, BEs may crash. https://github.com/StarRocks/starrocks/pull/34316
- Errors may be thrown if large amounts of data are loaded into a Primary Key table with persistent index enabled. https://github.com/StarRocks/starrocks/pull/34352
- After StarRocks is upgraded from v2.4 or earlier to a later version, compaction scores may rise unexpectedly. https://github.com/StarRocks/starrocks/pull/34618
- If INFORMATION_SCHEMA is queried by using the database driver MariaDB ODBC, the CATALOG_NAME column returned in the schemata view holds only null values. https://github.com/StarRocks/starrocks/pull/34627
- FEs crash due to the abnormal data loaded and cannot restart. https://github.com/StarRocks/starrocks/pull/34590
- If schema changes are being executed while a Stream Load job is in the PREPARD state, a portion of the source data to be loaded by the job is lost. https://github.com/StarRocks/starrocks/pull/34381
- Including two or more slashes (/) at the end of the HDFS storage path causes the backup and restore of the data from HDFS to fail. https://github.com/StarRocks/starrocks/pull/34601
- Setting the session variable enable_load_profile to true makes Stream Load jobs prone to fail. https://github.com/StarRocks/starrocks/pull/34544
- Performing partial updates in column mode onto a Primary Key table causes some tablets of the table to show data inconsistencies between their replicas. https://github.com/StarRocks/starrocks/pull/34555
- The partition_live_number property added by using the ALTER TABLE statement does not take effect. https://github.com/StarRocks/starrocks/pull/34842
- FEs fail to start and report the error "failed to load journal type 118". https://github.com/StarRocks/starrocks/pull/34590
- Setting the FE parameter recover_with_empty_tablet to true may cause FEs to crash. https://github.com/StarRocks/starrocks/pull/33071
- Failures in replaying replica operations may cause FEs to crash. https://github.com/StarRocks/starrocks/pull/32295

Compatibility Changes
Parameters
- Added an FE configuration item [enable_statistics_collect_profile](https://github.com/StarRocks/starrocks/blob/main/docs/en/administration/Configuration.md#enable_statistics_collect_profile), which controls whether to generate profiles for statistics queries. The default value is false. https://github.com/StarRocks/starrocks/pull/33815
- The FE configuration item [mysql_server_version](https://github.com/StarRocks/starrocks/blob/main/docs/en/administration/Configuration.md#mysql_server_version) is now mutable. The new setting can take effect for the current session without requiring an FE restart. https://github.com/StarRocks/starrocks/pull/34033
- Added a BE/CN configuration item [update_compaction_ratio_threshold](https://github.com/StarRocks/starrocks/blob/main/docs/en/administration/Configuration.md#update_compaction_ratio_threshold), which controls the maximum proportion of data that a compaction can merge for a Primary Key table in a StarRocks shared-data cluster. The default value is 0.5. We recommend shrinking this value if a single tablet becomes excessively large. For a StarRocks shared-nothing cluster, the proportion of data that a compaction can merge for a Primary Key table is still automatically adjusted. https://github.com/StarRocks/starrocks/pull/35129
System Variables
- Added a session variable cbo_decimal_cast_string_strict, which controls how the CBO converts data from the DECIMAL type to the STRING type. If this variable is set to true, the logic built in v2.5.x and later versions prevails and the system implements strict conversion (namely, the system truncates the generated string and fills 0s based on the scale length). If this variable is set to false, the logic built in versions earlier than v2.5.x prevails and the system processes all valid digits to generate a string. The default value is true. https://github.com/StarRocks/starrocks/pull/34208
- Added a session variable cbo_eq_base_type, which specifies the data type used for data comparison between DECIMAL-type data and STRING-type data. The default value is VARCHAR, and DECIMAL is also a valid value. https://github.com/StarRocks/starrocks/pull/34208
- Added a session variable big_query_profile_second_threshold. When the session variable [enable_profile](https://github.com/StarRocks/starrocks/blob/main/docs/en/reference/System_variable.md#enable_profile) is set to false and the amount of time taken by a query exceeds the threshold specified by the big_query_profile_second_threshold variable, a profile is generated for that query. https://github.com/StarRocks/starrocks/pull/33825

Release date: November 2, 2023

New Features
+ Supports sort keys for Primary Key tables created in shared-data StarRocks clusters.
+ Supports using the str2date function to specify partition expressions for asynchronous materialized views. This helps facilitate incremental updates and query rewrites of asynchronous materialized views created on tables that reside in external catalogs and use the STRING-type data as their partitioning expressions. [29923](https://github.com/StarRocks/starrocks/pull/29923) [#31964](https://github.com/StarRocks/starrocks/pull/31964)
+ Added a new session variable enable_query_tablet_affinity, which controls whether to direct multiple queries against the same tablet to a fixed replica. This session variable is set to false by default. [33049](https://github.com/StarRocks/starrocks/pull/33049)
+ Added the utility function is_role_in_session, which is used to check whether the specified roles are activated in the current session. It supports checking nested roles granted to a user. [32984](https://github.com/StarRocks/starrocks/pull/32984)
+ Supports setting resource group-level query queue, which is controlled by the global variable enable_group_lelvel_query_queue (default value: false). When the global-level or resource group-level resource consumption reaches a predefined threshold, new queries are placed in queue, and will be run when both the global-level resource consumption and the resource group-level resource consumption fall below their thresholds.
- Users can set concurrency_limit for each resource group to limit the maximum number of concurrent queries allowed per BE.
- Users can set max_cpu_cores for each resource group to limit the maximum CPU consumption allowed per BE.
+ Added two parameters, plan_cpu_cost_range and plan_mem_cost_range, for resource group classifiers.
- plan_cpu_cost_range: the CPU consumption range estimated by the system. The default value NULL indicates no limit is imposed.
- plan_mem_cost_range: the memory consumption range estimated by the system. The default value NULL indicates no limit is imposed.
Improvements
+ Window functions COVAR_SAMP, COVAR_POP, CORR, VARIANCE, VAR_SAMP, STD, and STDDEV_SAMP now support the ORDER BY clause and Window clause. [30786](https://github.com/StarRocks/starrocks/pull/30786)
+ An error instead of NULL is returned if a decimal overflow occurs during queries on the DECIMAL type data. [30419](https://github.com/StarRocks/starrocks/pull/30419)
+ The number of concurrent queries allowed in a query queue is now managed by the leader FE. Each follower FE notifies of the leader FE when a query starts and finishes. If the number of concurrent queries reaches the global-level or resource group-level concurrency_limit, new queries are rejected or placed in queue.
Bug Fixes
Fixed the following issues:
+ Spark or Flink may report data read errors due to inaccurate memory usage statistics. [30702](https://github.com/StarRocks/starrocks/pull/30702) [#30751](https://github.com/StarRocks/starrocks/pull/30751)
+ Memory usage statistics for Metadata Cache are inaccurate. [31978](https://github.com/StarRocks/starrocks/pull/31978)
BEs crash when libcurl is invoked. [31667](https://github.com/StarRocks/starrocks/pull/31667)
+ When StarRocks materialized views created on Hive views are refreshed, an error "java.lang.ClassCastException: com.starrocks.catalog. HiveView cannot be cast to com.starrocks.catalog. HiveMetaStoreTable" is returned. [31004](https://github.com/StarRocks/starrocks/pull/31004)
+ If the ORDER BY clause contains aggregate functions, an error "java.lang.IllegalStateException: null" is returned. [30108](https://github.com/StarRocks/starrocks/pull/30108)
+ In shared-data StarRocks clusters, the information of table keys is not recorded in information_schema.COLUMNS. As a result, DELETE operations cannot be performed when data is loaded by using Flink Connector. [31458](https://github.com/StarRocks/starrocks/pull/31458)
+ When data is loaded by using Flink Connector, the load job is suspended unexpectedly if there are highly concurrent load jobs and both the number of HTTP threads and the number of Scan threads have reached their upper limits. [32251](https://github.com/StarRocks/starrocks/pull/32251)
+ When a field of only a few bytes is added, executing SELECT COUNT(*) before the data change finishes returns an error that reads "error: invalid field name". [33243](https://github.com/StarRocks/starrocks/pull/33243)
+ Query results are incorrect after the query cache is enabled. [32781](https://github.com/StarRocks/starrocks/pull/32781)
+ Queries fail during hash joins, causing BEs to crash. [32219](https://github.com/StarRocks/starrocks/pull/32219)
+ DATA_TYPE and COLUMN_TYPE for BINARY or VARBINARY data types are displayed as unknown in the information_schema.columns view. [32678](https://github.com/StarRocks/starrocks/pull/32678)
Behavior Change
+ From v3.1.4 onwards, persistent indexing is enabled by default for Primary Key tables created in new StarRocks clusters (this does not apply to existing StarRocks clusters whose versions are upgraded to v3.1.4 from an earlier version). [33374](https://github.com/StarRocks/starrocks/pull/33374)
+ A new FE parameter enable_sync_publish which is set to true by default is added. When this parameter is set to true, the Publish phase of a data load into a Primary Key table returns the execution result only after the Apply task finishes. As such, the data loaded can be queried immediately after the load job returns a success message. However, setting this parameter to true may cause data loads into Primary Key tables to take a longer time. (Before this parameter is added, the Apply task is asynchronous with the Publish phase.) [27055](https://github.com/StarRocks/starrocks/pull/27055)

Release date: September 25, 2023

New Features
- The aggregate function [group_concat](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/string-functions/group_concat) supports the DISTINCT keyword and the ORDER BY clause. [#28778](https://github.com/StarRocks/starrocks/pull/28778)
- [Stream Load](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-manipulation/STREAM%20LOAD), [Broker Load](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-manipulation/BROKER%20LOAD), [Kafka Connector](https://docs.starrocks.io/en-us/3.1/loading/Kafka-connector-starrocks), [Flink Connector](https://docs.starrocks.io/en-us/3.1/loading/Flink-connector-starrocks), and [Spark Connector](https://docs.starrocks.io/en-us/3.1/loading/Spark-connector-starrocks) support partial updates in column mode on a Primary Key table. [#28288](https://github.com/StarRocks/starrocks/pull/28288)
- Data in partitions can be automatically cooled down over time. (This feature is not supported for [list partitioning](https://docs.starrocks.io/en-us/3.1/table_design/list_partitioning).) [#29335](https://github.com/StarRocks/starrocks/pull/29335) [#29393](https://github.com/StarRocks/starrocks/pull/29393)

Improvements
- Executing SQL commands with invalid comments now returns results consistent with MySQL. [30210](https://github.com/StarRocks/starrocks/pull/30210)

Bug Fixes
Fixed the following issues:

- If the [BITMAP](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-types/BITMAP) or [HLL](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-types/HLL) data type is specified in the WHERE clause of a [DELETE](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-manipulation/DELETE) statement to be executed, the statement cannot be properly executed. [#28592](https://github.com/StarRocks/starrocks/pull/28592)
- After a follower FE is restarted, CpuCores statistics are not up-to-date, resulting in query performance degradation. [28472](https://github.com/StarRocks/starrocks/pull/28472) [#30434](https://github.com/StarRocks/starrocks/pull/30434)
- The execution cost of the [to_bitmap()](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/bitmap-functions/to_bitmap) function is incorrectly calculated. As a result, an inappropriate execution plan is selected for the function after materialized views are rewritten. [#29961](https://github.com/StarRocks/starrocks/pull/29961)
- In certain use cases of the shared-data architecture, after a follower FE is restarted, queries submitted to the follower FE return an error that reads "Backend node not found. Check if any backend node is down". [28615](https://github.com/StarRocks/starrocks/pull/28615)
- If data is continuously loaded into a table that is being altered by using the [ALTER TABLE](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-definition/ALTER%20TABLE) statement, an error "Tablet is in error state" may be thrown. [#29364](https://github.com/StarRocks/starrocks/pull/29364)
- Modifying the FE dynamic parameter max_broker_load_job_concurrency using the ADMIN SET FRONTEND CONFIG command does not take effect. [29964](https://github.com/StarRocks/starrocks/pull/29964) [#29720](https://github.com/StarRocks/starrocks/pull/29720)
- BEs crash if the time unit in the [date_diff()](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/date-time-functions/date_diff) function is a constant but the dates are not constants. [#29937](https://github.com/StarRocks/starrocks/issues/29937)
- In the shared-data architecture, automatic partitioning does not take effect after asynchronous load is enabled. [29986](https://github.com/StarRocks/starrocks/issues/29986)
- If users create a Primary Key table by using the [CREATE TABLE LIKE](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-definition/CREATE%20TABLE%20LIKE) statement, an error "Unexpected exception: Unknown properties: {persistent_index_type=LOCAL}" is thrown. [#30255](https://github.com/StarRocks/starrocks/pull/30255)
- Restoring Primary Key tables causes metadata inconsistency after BEs are restarted. [30135](https://github.com/StarRocks/starrocks/pull/30135)
- If users load data into a Primary Key table on which truncate operations and queries are concurrently performed, an error "java.lang.NullPointerException" is thrown in certain cases. [30573](https://github.com/StarRocks/starrocks/pull/30573)
- If predicate expressions are specified in materialized view creation statements, the refresh results of those materialized views are incorrect. [29904](https://github.com/StarRocks/starrocks/pull/29904)
- After users upgrade their StarRocks cluster to v3.1.2, the storage volume properties of the tables created before the upgrade are reset to null. [30647](https://github.com/StarRocks/starrocks/pull/30647)
- If checkpointing and restoration are concurrently performed on tablet metadata, some tablet replicas will be lost and cannot be retrieved. [30603](https://github.com/StarRocks/starrocks/pull/30603)
- If users use CloudCanal to load data into table columns that are set to NOT NULL but have no default value specified, an error "Unsupported dataFormat value is : \N" is thrown. [30799](https://github.com/StarRocks/starrocks/pull/30799)

Release date: August 25, 2023

Bug Fixes
Fixed the following issues:

- If a user specifies which database is to be connected by default and the user only has permissions on tables in the database but does not have permissions on the database, an error stating that the user does not have permissions on the database is thrown. [29767](https://github.com/StarRocks/starrocks/pull/29767)
- The values returned by the RESTful API action show_data for cloud-native tables are incorrect. [29473](https://github.com/StarRocks/starrocks/pull/29473)
- BEs crash if queries are canceled while the [array_agg()](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/array-functions/array_agg) function is being run. [#29400](https://github.com/StarRocks/starrocks/issues/29400)
- The Default field values returned by the [SHOW FULL COLUMNS](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/Administration/SHOW%20FULL%20COLUMNS) statement for columns of the [BITMAP](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-types/BITMAP) or [HLL](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-types/HLL) data type are incorrect. [#29510](https://github.com/StarRocks/starrocks/pull/29510)
- If the [array_map()](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/array-functions/array_map) function in queries involves multiple tables, the queries fail due to pushdown strategy issues. [#29504](https://github.com/StarRocks/starrocks/pull/29504)
Queries against ORC-formatted files fail because the bugfix ORC-1304 ([apache/orc1299](https://github.com/apache/orc/pull/1299)) from Apache ORC is not merged. [#29804](https://github.com/StarRocks/starrocks/pull/29804)

Behavior Change
- For a newly deployed StarRocks v3.1 cluster, you must have the USAGE privilege on the destination external catalog if you want to run [SET CATALOG](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/data-definition/SET%20CATALOG) to switch to that catalog. You can use [GRANT](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-statements/account-management/GRANT) to grant the required privileges.

- For a v3.1 cluster upgraded from an earlier version, you can run SET CATALOG with inherited privilege.

Release date: August 18, 2023

New Features
- Supports Azure Blob Storage for [shared-data clusters](https://docs.starrocks.io/en-us/3.1/deployment/deploy_shared_data).
- Supports aggregate functions [COVAR_SAMP](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/aggregate-functions/covar_samp), [COVAR_POP](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/aggregate-functions/covar_pop), and [CORR](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/aggregate-functions/corr).
- Supports the following [window functions](https://docs.starrocks.io/en-us/3.1/sql-reference/sql-functions/Window_function): COVAR_SAMP, COVAR_POP, CORR, VARIANCE, VAR_SAMP, STD, and STDDEV_SAMP.

Improvements
- Supports implicit conversions for all compound predicates and for all expressions in the WHERE clause. You can enable or disable implicit conversions by using the [session variable](https://docs.starrocks.io/en-us/3.1/reference/System_variable) ENABLE_STRICT_TYPE. The default value of this session variable is false.

Bug Fixes
Fixed the following issues:

- When data is loaded into tables that have multiple replicas, a large number of invalid log records are written if some partitions of the tables are empty. [28824](https://github.com/StarRocks/starrocks/issues/28824)
- Inaccurate estimation of average row size causes partial updates in column mode on Primary Key tables to occupy excessively large memory. [27485](https://github.com/StarRocks/starrocks/pull/27485)
- If clone operations are triggered on tablets in an ERROR state, disk usage increases. [28488](https://github.com/StarRocks/starrocks/pull/28488)
- Compaction causes cold data to be written to the local cache. [28831](https://github.com/StarRocks/starrocks/pull/28831)