Systemds

[[SYSTEMDS-3480](https://issues.apache.org/jira/browse/SYSTEMDS-3480)] - Verify release scripts with github workflows
[[SYSTEMDS-3482](https://issues.apache.org/jira/browse/SYSTEMDS-3482)] - Parallel Hadoop IO startup
[[SYSTEMDS-3484](https://issues.apache.org/jira/browse/SYSTEMDS-3484)] - FrameAppend optimization
[[SYSTEMDS-3485](https://issues.apache.org/jira/browse/SYSTEMDS-3485)] - Precompile detect type patterns
[[SYSTEMDS-3486](https://issues.apache.org/jira/browse/SYSTEMDS-3486)] - Character Array Type
[[SYSTEMDS-3487](https://issues.apache.org/jira/browse/SYSTEMDS-3487)] - Array primitives with null
[[SYSTEMDS-3488](https://issues.apache.org/jira/browse/SYSTEMDS-3488)] - Compressed Frame Write
[[SYSTEMDS-3489](https://issues.apache.org/jira/browse/SYSTEMDS-3489)] - CLA Compress NaN
[[SYSTEMDS-3490](https://issues.apache.org/jira/browse/SYSTEMDS-3490)] - Compressed Transform Encode
[[SYSTEMDS-3491](https://issues.apache.org/jira/browse/SYSTEMDS-3491)] - CLA Specialized Column Indexes

Test

[[SYSTEMDS-3395](https://issues.apache.org/jira/browse/SYSTEMDS-3395)] - ColGroup Equivalence Tests
[[SYSTEMDS-3397](https://issues.apache.org/jira/browse/SYSTEMDS-3397)] - Python NN testExample

Wish

[[SYSTEMDS-3171](https://issues.apache.org/jira/browse/SYSTEMDS-3171)] - GIO - Mapping from binary data

Task

[[SYSTEMDS-209](https://issues.apache.org/jira/browse/SYSTEMDS-209)] - Algorithm wrappers (ml pipelines, ml context)
[[SYSTEMDS-563](https://issues.apache.org/jira/browse/SYSTEMDS-563)] - MR operations over frames
[[SYSTEMDS-3148](https://issues.apache.org/jira/browse/SYSTEMDS-3148)] - Federated Performance Tests
[[SYSTEMDS-3228](https://issues.apache.org/jira/browse/SYSTEMDS-3228)] - Builtin for k nearest neighbor graph construction
[[SYSTEMDS-3229](https://issues.apache.org/jira/browse/SYSTEMDS-3229)] - WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform
[[SYSTEMDS-3241](https://issues.apache.org/jira/browse/SYSTEMDS-3241)] - Federated quantile
[[SYSTEMDS-3291](https://issues.apache.org/jira/browse/SYSTEMDS-3291)] - Apply builtin for mice
[[SYSTEMDS-3348](https://issues.apache.org/jira/browse/SYSTEMDS-3348)] - Federated Monitoring Tool
[[SYSTEMDS-3496](https://issues.apache.org/jira/browse/SYSTEMDS-3496)] - New builtin function auc (area under ROC curve)

Dependency upgrade

[[SYSTEMDS-3375](https://issues.apache.org/jira/browse/SYSTEMDS-3375)] - CUDA11 / CUDNN8 support

Documentation

[[SYSTEMDS-3407](https://issues.apache.org/jira/browse/SYSTEMDS-3407)] - GMM is missing docs for seed and verbose
[[SYSTEMDS-3434](https://issues.apache.org/jira/browse/SYSTEMDS-3434)] - Python API does not include params for all kvargs

New Contributors
* fathollahzadeh made their first contribution in https://github.com/apache/systemds/pull/1469
* morf1us made their first contribution in https://github.com/apache/systemds/pull/1520

**Full Changelog**: https://github.com/apache/systemds/compare/2.2.0-rc1...3.0.0-rc2

What's Changed

* [MINOR] Update next dev version in the project by j143 in https://github.com/apache/systemds/pull/1789
* [MINOR] Cleanups in cleaning pipelines by Shafaq-Siddiqi in https://github.com/apache/systemds/pull/1792
* [MINOR] Windows action try out by Baunsgaard in https://github.com/apache/systemds/pull/1795
* Bump jackson-databind from 2.13.4.1 to 2.13.4.2 by dependabot in https://github.com/apache/systemds/pull/1799
* [MINOR] MacOS guide update by Baunsgaard in https://github.com/apache/systemds/pull/1823
* [SYSTEMDS-3490] Compressed Transformencode Binning by Baunsgaard in https://github.com/apache/systemds/pull/1824
* [MINOR] Federated Tutorial by Baunsgaard in https://github.com/apache/systemds/pull/1851
* [MINOR] Split balanced minor optimization by Baunsgaard in https://github.com/apache/systemds/pull/1872
* [MINOR] Timeout test by Baunsgaard in https://github.com/apache/systemds/pull/1878
* Bump actions/checkout from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1892
* [MINOR] Refine internal sparse printing by Baunsgaard in https://github.com/apache/systemds/pull/1893
* [MINOR] Frame Float detection refinement by Baunsgaard in https://github.com/apache/systemds/pull/1896
* Bump docker/metadata-action from 4 to 5 by dependabot in https://github.com/apache/systemds/pull/1906
* Bump docker/setup-buildx-action from 2 to 3 by dependabot in https://github.com/apache/systemds/pull/1907
* Bump docker/build-push-action from 4 to 5 by dependabot in https://github.com/apache/systemds/pull/1908
* Bump docker/login-action from 2 to 3 by dependabot in https://github.com/apache/systemds/pull/1909
* Bump actions/setup-node from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1931
* Bump actions/setup-java from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1950
* [SYSTEMDS-3652] Adding support for removing duplicate rows to the unique() builtin function by BACtaki in https://github.com/apache/systemds/pull/1949
* Bump actions/upload-artifact from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1962
* Bump actions/download-artifact from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1963
* Bump actions/setup-python from 4 to 5 by dependabot in https://github.com/apache/systemds/pull/1954
* Bump actions/cache from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1984
* [SYSTEMDS-3670] Added early-stopping mechanism to tSNE by ramesesz in https://github.com/apache/systemds/pull/1990
* [MINOR] copy shaded jar instead of moving by j143 in https://github.com/apache/systemds/pull/1996
* [MINOR] Update generated documentation by j143 in https://github.com/apache/systemds/pull/1997
* [SYSTEMDS-3672] GIO Test Cases false positive by fathollahzadeh in https://github.com/apache/systemds/pull/1999
* [SYSTEMDS-2926] AWS scripts update for EMR-7.0.0 by lachezar-n in https://github.com/apache/systemds/pull/2003

New Contributors
* BACtaki made their first contribution in https://github.com/apache/systemds/pull/1949
* ramesesz made their first contribution in https://github.com/apache/systemds/pull/1990
* lachezar-n made their first contribution in https://github.com/apache/systemds/pull/2003

**Full Changelog**: https://github.com/apache/systemds/compare/3.1.0-rc1...3.2.0-rc2

What's Changed

Sub-task

[[SYSTEMDS-2411](https://issues.apache.org/jira/browse/SYSTEMDS-2411)] - Performance codegen kmeans mnist80m w/ compression
[[SYSTEMDS-3088](https://issues.apache.org/jira/browse/SYSTEMDS-3088)] - Prefetch instruction
[[SYSTEMDS-3098](https://issues.apache.org/jira/browse/SYSTEMDS-3098)] - Broadcast instruction
[[SYSTEMDS-3256](https://issues.apache.org/jira/browse/SYSTEMDS-3256)] - Create apply functions for cleaning primitives
[[SYSTEMDS-3286](https://issues.apache.org/jira/browse/SYSTEMDS-3286)] - LogicalEnumerator change with transitions concept and cleanups
[[SYSTEMDS-3347](https://issues.apache.org/jira/browse/SYSTEMDS-3347)] - Flatten the nested loop for parallel pipelines execution
[[SYSTEMDS-3376](https://issues.apache.org/jira/browse/SYSTEMDS-3376)] - Adding apply_pipeline() builtin for cleaning pipelines API
[[SYSTEMDS-3401](https://issues.apache.org/jira/browse/SYSTEMDS-3401)] - Release docker images with GitHub actions
[[SYSTEMDS-3422](https://issues.apache.org/jira/browse/SYSTEMDS-3422)] - Add monitoring tool testing workflows
[[SYSTEMDS-3443](https://issues.apache.org/jira/browse/SYSTEMDS-3443)] - Asynchronous Execution and Persist Spark Transformations
[[SYSTEMDS-3466](https://issues.apache.org/jira/browse/SYSTEMDS-3466)] - Future-based asynchronous execution of Spark actions
[[SYSTEMDS-3469](https://issues.apache.org/jira/browse/SYSTEMDS-3469)] - New operator linearization order to maximize inter-operator parallelism
[[SYSTEMDS-3470](https://issues.apache.org/jira/browse/SYSTEMDS-3470)] - Lineage-based reuse of Spark actions
[[SYSTEMDS-3473](https://issues.apache.org/jira/browse/SYSTEMDS-3473)] - Push down rmvar instructions for asynchronous instructions
[[SYSTEMDS-3474](https://issues.apache.org/jira/browse/SYSTEMDS-3474)] - Lineage-based reuse of asynchronous operators
[[SYSTEMDS-3479](https://issues.apache.org/jira/browse/SYSTEMDS-3479)] - Persist and reuse Spark RDDs
[[SYSTEMDS-3497](https://issues.apache.org/jira/browse/SYSTEMDS-3497)] - Refactor to add LOP rewrite step in compilation

Bug

[[SYSTEMDS-1026](https://issues.apache.org/jira/browse/SYSTEMDS-1026)] - Fix memory configuration in sparkDML.sh
[[SYSTEMDS-1281](https://issues.apache.org/jira/browse/SYSTEMDS-1281)] - OOM Error On Binary Write
[[SYSTEMDS-1283](https://issues.apache.org/jira/browse/SYSTEMDS-1283)] - Out of memory error
[[SYSTEMDS-2948](https://issues.apache.org/jira/browse/SYSTEMDS-2948)] - CLA Improved Run estimation
[[SYSTEMDS-3045](https://issues.apache.org/jira/browse/SYSTEMDS-3045)] - AttributeError: Function definition not found
[[SYSTEMDS-3272](https://issues.apache.org/jira/browse/SYSTEMDS-3272)] - applySchema built-in to set the schema of frame from DML
[[SYSTEMDS-3339](https://issues.apache.org/jira/browse/SYSTEMDS-3339)] - CSR TSMM left with filled rows bug
[[SYSTEMDS-3353](https://issues.apache.org/jira/browse/SYSTEMDS-3353)] - Sparse TSMM dense row blocks CSR
[[SYSTEMDS-3354](https://issues.apache.org/jira/browse/SYSTEMDS-3354)] - py4j.Py4JException: Method exceptionString([class org.apache.spark.SparkConf]) does not exist
[[SYSTEMDS-3355](https://issues.apache.org/jira/browse/SYSTEMDS-3355)] - MatrixBlock size using CSR when allowed
[[SYSTEMDS-3379](https://issues.apache.org/jira/browse/SYSTEMDS-3379)] - Federated Nan Values
[[SYSTEMDS-3390](https://issues.apache.org/jira/browse/SYSTEMDS-3390)] - countDistinctApprox() operation in AggregateUnaryCPInstruction is inefficient for row/col aggregations
[[SYSTEMDS-3391](https://issues.apache.org/jira/browse/SYSTEMDS-3391)] - Correct the release artifact generation date
[[SYSTEMDS-3394](https://issues.apache.org/jira/browse/SYSTEMDS-3394)] - Log4j incompatible dependencies
[[SYSTEMDS-3396](https://issues.apache.org/jira/browse/SYSTEMDS-3396)] - ConcurrentModificationException in federated execution
[[SYSTEMDS-3398](https://issues.apache.org/jira/browse/SYSTEMDS-3398)] - Jackson Core missing for json writing and reading in reduced binary
[[SYSTEMDS-3400](https://issues.apache.org/jira/browse/SYSTEMDS-3400)] - Fix Java doc warnings
[[SYSTEMDS-3408](https://issues.apache.org/jira/browse/SYSTEMDS-3408)] - Enque output not UTF-8 python
[[SYSTEMDS-3409](https://issues.apache.org/jira/browse/SYSTEMDS-3409)] - Read CSV directly without mtd python
[[SYSTEMDS-3411](https://issues.apache.org/jira/browse/SYSTEMDS-3411)] - Python configuration not loading defaults
[[SYSTEMDS-3412](https://issues.apache.org/jira/browse/SYSTEMDS-3412)] - Matrix Multiplication crash in Spark
[[SYSTEMDS-3414](https://issues.apache.org/jira/browse/SYSTEMDS-3414)] - Pipelines failing in Hybrid execution
[[SYSTEMDS-3415](https://issues.apache.org/jira/browse/SYSTEMDS-3415)] - Built-in tests failure in Git actions
[[SYSTEMDS-3416](https://issues.apache.org/jira/browse/SYSTEMDS-3416)] - Cleaning Pipelines failed with No space left on device
[[SYSTEMDS-3417](https://issues.apache.org/jira/browse/SYSTEMDS-3417)] - IndexOutOfBounds due to int overflow on replace
[[SYSTEMDS-3418](https://issues.apache.org/jira/browse/SYSTEMDS-3418)] - Cleaning Pipelines: Replace function failure in hybrid execution
[[SYSTEMDS-3419](https://issues.apache.org/jira/browse/SYSTEMDS-3419)] - Cleaning Pipelines: Block Sizes mismatch
[[SYSTEMDS-3420](https://issues.apache.org/jira/browse/SYSTEMDS-3420)] - Cleaning Pipelines in hybrid mode: Invalid block dimensions error
[[SYSTEMDS-3424](https://issues.apache.org/jira/browse/SYSTEMDS-3424)] - Federated Statistics print in non federated scenario
[[SYSTEMDS-3425](https://issues.apache.org/jira/browse/SYSTEMDS-3425)] - Spark Aggregate Binary operations parse to Fed instruction
[[SYSTEMDS-3432](https://issues.apache.org/jira/browse/SYSTEMDS-3432)] - FederationUtils.bindResponses causes out of memory because of sparse matrices.
[[SYSTEMDS-3433](https://issues.apache.org/jira/browse/SYSTEMDS-3433)] - Python IDE test Docs fail
[[SYSTEMDS-3435](https://issues.apache.org/jira/browse/SYSTEMDS-3435)] - MSVM robustness for non-existing classes
[[SYSTEMDS-3436](https://issues.apache.org/jira/browse/SYSTEMDS-3436)] - CLA ArrayOutOfBounds in sample
[[SYSTEMDS-3437](https://issues.apache.org/jira/browse/SYSTEMDS-3437)] - CLA Invalid Unique estimate DDC
[[SYSTEMDS-3439](https://issues.apache.org/jira/browse/SYSTEMDS-3439)] - Federated read cache cannot be disabled
[[SYSTEMDS-3442](https://issues.apache.org/jira/browse/SYSTEMDS-3442)] - Monitoring Heavy hitters not always correct list
[[SYSTEMDS-3451](https://issues.apache.org/jira/browse/SYSTEMDS-3451)] - Slow Federated Mlogreg on Criteo (dummy-coded)
[[SYSTEMDS-3452](https://issues.apache.org/jira/browse/SYSTEMDS-3452)] - Incorrect warning when reading scalars
[[SYSTEMDS-3476](https://issues.apache.org/jira/browse/SYSTEMDS-3476)] - Spark with default settings
[[SYSTEMDS-3477](https://issues.apache.org/jira/browse/SYSTEMDS-3477)] - Cleaning Pipelines: Task Parallel Experiments failing in spark mode
[[SYSTEMDS-3498](https://issues.apache.org/jira/browse/SYSTEMDS-3498)] - Unique() crashes with iterator EOF on vectors with >1K distinct items
[[SYSTEMDS-3500](https://issues.apache.org/jira/browse/SYSTEMDS-3500)] - Perftest: Mlogreg on 1M_1k_dense w/ unnecessary spark jobs
[[SYSTEMDS-3501](https://issues.apache.org/jira/browse/SYSTEMDS-3501)] - Perftest: lmDS on 1M_1k_dense with unnecessary spark tsmm
[[SYSTEMDS-3503](https://issues.apache.org/jira/browse/SYSTEMDS-3503)] - Java doc warnings

Epic

[[SYSTEMDS-450](https://issues.apache.org/jira/browse/SYSTEMDS-450)] - Extended spark interfaces
[[SYSTEMDS-3445](https://issues.apache.org/jira/browse/SYSTEMDS-3445)] - Combining compression schemes together
[[SYSTEMDS-3459](https://issues.apache.org/jira/browse/SYSTEMDS-3459)] - Reorganization and cleanup of the internal representation of FrameBlocks

New Feature

[[SYSTEMDS-2551](https://issues.apache.org/jira/browse/SYSTEMDS-2551)] - Federated Compression Instruction
[[SYSTEMDS-2699](https://issues.apache.org/jira/browse/SYSTEMDS-2699)] - CLA IO Compressed Matrices
[[SYSTEMDS-2754](https://issues.apache.org/jira/browse/SYSTEMDS-2754)] - Compressed Max/Min Index support.
[[SYSTEMDS-2830](https://issues.apache.org/jira/browse/SYSTEMDS-2830)] - Functional Compression
[[SYSTEMDS-3280](https://issues.apache.org/jira/browse/SYSTEMDS-3280)] - Homomorphic Encryption for Federated Parameter Servers
[[SYSTEMDS-3303](https://issues.apache.org/jira/browse/SYSTEMDS-3303)] - NN Builtin: Attention Layer
[[SYSTEMDS-3325](https://issues.apache.org/jira/browse/SYSTEMDS-3325)] - Multi-threaded tokenization
[[SYSTEMDS-3337](https://issues.apache.org/jira/browse/SYSTEMDS-3337)] - CLA TSMM direct multiplication
[[SYSTEMDS-3360](https://issues.apache.org/jira/browse/SYSTEMDS-3360)] - Federated async compression
[[SYSTEMDS-3361](https://issues.apache.org/jira/browse/SYSTEMDS-3361)] - Federated Workload-aware Compression
[[SYSTEMDS-3369](https://issues.apache.org/jira/browse/SYSTEMDS-3369)] - Timout setting for all federated tests
[[SYSTEMDS-3374](https://issues.apache.org/jira/browse/SYSTEMDS-3374)] - Federated primitive for transferring a local data object to a federated representation
[[SYSTEMDS-3404](https://issues.apache.org/jira/browse/SYSTEMDS-3404)] - Synchronous with backup workers mode for Parameter Servers
[[SYSTEMDS-3405](https://issues.apache.org/jira/browse/SYSTEMDS-3405)] - Federated Write at site
[[SYSTEMDS-3438](https://issues.apache.org/jira/browse/SYSTEMDS-3438)] - CLA RowSlice compressed return
[[SYSTEMDS-3478](https://issues.apache.org/jira/browse/SYSTEMDS-3478)] - Bitset array for Frames
[[SYSTEMDS-3481](https://issues.apache.org/jira/browse/SYSTEMDS-3481)] - Frame from MatrixBlock improvement
[[SYSTEMDS-3493](https://issues.apache.org/jira/browse/SYSTEMDS-3493)] - Python windows Install
[[SYSTEMDS-3494](https://issues.apache.org/jira/browse/SYSTEMDS-3494)] - Python 3.9 support
[[SYSTEMDS-3495](https://issues.apache.org/jira/browse/SYSTEMDS-3495)] - Parallel Compressed Encode

Story

[[SYSTEMDS-2783](https://issues.apache.org/jira/browse/SYSTEMDS-2783)] - Lineage-based reuse in federated execution
[[SYSTEMDS-3087](https://issues.apache.org/jira/browse/SYSTEMDS-3087)] - Memory management and lazy evaluation in dynamic environments
[[SYSTEMDS-3463](https://issues.apache.org/jira/browse/SYSTEMDS-3463)] - Add unique() built-in function

Improvement

[[SYSTEMDS-295](https://issues.apache.org/jira/browse/SYSTEMDS-295)] - Unexpected order when print !boolean
[[SYSTEMDS-1169](https://issues.apache.org/jira/browse/SYSTEMDS-1169)] - Clean Up and Automate Python Tests
[[SYSTEMDS-1406](https://issues.apache.org/jira/browse/SYSTEMDS-1406)] - Fix whitespace issues in main algorithms
[[SYSTEMDS-1532](https://issues.apache.org/jira/browse/SYSTEMDS-1532)] - Introduce Python scripts to launch SystemML from the Command Line
[[SYSTEMDS-2513](https://issues.apache.org/jira/browse/SYSTEMDS-2513)] - Improve the Development and user experience on Windows
[[SYSTEMDS-2897](https://issues.apache.org/jira/browse/SYSTEMDS-2897)] - CLA decompressing write
[[SYSTEMDS-3185](https://issues.apache.org/jira/browse/SYSTEMDS-3185)] - Federated Multi Tenant Backend
[[SYSTEMDS-3187](https://issues.apache.org/jira/browse/SYSTEMDS-3187)] - Add documentation for the release scripts
[[SYSTEMDS-3192](https://issues.apache.org/jira/browse/SYSTEMDS-3192)] - Test large dense block compression
[[SYSTEMDS-3254](https://issues.apache.org/jira/browse/SYSTEMDS-3254)] - CountDistinct Col and Row & Unique
[[SYSTEMDS-3282](https://issues.apache.org/jira/browse/SYSTEMDS-3282)] - Upper bound for number of decoders
[[SYSTEMDS-3283](https://issues.apache.org/jira/browse/SYSTEMDS-3283)] - Multi-threaded ctable instruction
[[SYSTEMDS-3319](https://issues.apache.org/jira/browse/SYSTEMDS-3319)] - CLA Generalize Bin Packing
[[SYSTEMDS-3323](https://issues.apache.org/jira/browse/SYSTEMDS-3323)] - CLA move combine of empty to estim
[[SYSTEMDS-3328](https://issues.apache.org/jira/browse/SYSTEMDS-3328)] - Federated transform for equi-height
[[SYSTEMDS-3359](https://issues.apache.org/jira/browse/SYSTEMDS-3359)] - Sample-based Recode Map Size Estimation
[[SYSTEMDS-3386](https://issues.apache.org/jira/browse/SYSTEMDS-3386)] - Refactor runtime replacement of CP or SP instructions with FED instructions
[[SYSTEMDS-3393](https://issues.apache.org/jira/browse/SYSTEMDS-3393)] - Use Java (JDK17) SIMD Implementation
[[SYSTEMDS-3413](https://issues.apache.org/jira/browse/SYSTEMDS-3413)] - Add row/col aggregation support to countDistinct() builtin function
[[SYSTEMDS-3429](https://issues.apache.org/jira/browse/SYSTEMDS-3429)] - Use Local Level of Parallelism when Transformencoding in Federated Mode
[[SYSTEMDS-3440](https://issues.apache.org/jira/browse/SYSTEMDS-3440)] - Federated Requests Coordinator Hostname
[[SYSTEMDS-3444](https://issues.apache.org/jira/browse/SYSTEMDS-3444)] - Spark Write CLA
[[SYSTEMDS-3446](https://issues.apache.org/jira/browse/SYSTEMDS-3446)] - DDC Append
[[SYSTEMDS-3447](https://issues.apache.org/jira/browse/SYSTEMDS-3447)] - SDC Append
[[SYSTEMDS-3448](https://issues.apache.org/jira/browse/SYSTEMDS-3448)] - Uncompressed Append
[[SYSTEMDS-3449](https://issues.apache.org/jira/browse/SYSTEMDS-3449)] - Const/Empty append
[[SYSTEMDS-3450](https://issues.apache.org/jira/browse/SYSTEMDS-3450)] - DDCFOR Append
[[SYSTEMDS-3453](https://issues.apache.org/jira/browse/SYSTEMDS-3453)] - Offsets Append
[[SYSTEMDS-3454](https://issues.apache.org/jira/browse/SYSTEMDS-3454)] - CLA Sheme primitive
[[SYSTEMDS-3455](https://issues.apache.org/jira/browse/SYSTEMDS-3455)] - Improved multi-threaded unary operations
[[SYSTEMDS-3456](https://issues.apache.org/jira/browse/SYSTEMDS-3456)] - MatrixBlock equals
[[SYSTEMDS-3457](https://issues.apache.org/jira/browse/SYSTEMDS-3457)] - MatrixBlock equals Sparse Specialization
[[SYSTEMDS-3458](https://issues.apache.org/jira/browse/SYSTEMDS-3458)] - Add support for Spark backend to countDistinct() builtin function
[[SYSTEMDS-3460](https://issues.apache.org/jira/browse/SYSTEMDS-3460)] - Move FrameBlock out of MatrixBlock path.
[[SYSTEMDS-3461](https://issues.apache.org/jira/browse/SYSTEMDS-3461)] - FrameBlock Arrays separation
[[SYSTEMDS-3462](https://issues.apache.org/jira/browse/SYSTEMDS-3462)] - FrameBlock Iterators Factory Pattern
[[SYSTEMDS-3464](https://issues.apache.org/jira/browse/SYSTEMDS-3464)] - Python Combine Write
[[SYSTEMDS-3465](https://issues.apache.org/jira/browse/SYSTEMDS-3465)] - Typed return on CacheBlock Interface Slice
[[SYSTEMDS-3467](https://issues.apache.org/jira/browse/SYSTEMDS-3467)] - Add support for MULTI_BLOCK Spark backend support for countDistinct()
[[SYSTEMDS-3471](https://issues.apache.org/jira/browse/SYSTEMDS-3471)] - Enable multi-threaded transformencode/apply
[[SYSTEMDS-3472](https://issues.apache.org/jira/browse/SYSTEMDS-3472)] - Spark Append Frame Bug

What's Changed

* [SYSTEMDS-3193] Promote svn repo `dev/systemds/2.x-rc` to `release/systemds/2.x` by j143 in https://github.com/apache/systemds/pull/1427
* [SYSTEMDS-3190] Do not run workflow for empty checks by j143 in https://github.com/apache/systemds/pull/1426
* [SYSTEMDS-2599] Development branch name change to main by j143 in https://github.com/apache/systemds/pull/1429
* [MINOR] Update java language level by Baunsgaard in https://github.com/apache/systemds/pull/1430
* [MINOR] Update docs to `2.3.0-SNAPSHOT` by j143 in https://github.com/apache/systemds/pull/1433
* [SYSTEMDS-3196] Update `org.apache` parent pom version to 24 by j143 in https://github.com/apache/systemds/pull/1436
* [SYSTEMDS-3212] Move Prefetch threadpool to CommonThreadPool by phaniarnab in https://github.com/apache/systemds/pull/1450
* [MINOR] Upgrade junit version to 4.13.2 by j143 in https://github.com/apache/systemds/pull/1451
* [MINOR] Silence mvn package download info by j143 in https://github.com/apache/systemds/pull/1465
* [MINOR] Fix imports in BuiltinUnaryGPUInstructionTest by j143 in https://github.com/apache/systemds/pull/1467
* [MINOR][PYTHON-DOC] Update Multiple Federated Environments Example Result by fathollahzadeh in https://github.com/apache/systemds/pull/1469
* [SYSTEMDS-3236] Cache-friendly Apply phase for sparse target matrix by phaniarnab in https://github.com/apache/systemds/pull/1473
* [SYSTEMDS-3239] Train and predict in different Contexts Python by Baunsgaard in https://github.com/apache/systemds/pull/1474
* [SYSTEMDS-3240] Fix IOGEN test path if the iogen directory doesn't exist by fathollahzadeh in https://github.com/apache/systemds/pull/1475
* [SYSTEMDS-3187] Add documentation for the release scripts by j143 in https://github.com/apache/systemds/pull/1478
* [MINOR] Cleaning Pipelines cleanups by Shafaq-Siddiqi in https://github.com/apache/systemds/pull/1493
* Action for building docker images automatically by j143 in https://github.com/apache/systemds/pull/1441
* [SYSTEMDS-3268] Publish Docker images on schedule by j143 in https://github.com/apache/systemds/pull/1500
* [SYSTEMDS-3270][DOCS] Docker usage documentation by j143 in https://github.com/apache/systemds/pull/1501
* [SYSTEMDS-3271] Publish docker test image when requested by Baunsgaard in https://github.com/apache/systemds/pull/1502
* [SYSTEMDS-3273] Federated Timeout by Baunsgaard in https://github.com/apache/systemds/pull/1508
* Bump actions/setup-python from 1 to 2.3.1 by dependabot in https://github.com/apache/systemds/pull/1512
* [MINOR][DOC] xgboost function y parameter correct usage by j143 in https://github.com/apache/systemds/pull/1532
* Bump actions/setup-python from 2.3.1 to 2.3.2 by dependabot in https://github.com/apache/systemds/pull/1538
* [MINOR] Refactoring input and output parameters of dbscanApply and db… by Shafaq-Siddiqi in https://github.com/apache/systemds/pull/1541
* [SYSTEMDS-3288] CLA SDC isolated DefaultTuple by Baunsgaard in https://github.com/apache/systemds/pull/1533
* [SYSTEMDS-3184] Builtin for computing information gain using entropy and gini by morf1us in https://github.com/apache/systemds/pull/1520
* [DOCS] Add missing data files in docs by j143 in https://github.com/apache/systemds/pull/1548
* Bump actions/setup-python from 2.3.2 to 3 by dependabot in https://github.com/apache/systemds/pull/1552
* Bump actions/checkout from 2 to 3 by dependabot in https://github.com/apache/systemds/pull/1553
* [MINOR] Performance improvements in cleaning pipelines by Shafaq-Siddiqi in https://github.com/apache/systemds/pull/1556
* Bump actions/cache from 2 to 3 by dependabot in https://github.com/apache/systemds/pull/1570
* [Cleaning Pipelines] MINOR improvements in allocation of resources by Shafaq-Siddiqi in https://github.com/apache/systemds/pull/1576
* keep release scripts concise by j143 in https://github.com/apache/systemds/pull/1549
* Bump actions/setup-java from 2 to 3 by dependabot in https://github.com/apache/systemds/pull/1582
* Create `CITATION.cff` by j143 in https://github.com/apache/systemds/pull/1449
* [SYSTEMDS-3345] Add support for as.boolean(X) by Baunsgaard in https://github.com/apache/systemds/pull/1579
* [SYSTEMDS-3022] Avoid cudaMemset() where possible by corepointer in https://github.com/apache/systemds/pull/1572
* Bump jackson-databind from 2.12.3 to 2.12.6.1 by dependabot in https://github.com/apache/systemds/pull/1577
* [MINOR] Github actions use Java OpenJDK adopt-hotspot by Baunsgaard in https://github.com/apache/systemds/pull/1598
* Bump docker/login-action from 1 to 2 by dependabot in https://github.com/apache/systemds/pull/1606
* Bump docker/build-push-action from 2 to 3 by dependabot in https://github.com/apache/systemds/pull/1605
* Bump docker/setup-buildx-action from 1 to 2 by dependabot in https://github.com/apache/systemds/pull/1604
* Bump docker/metadata-action from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1607
* generate proto file by j143 in https://github.com/apache/systemds/pull/1620
* [MINOR] Add `protobuf` parameter for consistency by j143 in https://github.com/apache/systemds/pull/1621
* Bump actions/setup-python from 3 to 4 by dependabot in https://github.com/apache/systemds/pull/1633
* [MINOR] Minor fixes i.e., validation checks, formatting e.t.c. by Shafaq-Siddiqi in https://github.com/apache/systemds/pull/1636

New Contributors
* fathollahzadeh made their first contribution in https://github.com/apache/systemds/pull/1469
* morf1us made their first contribution in https://github.com/apache/systemds/pull/1520

**Full Changelog**: https://github.com/apache/systemds/compare/2.2.0-rc1...3.0.0-rc2