Highlights
- Support Ray 1.9.0 - 2.0.0
- Support Spark 3.1/3.2
- Hive support
- Ray placement group support
- Support multiple users running RayDP on the same node
- Support fractional resource scheduling
- Updated Estimator API using Ray Dataset and Ray Train
- Support custom Spark location by picking up $SPARK_HOME
- RayDP executor extra class path support
- Support data ownership transfer for conversion from Spark Dataframe to Ray Dataset
- New Colab tutorials
Thanks Bowen0729, carsonwang, hezhaozhao-git, jjyao, KepingYan, kira-lin, marin-ma, n1CkS4x0, pang-wu, wybryan, Yard1 for their contributions to the release!