Graphscope

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.27.0 release. This release contains many important features and improvements to GraphScope FLEX、gsctl command-line utility、Graph Interactive Engine (GIE) and Groot persistent storage.

We highlight the following improvements included in this release:

**1. `gsctl` command-line utility**

`gsctl` is a command-line utility designed to simplify the usage of GraphScope. With gsctl, users can easily interact with [GraphScope Flex](https://github.com/alibaba/GraphScope/tree/main/flex) product, such as Interactive(for high QPS queries) and Insight(for online BI analysis), build binaries and docker images, and perform various utility tasks related to GraphScope. See more detailed informaton from [doc](https://graphscope.io/docs/latest/utilities/gs).

**Key Features of gsctl**

- Utility Scripts: gsctl provides a collection of utility scripts that simplify common tasks such as installing dependencies, building binaries and docker images, and running test suites.
- Client/Server Mode: With the client/server mode, users can connect to a launched coordinator and interact with specific GraphScope products behind it. This mode enables managing sessions, resources, and other product-specific functionalities.
- Scope Management: gsctl allows users to switch between global and local scopes, enabling them to work with graphs and resources at different levels. This flexibility enhances productivity and enables efficient management of graph computation tasks.
- Configuration Management: gsctl includes configuration options that can be customized to suit specific requirements. Users can specify a custom coordinator endpoint and manage other configuration settings as needed.

**2. [GraphScope FLEX](https://github.com/alibaba/GraphScope/tree/main/flex)**
- Support build docker images through `gsctl flexbuild` command.
bash
$ git clone https://github.com/alibaba/GraphScope.git && cd GraphScope
$ python3 gsctl.py flexbuild insight/interactive --app docker


- Support deploy `Interactive` and `Insight` product through docker image.
bash
$ docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex
launch engine and coordinator service: 12312 gremlin endpoint, 55556 grpc endpoint, 8080 coordinator endpoint
$ docker run -p 12312:12312 -p 55556:55556 -p 8080:8080 registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex /usr/local/groot/bin/start_local_cluster.sh

- Connect to a launched coordinator and interact with the products
bash
graphscopehost:/work/python$ gsctl connect --coordinator-endpoint http://127.0.0.1:8081
[SUCCESS] Connected to http://127.0.0.1:8081, coordinator is serving with GRAPHSCOPE_INSIGHT mode.

Try 'gsctl --help' for help.

graphscopehost:/work/python$ gsctl ls
schema
├── vertex types
│ ├── person
│ │ ├── Property(name: id, type: LONG, is_primary_key: True)
│ │ ├── Property(name: name, type: STRING, is_primary_key: False)
│ │ └── Property(name: age, type: LONG, is_primary_key: False)
│ └── software
│ ├── Property(name: id, type: LONG, is_primary_key: True)
│ ├── Property(name: name, type: STRING, is_primary_key: False)
│ └── Property(name: lang, type: STRING, is_primary_key: False)
└── edge types
├── (person) -[created]-> (software)
│ ├── Property(name: edge_id, type: LONG, is_primary_key: False)
│ └── Property(name: weight, type: DOUBLE, is_primary_key: False)
└── (person) -[knows]-> (person)
├── Property(name: edge_id, type: LONG, is_primary_key: False)
└── Property(name: weight, type: DOUBLE, is_primary_key: False)


**3. Enhancements for Groot**
- Integrate a prototype of OpenTelemetry into Groot, users could switch on option `trace.enabled=true`, which will have a jaeger container that collect traces. And it will serve a website on `POD_IP:16686`.

Docker Images

shell
Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.27.0

Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.27.0

Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.27.0
Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.27.0

Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.27.0

GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0

Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest


What's Changed
* feat(flex): refactor memory management of mmap_array to support Hugepage. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3505
* fix(interactive): Fix install_dependency.sh by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3500
* fix(flex): Fixed the usage of storage strategy. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3506
* fix(interactive): Fix building flex with `-DBUILD_HQPS=OFF` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3497
* fix(interactive): 离线全量导入时, 新增wait参数, 在download全部完成后, wait指定时间(单位ms), 再进行提交 by bufapiqi in https://github.com/alibaba/GraphScope/pull/3509
* ci: Fix ci failure and pre-release v0.27.0 by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3512
* feat(flex): Configure the memory strategy with memory-level. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3511
* refactor: coordinator and gsctl under FLEX architecture by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3481
* chore(deps): bump werkzeug from 0.16.1 to 2.3.8 in /flex/coordinator by dependabot in https://github.com/alibaba/GraphScope/pull/3518
* chore(deps): bump flask from 2.1.1 to 2.2.5 in /flex/coordinator by dependabot in https://github.com/alibaba/GraphScope/pull/3519
* docs(interactive): Update doc by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3514
* ci: fixes a syntax error in CI by sighingnow in https://github.com/alibaba/GraphScope/pull/3521
* fix(interactive): Fix argument passing in hqps_app by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3522
* chore: change manylinux2014-ci to manylinux2014 by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3529
* feat(flex): Implemented Immutable Csrs. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3527
* fix(interactive): Fix interactive building Dockerfile by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3528
* fix(interactive): Fix error when loading edge with string properties by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3523
* feat(interactive): Update graph `graph_algo` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3486
* fix(interactive): support non-blocking data loading interface in coordinator and fix failure during flexbuild process by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3530
* feat(interactive): Decouple ingestor and store, remote direct RPC connections. by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3525
* Update README.md for SIGMOD 2024 paper by wenyuanyu in https://github.com/alibaba/GraphScope/pull/3534
* fix(interactive): Integrate Gremlin Into New Compilation Stack by shirly121 in https://github.com/alibaba/GraphScope/pull/3332
* fix(k8s): Bump up vineyard to v0.20.3 by dashanji in https://github.com/alibaba/GraphScope/pull/3532
* fix(interactive): Fix sinking `ElementID` for TwoLabelVertexSet by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3535
* fix(interactive): Fix `PathExpand` related bugs by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3538
* fix(interactive): fix install_dependencies by liulx20 in https://github.com/alibaba/GraphScope/pull/3539
* fix(flex): Fix warning when compiling with `AOCC` compilers by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3510
* fix(interactive): Fix `test_gremlin_timeout` in Python Unit Tests by shirly121 in https://github.com/alibaba/GraphScope/pull/3536
* fix(interactive): Fix Bugs of Group Returning Results from Compiler by shirly121 in https://github.com/alibaba/GraphScope/pull/3533
* ci: fixes some dependencies issues by sighingnow in https://github.com/alibaba/GraphScope/pull/3553
* chore: Spelling comments by jsoref in https://github.com/alibaba/GraphScope/pull/3549
* docs: fixes typos in documentation by jsoref in https://github.com/alibaba/GraphScope/pull/3554
* fix(analytical): Fix Louvain algorithm's move on const message by songqing in https://github.com/alibaba/GraphScope/pull/3551
* docs: fixes typos in YAML configurations by jsoref in https://github.com/alibaba/GraphScope/pull/3555
* feat: Impl the file upload interface for FLEX dataloading by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3557
* fix(interactive): support src and dst of an edge with different type by liulx20 in https://github.com/alibaba/GraphScope/pull/3552
* feat(interactive): Support PTHash indexer by liulx20 in https://github.com/alibaba/GraphScope/pull/3558
* fix(interactive): Fix secondary catchup error by reopening secondary by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3567
* fix(interactive): Fix a bug in pkscan with expected values given in `Within` by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3540
* fix(analytical): fix bug that flash app can not run on string oid graph by acezen in https://github.com/alibaba/GraphScope/pull/3570
* fix: Spelling python graphscope tests by jsoref in https://github.com/alibaba/GraphScope/pull/3568
* docs: fixes typos in flex coordinator by jsoref in https://github.com/alibaba/GraphScope/pull/3556
* fix(interactive): Fix typo in cypher query examples by jsoref in https://github.com/alibaba/GraphScope/pull/3559
* fix(analytical): trim the physical memory usage when unloading graphs by sighingnow in https://github.com/alibaba/GraphScope/pull/3548
* fix(interactive): snapshot id may lag behind in some scenarios by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3576
* feat(interactive): Add profile to build compiler fat-jar by lnfjpt in https://github.com/alibaba/GraphScope/pull/3574
* ci: fixes typos in flex coordinator by jsoref in https://github.com/alibaba/GraphScope/pull/3572
* fix(interactive): Drop unused classes and modules in groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3577
* fix: fixes typos/spelling in flex engines by jsoref in https://github.com/alibaba/GraphScope/pull/3582
* fix: fixes typos/spelling in flex codegen by jsoref in https://github.com/alibaba/GraphScope/pull/3581
* fix(interactive): Migrating Physical Layer into New Compilation Stack by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3560
* fix: fixes the compiler error under rust >= 1.76.0 by sighingnow in https://github.com/alibaba/GraphScope/pull/3584
* fix(interactive): 离线全量导入支持vipserver发现endpoint, 支持replay, 支持compact, 支持reopen by bufapiqi in https://github.com/alibaba/GraphScope/pull/3585
* fix(python): Fixes `graph.unload()`: makes the warning loudly, and use the new DelData API by sighingnow in https://github.com/alibaba/GraphScope/pull/3593
* fix: fixes spelling errors in analytical engine java by jsoref in https://github.com/alibaba/GraphScope/pull/3588
* fix(interactive): Use a larger version upper bound by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3595
* fix(interactive): Rename `modern_graph.yaml` to `graph.yaml` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3603
* feat(interactive): Add `entrypoint.sh` for Interactive by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3604
* refactor(interactive): Wrap schema parsing errors by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3607
* fix(analytical): remove `scripts/install_deps.sh` and update the related document by acezen in https://github.com/alibaba/GraphScope/pull/3608
* fix(interactive): Fix Bug When Global Id Parameters Having Long Type by shirly121 in https://github.com/alibaba/GraphScope/pull/3597
* feat(interactive): Add flag files to track bulk loading progress by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3605
* fix(interactive): Fix Bug in Operator Precedence of `not` by shirly121 in https://github.com/alibaba/GraphScope/pull/3598
* fix(interactive): Migrating Physical Layer (Repartition) into New Compilation Stack by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3586
* fix(interactive): Fix `interactive-runtime.Dockerfile` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3614
* fix: fixes spelling in analytical engine core by jsoref in https://github.com/alibaba/GraphScope/pull/3587
* fix(analytical): Fix graph template parsing in Java SDK by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3617
* fix(analytical): Fix documentation for GRAPE-JDK by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3618
* fix(interactive): use pk hash to generate eid if has pk by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3621
* fix(interactive): Support Gremlin `union` Step in Calcite-Based IR Layer by shirly121 in https://github.com/alibaba/GraphScope/pull/3594
* refactor(interactive): Add type and queryStr for `StoredProcedureMeta` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3622
* feat(interactive): Add OpenTelemetry instrument for flex interactive by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3628
* fix(interactive): fix a bug in PathExpand with 'PATH_OPT' as 'SIMPLE' by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3623
* fix(interactive): Add Cypher Queries used in GOpt Benchmark by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3625
* fix(python): Enable the BUILD_VINEYARD_GRAPH_WITH_GAR compile option in gsctl by acezen in https://github.com/alibaba/GraphScope/pull/3627
* fix(interactive): Fix CMakeLists.txt when otel not found in flex by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3633
* ci: Building vineyard-dev image by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3635
* fix(learning): fix gltorch wheel building by LiSu in https://github.com/alibaba/GraphScope/pull/3629
* refactor(interactive): Fix bug and use `ServiceConfig` to gather all configurations by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3634
* fix(interactive): Fix CI for HQPS engine by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3638
* fix(interactive): Fix argument parsing for `enable-admin-service` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3640
* feat(interactive): Support gremlin query. by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3575
* feat: Integrate groot into flex architecture by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3566
* fix(interactive): Fix Timeout Bug & Process Results From Pegasus Synchronously by shirly121 in https://github.com/alibaba/GraphScope/pull/3630
* feat(interactive): Integrate otel into groot, and add charts for one-pod by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3641
* fix(interactive): Fix CI Bug Caused By PR 3630 by shirly121 in https://github.com/alibaba/GraphScope/pull/3647
* fix(interactive): Minor typo fix by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3643
* refactor(interactive): Support Gremlin Expression in Calcite-Based IR Layer by shirly121 in https://github.com/alibaba/GraphScope/pull/3637
* fix(interactive): Fix the incorrect casting from int into string when converting from `YAML` to `JSON` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3649
* refactor: Make gsctl to be more like a database design by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3648
* refactor: Make gsctl to be more like a database design for Groot storage by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3651
* fix(interactive): Fix failing to install Arrow by liulx20 in https://github.com/alibaba/GraphScope/pull/3652
* fix(interactive): Introduce the high-order statistics provider `GLogue` into the new Compilation Stack by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3609
* feat(interactive): Enable starting compiler as a subprocess of interactive server by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3650
* fix(interactive): Prevent persist invalid meta during abnormal shutdown. by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3661
* ci: Fixes CI failure on macOS by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3662
* fix(interactive): add store-state service back by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3664
* fix(interactive): Fix the occasionally incorrect pegasus server routing configuration by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3673
* fix(learning): Fix torch version to be compatible with gcc8 and python3.7 by LiSu in https://github.com/alibaba/GraphScope/pull/3677
* fix(interactive): Fix unexpected alias when fusion `Expand` + `GetV` by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3676
* fix(learning): specify proper torch version for macos by LiSu in https://github.com/alibaba/GraphScope/pull/3679
* fix(interactive): fix the bug in missing metadata in `PathExpand` by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3667

New Contributors
* jsoref made their first contribution in https://github.com/alibaba/GraphScope/pull/3549

**Full Changelog**: https://github.com/alibaba/GraphScope/compare/v0.26.0...v0.27.0

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.26.0 release. This release contains many important features and improvements to GraphScope FLEX and Graph Interactive Engine (GIE), including secondary instance and type inference in GIE, loading from distributed data source and Macs deployment in FLEX Interactive.

We highlight the following improvements included in this release:


**1. FLEX Interactive**


- Support use `GraphScope Interactive` on macos. Download the latest release to give it a shot [interactive-latest](https://interactive-release.oss-cn-hangzhou.aliyuncs.com/interactive-latest.zip)
- Support bulk loading with odps table as data source. Here is an example
yaml
graph: dd_graph
loading_config:
data_source:
scheme: odps file, odps
import_option: init init, overwrite
format:
type: arrow
vertex_mappings:
- type_name: Person must align with the schema
inputs:
- your_proj_name/table_name/partition_col_name=paritition_name
column_mappings:
- column:
index: 0 can be omitted if the index is the same as the property index
name: user_id can be omitted if the name is not known
property: id
edge_mappings:
- type_triplet:
edge: Knows
source_vertex: Person
destination_vertex: Person
inputs:
- your_proj_name/table_name/partition_col_name=paritition_name
source_vertex_mappings:
- column:
index: 0
name: src_user_id
property: id
destination_vertex_mappings:
- column:
index: 1
name: dst_user_id
property: id


**2. Enhancements for GIE**

- Support secondary instance in Groot. It enables user open another read-only instance along with the original instance, providing additional read qps to users.
- How to use: Set `secondary.enabled=true` in helm charts to enable secondary mode. Other options could remain unchanges with the primary instance.
- Note in secondary instance it could only be queried upon, the modification of schema and data should be routed to primary instance.
- See more details in the [doc](https://graphscope.io/docs/latest/storage_engine/groot#secondary-instance)

- Support automatic type inference in graph queries, which can affect query execution in two ways:
- Checking if vertex and edge types in the graph conform to user-defined schemas

Now compiler can infer that 'kno' is a nonexistent edge type and returns error
Match (a:person)-[b:kno]->(c) Return a, b, c;

- Inferring vertex and edge types in the graph.

Now compiler can infer a and b must both be person types, and execution can be more efficient
Match (a)-[:knows]->(b) Return labels(a), labels(b)
=> Match (a:person)-[:knows]->(b:person) Return labels(a), labels(b)

See more details in the [doc](https://graphscope.io/docs/latest/interactive_engine/optimizer)

**3. Other enhancements and bug fixes**
- Introduce `graph_algo` as another example graph. https://github.com/alibaba/GraphScope/tree/main/flex/interactive/examples/graph_algo.
- Support String `DT_STRING` as edge property type.
- Refactor Flex/CMakeLists.txt to support use installed flex via `find_package(flex)`.
- Fix decode error in Groot
- Fix memory leak in Pegasus and data race in Groot
- Fix GIE peers_contains bugs in count operator in subtasks
- Fix Pegasus unexpect result of operator aggregate() + iterate()
- Refine Log Printing Messages in Compiler
- Fix bugs of Groot bulk loading when it retry to download source files
- Fix bugs in GIE for edge filtering

Docker Images

shell
Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.26.0

Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.26.0

Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.26.0
Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.26.0

Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.26.0

GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.26.0

Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest


What's Changed
* ci: fixes failure during pushing image and pre-release v0.26.0 by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3339
* fix(interactive): Fix decode error in groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3340
* fix(interactive): Check vertex existence whenever queried by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3343
* feat(interactive): Support more data types for vertex/edge property by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3346
* feat(interactive): Support fold_partiton_by_key by lnfjpt in https://github.com/alibaba/GraphScope/pull/3324
* fix(interactive): fix a memory leak bug in IR Core by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3349
* fix(interactive): replace `ok_or()` by `ok_or_else()` for lazy evaluation by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3350
* feat(flex): support LFIndexer resize by liulx20 in https://github.com/alibaba/GraphScope/pull/3351
* refactor(interactive): Refactor CSVFragmentLoader to remove redundant code by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3352
* fix(interactive): Fix support for PathExpand for multiple triplets. by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3335
* fix(interactive): Remove Unnecessary JNA Structure Creation to Save Memory Consumption by shirly121 in https://github.com/alibaba/GraphScope/pull/3354
* fix(interactive): extension use hash(srcId, dstId, edgeLabelId, edgePks) instead of increase eid by bufapiqi in https://github.com/alibaba/GraphScope/pull/3357
* fix(interactive): Comparison between `std::shared_ptr<arrow::DataType>` should use `Equals()` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3361
* chore: fixes the missing tzdata issue in graphscope ubuntu-22.04 based images by sighingnow in https://github.com/alibaba/GraphScope/pull/3363
* chore: make sure tzdata can be installed by sighingnow in https://github.com/alibaba/GraphScope/pull/3365
* fix(interactive): fix memory leak in pegasus by lnfjpt in https://github.com/alibaba/GraphScope/pull/3367
* fix(interactive): add some optimizer microbenchmark queries by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3369
* fix(interactive): Fix memory leak in pegasus and data race in groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3371
* chore: Add writer interface for groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3373
* fix(interactive): Add a new release-with-debug-info profile by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3372
* fix(interactive): fix the bug in filtering after intersection in GIE Runtime in distributed computation by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3359
* feat(interactive): Add Rate Limiter for Control Flow by shirly121 in https://github.com/alibaba/GraphScope/pull/3368
* refactor(interactive): Improve the code of checking if a given label sets contain all labels in the schema by longbinlai in https://github.com/alibaba/GraphScope/pull/3374
* feat(interactive): Implement admin http service for hqps engine by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3322
* fix(interactive): Support Start Tag for Graph Operators in Compiler by shirly121 in https://github.com/alibaba/GraphScope/pull/3376
* fix(interactive): Fix documentation for interactive. by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3384
* docs: Fix GIE Benchmark Readme bug by JackyYangPassion in https://github.com/alibaba/GraphScope/pull/3389
* feat(interactive): Enable replay WAL records from offset of timestamp in groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3385
* refactor(interactive): flex storage with mmap array by liulx20 in https://github.com/alibaba/GraphScope/pull/3356
* fix(interactive): event message error by lnfjpt in https://github.com/alibaba/GraphScope/pull/3380
* fix(interactive): fix peers_contains bugs in count operator in subtasks by lnfjpt in https://github.com/alibaba/GraphScope/pull/3390
* fix(interactive): fix unexpect result of operator aggregate() + iterate() by lnfjpt in https://github.com/alibaba/GraphScope/pull/3391
* fix(interactive): Fix Start Time Metric in Compiler Log by shirly121 in https://github.com/alibaba/GraphScope/pull/3393
* fix(interactive): Refine Log Printing Messages in Compiler by shirly121 in https://github.com/alibaba/GraphScope/pull/3381
* feat(interactive): Introduce explicit barrier for actors when switching query service to a different graph by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3395
* feat(interactive): Add support for loading graph with odps table as data source by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3305
* refactor(interactive): Support Logical Operator `within` in Index Predicate by shirly121 in https://github.com/alibaba/GraphScope/pull/3398
* feat(interactive): support string as edge property type by liulx20 in https://github.com/alibaba/GraphScope/pull/3403
* fix(interactive): Pass acid by liulx20 in https://github.com/alibaba/GraphScope/pull/3415
* fix(python): fixes the result validation for edge sets by sighingnow in https://github.com/alibaba/GraphScope/pull/3421
* fix(coordinator): Fix coordinator monitor bug by JackyYangPassion in https://github.com/alibaba/GraphScope/pull/3420
* chore: bump vineyard version to 0.18.2 by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3392
* fix(python): fixes the error handling of starting analytical engine process by sighingnow in https://github.com/alibaba/GraphScope/pull/3422
* ci: skip some GIE test cases due to the upgrade of vineyard by sighingnow in https://github.com/alibaba/GraphScope/pull/3424
* fix(interactive): Remove gcc option `-mno-avx512f` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3429
* feat(interactive): Import `VARCHAR` to GraphScope interactive by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3404
* refactor(interactive): Remove dummy config `incoming/outgoing_edge_strategy` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3434
* fix(interactive): Fix bugs of Groot Bulk Load When Retrying to Download Source Files by shirly121 in https://github.com/alibaba/GraphScope/pull/3438
* feat(analytical): Support extend label data for graph in eager mode. by SighingSnow in https://github.com/alibaba/GraphScope/pull/3288
* ci: bump the required glt version to fixes MacOS CI failure by sighingnow in https://github.com/alibaba/GraphScope/pull/3440
* feat(interactive): Add helm chart for Interactive by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3387
* fix(interactive): fix `isnull` in expression evaluation by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3423
* ci: fixes the destination dir of ossutil64 download for `helm repo index` by sighingnow in https://github.com/alibaba/GraphScope/pull/3445
* chore: probing analytical engine in coordinator shouldn't be warnings by sighingnow in https://github.com/alibaba/GraphScope/pull/3446
* feat(interactive): Introduce `graph_algo` as another example graph by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3344
* feat(interactive): Pegasus return status codes of rpc by lnfjpt in https://github.com/alibaba/GraphScope/pull/3427
* fix(interactive): Support using interactive on macos by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3416
* feat(interactive): Implemented memory-only mode for storage of Interactive. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3444
* fix(interactive): Fix bugs When Counting on Multiple Columns by shirly121 in https://github.com/alibaba/GraphScope/pull/3442
* fix(interactive): Convert Label Condition to Type Constraint by shirly121 in https://github.com/alibaba/GraphScope/pull/3441
* fix(interactive): Fix the Invalid cross-device link error occurring with copy_file_range in the Docker environment. by liulx20 in https://github.com/alibaba/GraphScope/pull/3454
* fix(interactive): Fix bug in `ODPSFragmentLoader` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3460
* fix(interactive): fix dead lock caused by write lock by lnfjpt in https://github.com/alibaba/GraphScope/pull/3459
* fix(interactive): fix test error of groot and unsafe static reference by longbinlai in https://github.com/alibaba/GraphScope/pull/3469
* refactor(interactive): revert encoding of groot by longbinlai in https://github.com/alibaba/GraphScope/pull/3471
* feat(interactive): Introduce Graph Type Inference in Compiler by shirly121 in https://github.com/alibaba/GraphScope/pull/3375
* refactor(flex): Refactor CMakeLists.txt to support be imported by find_package by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3478
* feat(interactive): Support secondary instance of groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3479
* fix(analytical): fix the louvain algorithm that the result is not consistent with 183 by acezen in https://github.com/alibaba/GraphScope/pull/3472
* feat(flex): Implemented compaction. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3482
* feat(flex): Implemented basic support for hugepages. by luoxiaojian in https://github.com/alibaba/GraphScope/pull/3489
* fix(analytical): Fix vineyard image is not set correctly by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3491
* fix(analytical): Fix the rapidjson find_package by acezen in https://github.com/alibaba/GraphScope/pull/3492
* fix(python): prevent the GAE been blocked if GIE failed to generate subgraphs during subgraph queries by sighingnow in https://github.com/alibaba/GraphScope/pull/3499
* fix(core): enable date and time types support in vineyard-graph (and GIE) by sighingnow in https://github.com/alibaba/GraphScope/pull/3498
* Try fixes macos nightly CI by sighingnow in https://github.com/alibaba/GraphScope/pull/3502
* fix(interactive): fix a bug in runtime for edge filtering by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3494
* fix(interactive): Remove assert & unwrap used in pegasus by lnfjpt in https://github.com/alibaba/GraphScope/pull/3488
* ci: Disable to publish images and packages on the aarch64 platform by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3503

New Contributors
* bufapiqi made their first contribution in https://github.com/alibaba/GraphScope/pull/3357
* JackyYangPassion made their first contribution in https://github.com/alibaba/GraphScope/pull/3389
* SighingSnow made their first contribution in https://github.com/alibaba/GraphScope/pull/3288

**Full Changelog**: https://github.com/alibaba/GraphScope/compare/v0.25.0...v0.26.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.25.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), FLEX Interactive, and bug fix.

**1. Interactive under GraphScope Flex Architecture**

- Release [GraphScope Interactive](https://graphscope.io/docs/latest/flex/interactive_intro).
- Enrich the movie graph schema and example query statements.
- Implement a query cache to avoid recompiling the same adhoc query.
- Support using the property of string type as primary key for vertex.

**2. Enhancements for GIE**

- Support get disk status in groot， user could retrieve the total space and free space of the store pod via Python/Java client. Example API is [here](https://github.com/alibaba/GraphScope/blob/8235b2909e642e15e7d668b7cc579fbd5268c9da/python/graphscope/client/connection.py#L180)
- Enhancements to count(): We've implemented two primary optimizations to accelerate the count() operation:
- Optimized Counting of Vertices/Edges: Operations like g.V().count() that previously required a complete enumeration of vertices followed by a count, are now streamlined into a single fused operator. This operator efficiently counts vertices or edges in tandem with their iteration.
- Improved Worker Utilization: The prior count() implementation was limited to using a single worker per machine for counting tasks, irrespective of the available configured workers. Now, count() operations are distributed across all configured workers, leveraging parallel processing to boost performance.
- Users can now interact with GIE using [natural language](https://graphscope.io/docs/latest/interactive_engine/neo4j/llm_assistant). We've integrated the capability to call OpenAI-compatible APIs within GIE, enabling the translation of human language inputs into Cypher queries. This advancement allows for a more intuitive and accessible query experience. For instance, users can utilize our Python API to execute the following code:

python
from graphscope.langchain_prompt.query import query_to_cypher
endpoint = "https://xxx" # use your endpoint
api_key = "xxx" replace to your own api key
question = "贾宝玉是谁的儿子?"
cypher_sentence = query_to_cypher(graph, question, endpoint=endpoint, api_key=api_key)
print(cypher_sentence)


Docker Image
--------------

shell
Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.25.0

Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.25.0

Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.25.0
Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.25.0

Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.25.0

GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.25.0

Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest


What's Changed
* refactor(interactive): Generate Stored Procedure Config Automatically in Compiler by shirly121 in https://github.com/alibaba/GraphScope/pull/3140
* feat(interactive): Support `length()` Operator in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/3166
* fix(interactive): Fix bug in computing path length in GIE by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3172
* ci: Support to build GraphScope Images(both x86 and arm64) in CI workflow by FLYLX in https://github.com/alibaba/GraphScope/pull/3152
* ci: fixes failure of building images by adding actions/checkout step by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3174
* ci: Pre-release v0.25.0 by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3175
* chore: fixes the link for the flex directory by sighingnow in https://github.com/alibaba/GraphScope/pull/3176
* fix(interactive): Fix the bug in the RexNode to Proto Conversion by shirly121 in https://github.com/alibaba/GraphScope/pull/3171
* feat(interactive): Support `[]` Operator in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/3168
* refactor(interactive): Parse Graph Schema from Yaml Configuration by shirly121 in https://github.com/alibaba/GraphScope/pull/3145
* feat(python): Refactor coordinator for operator by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3148
* fix(interactive): don't require openssl-sys during building GIE by sighingnow in https://github.com/alibaba/GraphScope/pull/3181
* feat(interactive): Enable clear properties for groot node by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3180
* refactor(interactive): Support Nested Branch Logical Plan by MeloYang05 in https://github.com/alibaba/GraphScope/pull/3151
* refactor(interactive): Support `head(collect(XX))` in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/3178
* fix(python): fix default field in py311 by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3183
* feat(interactive): Support `IS_NULL` and `IS_NOT_NULL` in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/3167
* feat(interactive): Support Unfold step in GIE by Louyk14 in https://github.com/alibaba/GraphScope/pull/3143
* remove google-java-format.jar by Louyk14 in https://github.com/alibaba/GraphScope/pull/3196
* feat(interactive): Add schema management ability for groot java sdk by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3193
* refactor(interactive): Support DateFormats in GIE Runtime by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3197
* fix(interactive): Fix bugs of Chinese Encoding by shirly121 in https://github.com/alibaba/GraphScope/pull/3195
* refactor(flex): Replace the Adhoc csv reader with Arrow CSV reader by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3154
* refactor(interactive): Support `Date` Type and `EXTRACT` Operator in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/3169
* ci: Update pr-check.yml by yecol in https://github.com/alibaba/GraphScope/pull/3207
* feat(interactive): support project properties of a `path` by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3213
* feat(interactive): Enhancement pieces for groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3217
* fix(interactive): Fix bugs in Label Name to Id Conversion by shirly121 in https://github.com/alibaba/GraphScope/pull/3221
* feat(interactive): Support LDBC SNB IC1,4,7,10 for HQPS Engine by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3147
* chore: don't reinstall (may trigger upgrade) if libarrow-dev has already been installed by sighingnow in https://github.com/alibaba/GraphScope/pull/3241
* ci: fixes the Python dependency error in CI by sighingnow in https://github.com/alibaba/GraphScope/pull/3242
* fix(interactive): Not requiring eid in update/delete edges in groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3240
* fix(python): default session needs to be aware of g.set_option(num_workers=...) by sighingnow in https://github.com/alibaba/GraphScope/pull/3246
* refactor(interactive): Refactor the user interface of `Interactive` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3201
* feat(interactive): support unfold path entry in GIE Runtime by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3236
* fix(interactive): Fix several bugs in data-load-tools by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3249
* fix(interactive): Fix NPE when update/delete vertices by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3250
* build(interactive): Reduce image size by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3247
* fix(interactive): Support fusion of scan+count in GIE Runtime for optimization by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3233
* fix(flex): Support set edge strategy on each `vertex_type_pair_relation` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3257
* feat(learning): Integrate GLTorch into GraphScope by LiSu in https://github.com/alibaba/GraphScope/pull/3230
* feat(interactive): Support `labels` and `type` Functions in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/3255
* build(learning): bump torch from 1.13 to 1.13.1 in /python by dependabot in https://github.com/alibaba/GraphScope/pull/3264
* refactor(interactive): Introduce a new test set for Interactive by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3251
* docs(interactive): Add docs for GraphScope Interactive by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3267
* fix: Flex add rt_bench and fix some bug by liulx20 in https://github.com/alibaba/GraphScope/pull/3268
* chore: Enable retrieve disk status of store node from client by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3274
* fix(interactive): clear download path when start store service by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3276
* ci: requires macos 12 as homebrew (and apple) do not provide support for this old version by sighingnow in https://github.com/alibaba/GraphScope/pull/3278
* fix(interactive): Fix Implicit Type Conversion Issues in HQPS by shirly121 in https://github.com/alibaba/GraphScope/pull/3256
* fix(interactive): block service until graph schema is synced by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3280
* chore: Increase the disk size of minikube cluster by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3283
* fix(interactive): Increase ingestor queue buffer size by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3284
* fix(analytical): reduce louvain memory usage by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3273
* ci: fixes the check of libarrow-dev existence on Ubuntu by sighingnow in https://github.com/alibaba/GraphScope/pull/3291
* ci: fixes the handling the pytorch dependency by sighingnow in https://github.com/alibaba/GraphScope/pull/3290
* ci: increasing the default timeout setting for pip install by sighingnow in https://github.com/alibaba/GraphScope/pull/3292
* ci: free disk spaces and install correct torch without download many from pypi by sighingnow in https://github.com/alibaba/GraphScope/pull/3293
* ci: run test cases in subprocess with pytest-sdist by sighingnow in https://github.com/alibaba/GraphScope/pull/3294
* docs: fixes the filename of loading graphs page by sighingnow in https://github.com/alibaba/GraphScope/pull/3295
* feat(interactive): support extract intervals from temporal types in GIE by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3229
* fix(interactive): Support Index predicate for dynamic params in IR-Core by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3282
* fix(interactive): Support Index predicate with multiple expected values given via P.within() by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3300
* ci: macos: make sure $HOME/.local/bin present in PATH by sighingnow in https://github.com/alibaba/GraphScope/pull/3302
* refactor(coordinator): and gsctl to support FLEX interactive by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3297
* fix(interactive): Fix Bugs in Dynamic Params by shirly121 in https://github.com/alibaba/GraphScope/pull/3279
* fix(interactive): Distinguish 'label equals' or 'unique key equals' from Filter Conditions by shirly121 in https://github.com/alibaba/GraphScope/pull/3285
* fix(interactive): Introduce Cypher CI Tests for GIE on Experimental Store by shirly121 in https://github.com/alibaba/GraphScope/pull/3303
* feat(flex): Support using property of string type as primary key for vertex by liulx20 in https://github.com/alibaba/GraphScope/pull/3296
* chore: use target instead of name resolver factory by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3306
* fix(interactive): Update example movie graph and fix some bug by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3304
* feat(interactive): Add groot charts that start all components within one pod by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3308
* fix(interactive): Fix the problem of not returning properties together when returning a vertex/edge. by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3298
* feat(interactive): Reuse compiled native lib for same physical plan by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3301
* feat(interactive): Support Regex Match for String Values by shirly121 in https://github.com/alibaba/GraphScope/pull/3286
* fix(interactive): Ad-hoc Handling for Connection Validation in Gremlin by shirly121 in https://github.com/alibaba/GraphScope/pull/3315
* fix(interactive): fix bug in release cstr pointer in compiler by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3307
* fix(interactive): Fix Bugs in Converting Match into Join by shirly121 in https://github.com/alibaba/GraphScope/pull/3311
* fix(interactive): fix parallel scan in GIE by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3317
* feat: Enabling Natural Language Graph Queries in GraphScope with chatGPT 2648 by Suchun-sv in https://github.com/alibaba/GraphScope/pull/3271
* [doc] add llm doc index by longbinlai in https://github.com/alibaba/GraphScope/pull/3319
* fix(interactive): Implement Field Trimmer Rule in Compiler by JialuGong in https://github.com/alibaba/GraphScope/pull/3200
* feat(interactive): Fix Type Inference in 'ARRAY_VALUE_CONSTRUCTOR' and 'MAP_VALUE_CONSTRUCTOR' by shirly121 in https://github.com/alibaba/GraphScope/pull/3287
* fix(interactive): Fuse Source With Count to Optimize in Gremlin by shirly121 in https://github.com/alibaba/GraphScope/pull/3320
* fix(interactive): implement count with efficiency by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3326
* fix(interactive): download to a tmp file then move by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3331
* ci: explicitly set python version on MacOS CI by sighingnow in https://github.com/alibaba/GraphScope/pull/3329
* ci: fixes build failure with apache-arrow 14 by sighingnow in https://github.com/alibaba/GraphScope/pull/3333
* fix(interactive): Make Pegasus Related Settings Configurable Per Query by shirly121 in https://github.com/alibaba/GraphScope/pull/3321
* fix(interactive): Introduce `Degree Fusion` RBO Rules by shirly121 in https://github.com/alibaba/GraphScope/pull/3312
* feat(interactive): Support cancel in interactive by lnfjpt in https://github.com/alibaba/GraphScope/pull/3310
* fix(interactive): Fix a bug for CSVFragmentLoader by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3336
* ci: Fixes release failure of groot image by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3337

New Contributors
* Suchun-sv made their first contribution in https://github.com/alibaba/GraphScope/pull/3271
* JialuGong made their first contribution in https://github.com/alibaba/GraphScope/pull/3200

**Full Changelog**: https://github.com/alibaba/GraphScope/compare/v0.24.0...v0.25.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.24.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), Learning Engine(GLE), and Deployment.

We highlight the following improvements included in this release:

**1. Enhancements for GIE**

Key Features:
- Support the `sample()` step of Gremlin to randomly pick up a given number of traversers from a Gremlin traversal. Additionally, `coin()` step can now follow any step in a Gremlin traversal, instead of just `V()` and `E()`. For example,
groovy
g.V().sample(10) // Sample 10 traversers from `g.V()`
g.V().out().coin(0.2) // Randomly pick up 20% among all vertices' out neighbors

- Support the `union` step of Gremlin to combine the traversers from multiple traversals. For example,
groovy
g.V().union(out(), out().out()) // the results would now be one-hop (out once) and two-hop (out twice) neighbors of all vertices.

- Support the `unfold` step of Gremlin, which is a reversed operator of `fold` that flattens a collection of traversers into discrete elements. For example,
groovy
g.V().fold().unfold() // must obtain `g.V()` again

- Support the `identity()` step of Gremlin, which simply returns the current traversers, and is often used together with `union` step. For example,
groovy
g.V().union(identity(), out()) // return `V()` and its one-hop neighbors

- Support `isNull` in the `expr()` for verifying wether a certain property has `NULL` value. For example,
groovy
g.V().where(expr(".age isNull")) // the vertices without the field of `age` will be pruned


But fixes:
- Fix a bug of `count()` step followed by `order()` and `limit()` that can cause the `gaia` engine to abort unexpectedly. Now the following Gremlin query can be executed smoothly:
groovy
g.V().order().by('id', asc).limit(1).out().count()



**2. Enhancements for Deployment**

- All GraphScope Images support both `x86` and `arm64` platform
- This release includes the gsctl command-line utility for building and testing GraphScope. The gsctl tool provides several commands and options to streamline the development and deployment process. You can use the following command to install the graphscope-client package and then use gsctl:

bash
$ pip3 install graphscope-client
$ gsctl --help



**3. Other enhancements and bug fixes**

- Reduced the size of GRAPE-java shaded jar,reduce shaded jars' total size from 94MB to 58MB.
- Introduce Hiactor-based High-QPS Engine for Flex.
- Refine the schema definition for flex `rt_mutable_graph`
- Fix the `Py_None` reference count issue in GLE
- Fix the bug that GLE client only connects with a single server in distributed training
- Update the GLE-related tutorials

Docker Image
--------------

shell
Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.24.0

Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.24.0

Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.24.0
Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.24.0

Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.24.0

GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.24.0

Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest


What's Changed
* Fixes the docs generation for tagged releases by sighingnow in https://github.com/alibaba/GraphScope/pull/2970
* Fixes compliation error for non-builtin frames on MacOS by sighingnow in https://github.com/alibaba/GraphScope/pull/2972
* Fix memory leak in groot and add trace logs by siyuan0322 in https://github.com/alibaba/GraphScope/pull/2971
* Refine GC related codes in groot by siyuan0322 in https://github.com/alibaba/GraphScope/pull/2973
* Bump grpc-protobuf from 1.47.0 to 1.53.0 in /analytical_engine/java by dependabot in https://github.com/alibaba/GraphScope/pull/2977
* Bump up the version of kubernetes-log-export-action to v5 by dashanji in https://github.com/alibaba/GraphScope/pull/2976
* [GIE Compiler] Set Cypher Query Timeout from System Configurations by shirly121 in https://github.com/alibaba/GraphScope/pull/2975
* Support cypher query in python by andydiwenzhu in https://github.com/alibaba/GraphScope/pull/2952
* [Coordinator] Fix issue 2734: unifying timeout parameter in coordinators by TaoLbr1993 in https://github.com/alibaba/GraphScope/pull/2915
* [Bug Fix] Fix bugs of time counter in gremlin service by shirly121 in https://github.com/alibaba/GraphScope/pull/2906
* [GIE Compiler] Support Procedure Call in Cypher Queries by shirly121 in https://github.com/alibaba/GraphScope/pull/2927
* Pre-release 0.24.0 by lidongze0629 in https://github.com/alibaba/GraphScope/pull/2986
* Make GraphPlanner able to accept file as input by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/2980
* [Flex] Some modification on Flex code by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/2990
* [GIE] Introduce Aggregate.First proto definition by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/2991
* Update GLE tutorials and bump GLE version by LiSu in https://github.com/alibaba/GraphScope/pull/2983
* [GIE] Fix unbalanced data routing in DefaultRouter in GIE by BingqingLyu in https://github.com/alibaba/GraphScope/pull/2994
* Use arrow::LargeStringArray in trivial_tensor_t to store data larger than 2GB by acezen in https://github.com/alibaba/GraphScope/pull/2999
* Revised groot-client and add release phrase by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3000
* Retry download when checksum failed in groot data-loading process by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3006
* [Flex] Upgrade `libgrape-lite`version by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3004
* Add deprecated decorator and apply it on gremlin by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3007
* [GIE Doc] Refine Docs for Cypher by shirly121 in https://github.com/alibaba/GraphScope/pull/2995
* [GIE] Add LSQB QuerySet into Benchmark Tool by BingqingLyu in https://github.com/alibaba/GraphScope/pull/2997
* [GIE Compiler] Make Config path Configurable from User Given Parameters by shirly121 in https://github.com/alibaba/GraphScope/pull/2982
* unit test for pegasus timeout by lnfjpt in https://github.com/alibaba/GraphScope/pull/2964
* [GIE Test] Add Timeout CI Test in Python by shirly121 in https://github.com/alibaba/GraphScope/pull/2989
* Fix figure display issues in GIE LDBC tutorial by MeloYang05 in https://github.com/alibaba/GraphScope/pull/3009
* update libgrape-lite's commit id in Flex dockerfile by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3016
* [BugFix] Fix bug when process pk scan of multiple labels by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3013
* [Flex] Update flex dockerfile by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3020
* [Flex] Introducing Hiactor-based HQPS Engine into flex by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/2981
* Upgrade to latest vineyard (v0.16.1) by sighingnow in https://github.com/alibaba/GraphScope/pull/3019
* Enable perfect hash support in GAE by sighingnow in https://github.com/alibaba/GraphScope/pull/2959
* Fixes the Python version detection in MacOS CI by sighingnow in https://github.com/alibaba/GraphScope/pull/3022
* Update README.md by yecol in https://github.com/alibaba/GraphScope/pull/3027
* [Flex] Introducing Hiactor-based HQPS Engine into flex by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3024
* Fixed jdk_1.8 version for data-loading-tool by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3031
* unify gae, python and groot proto into proto directory by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3023
* Implement grin interface for flex by liulx20 in https://github.com/alibaba/GraphScope/pull/3010
* ci: Add check for conventional commit and docs by yecol in https://github.com/alibaba/GraphScope/pull/3033
* [GIE Compiler] Introduce LDBC CI Tests for Cypher Query by shirly121 in https://github.com/alibaba/GraphScope/pull/2984
* Update README.md to reflect GraphScope Flex by wenyuanyu in https://github.com/alibaba/GraphScope/pull/3035
* doc: Update README.md by adding latest news section by yecol in https://github.com/alibaba/GraphScope/pull/3036
* revise ldbc apps by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3034
* [Bug Fix] Fix Incremental Query Id Issues in Multiple Frontend by shirly121 in https://github.com/alibaba/GraphScope/pull/3026
* Fixes the implementation of adjlist in flatten fragment by sighingnow in https://github.com/alibaba/GraphScope/pull/3042
* Fixes resolve protobuf library on MacOS. by sighingnow in https://github.com/alibaba/GraphScope/pull/3043
* Propogate error message about argument mismatch to Python client by sighingnow in https://github.com/alibaba/GraphScope/pull/3044
* Fixes MacOS CI for UDFs by correcting the library path using install_name_tool by sighingnow in https://github.com/alibaba/GraphScope/pull/3045
* [BugFix] Fix networkx add edge segment fault bug by acezen in https://github.com/alibaba/GraphScope/pull/3032
* Add the performance & tunning page for GAE by sighingnow in https://github.com/alibaba/GraphScope/pull/3046
* Integrate sdk-common into common for simplisity by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3050
* fix: make LDBC metadata consistent in experiment store with other storages by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3048
* ci: add conventional checks for PR title and docs by yecol in https://github.com/alibaba/GraphScope/pull/3054
* fix: Fixes error in docs generation by removing the submodule by sighingnow in https://github.com/alibaba/GraphScope/pull/3056
* fix: Fix Incorrect Hyperlinks in GIE Doc `supported_gremlin_steps.md` by shirly121 in https://github.com/alibaba/GraphScope/pull/3057
* feat(analytical): Add the Graph.consolidate_columns() interface in Python client by sighingnow in https://github.com/alibaba/GraphScope/pull/3060
* feat(analytical): support use int32_t as vid_t to save memory for small graphs by sighingnow in https://github.com/alibaba/GraphScope/pull/3063
* refactor(analytical): Remove `nativeLibLoader` in GRAPE-jdk by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3065
* fix(analytical): fixes wrong result when flatten from projected arrow fragment by hanjf12 in https://github.com/alibaba/GraphScope/pull/3066
* ci: address the failure of packaging graphscope wheels on MacOS by sighingnow in https://github.com/alibaba/GraphScope/pull/3068
* Add config to enable multiple frontend support in groot deployment by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3073
* ci: Add GitHub workflow to build graphscope-dev:wheel image by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3072
* docs: add loading graphs doc by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3075
* ci: fixes workflow of building GraphScope wheel image by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3083
* ci(flex): Add flex in pr-check's scopes by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3084
* feat(analytical): Enable cython app on local vertex map graph by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3081
* chore: Refine the logic to probe GRAPHSCOPE_HOME by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3082
* refactor(analytical): Reduce shaded jar size. by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3076
* fix(interactive): fix unexpected results for some group().by() queries by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3099
* docs: add a note about the pip version requirements by sighingnow in https://github.com/alibaba/GraphScope/pull/3105
* fix(k8s): fixes the dockerfile to download kubectl using correct arch by sighingnow in https://github.com/alibaba/GraphScope/pull/3104
* fix(interactive): fix a bug of count operator by lnfjpt in https://github.com/alibaba/GraphScope/pull/3067
* fix(interactive): Fix Examples in GIE Doc by shirly121 in https://github.com/alibaba/GraphScope/pull/3108
* feat(interactive): Refine the GrootClient for concurrent and async write. by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3113
* fix(learning): bump GLE version to include the latest bug fix by LiSu in https://github.com/alibaba/GraphScope/pull/3115
* refactor(flex): Refine the schema definition of `rt_mutable_graph` by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3079
* chore: bump up vineyard version to v0.16.4 by sighingnow in https://github.com/alibaba/GraphScope/pull/3106
* chore: bump up gremlinpython to 3.7.0 by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3119
* refactor(interactive): Join operator in pegasus will repartition stream before join by lnfjpt in https://github.com/alibaba/GraphScope/pull/3118
* Update README.md to fix a spell error by xiezheng-XD in https://github.com/alibaba/GraphScope/pull/3123
* ci: fixes doc generation failure on CI by sighingnow in https://github.com/alibaba/GraphScope/pull/3125
* feat(interactive): Implement GraphScope Interactive by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3030
* fix(interactive): throw unsupported error when query `PathExpand()` within subtask by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3133
* chore: allow user pass variables for substitutions by siyuan0322 in https://github.com/alibaba/GraphScope/pull/3135
* fix(flex): Fix hqps engine's support for some simple `MATCH` queries by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3086
* fix(interactive): fix the docs of interactive engine by longbinlai in https://github.com/alibaba/GraphScope/pull/3131
* feat(interactive): support isNull expression evaluation in GIE by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3128
* feat(interactive): Support Identity step in GIE by Louyk14 in https://github.com/alibaba/GraphScope/pull/3111
* fix(interactive): fix bug in unfold case `g.V().fold().as("a").unfold()` in GIE Runtime by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3130
* feat(interactive): Support `Sample()` and `Coin()` steps in GIE by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3091
* refactor: Upgrade the build phase of image in CI with gsctl by FLYLX in https://github.com/alibaba/GraphScope/pull/3142
* fix(interactive): Support Multiple Matches in IR Core by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3134
* refactor(flex): Small modifications to the graph schema to fit the GIE compiler. by zhanglei1949 in https://github.com/alibaba/GraphScope/pull/3150
* refactor(interactive): Initiate Compiler Service with Yaml Configuration by shirly121 in https://github.com/alibaba/GraphScope/pull/3141
* ci: Support to build dev images(both x86 and aarch64) in CI workflow by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3153
* fix(interactive): fixes some unsafe rust code that cannot be compiled with recent stable rust compilers by sighingnow in https://github.com/alibaba/GraphScope/pull/3160
* refactor: Upgrade gsctl and modify files associated with gsctl by FLYLX in https://github.com/alibaba/GraphScope/pull/3149
* ci: fixes failure in building GraphScope wheel package by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3164
* feat(interactive): Support Optional Match and Where Not(subquery) in Compiler by shirly121 in https://github.com/alibaba/GraphScope/pull/3088
* refactor(interactive): refactor the implementation of `PathExpand` by BingqingLyu in https://github.com/alibaba/GraphScope/pull/3159
* Differentiating between arm64 and x86 wheel's name by lidongze0629 in https://github.com/alibaba/GraphScope/pull/3165

New Contributors
* hanjf12 made their first contribution in https://github.com/alibaba/GraphScope/pull/3066
* xiezheng-XD made their first contribution in https://github.com/alibaba/GraphScope/pull/3123
* Louyk14 made their first contribution in https://github.com/alibaba/GraphScope/pull/3111
* FLYLX made their first contribution in https://github.com/alibaba/GraphScope/pull/3142

**Full Changelog**: https://github.com/alibaba/GraphScope/compare/v0.23.0...v0.24.0

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.23.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), GraphScope Flex, and Deployment.

We highlight the following improvements included in this release:

**1. GraphScope Flex Technical Preview**

GraphScope Flex represents the ongoing evolution of GraphScope. In this release, we're excited to introduce a technical preview of GraphScope Flex. It highlights a modular design that reduces resource and cost requirements while providing a seamless, user-friendly experience for flexible deployment. It's currently under active development, and we look forward to your feedback.

Key Features:
- **Modular Design**: Assemble your stack much like LEGO blocks to customize your graph computing deployments.
- **Three-Layer Architecture**: Components are organized into an application layer, execution layer, and storage layer.
- **Flexible Builds and Deployments**: Use the `flexbuild` script to build a custom deployment tailored for your specific use case.

You can explore GraphScope Flex in the `flex/` directory or through the released artifacts
- `graphscope_flex_db_cppsp_hiactor_mcsr`,
- `graphscope_flex_olap_builtin_grape-cpu`, and
- `graphscope_flex_gnn_gnnmodels_graphlearn_tensorflow_vineyard.so`

packages for high-QPS interactive queries, graph analytics, and graph learning task use cases, respectively. Dive in and discover what GraphScope Flex has to offer!

**2. Enhancements for GIE**

- Support the recording of both vertices and edges during path expansion. In the past, path expansion had the option to yield either all vertices or the end vertex of the path. However, due to the requirements of both Gremlin and our users, it's essential to also retrieve the edges of the path, besides the vertices. To activate this feature, use `with('RESULT_OPT', 'ALL_V_E')` in the path expansion syntactic sugar. Here is an example:

gremlin> g.V().out("1..3", "knows").with('RESULT_OPT', 'ALL_V_E')
==>[v[1], e[0][1-knows->2], v[2]]
==>[v[1], e[2][1-knows->4], v[4]]

- We are now happy to introduce the capability of querying Cypher in GIE, by integrating Neo4j's bolt service in our system. Please follow the [guide](https://graphscope.io/docs/latest/interactive_engine/neo4j/cypher_sdk) to enable bolt service (only on local) for querying with Cypher. We also attempt to make the syntax of Cypher as close as [openCypher](https://opencypher.org/), and the details of our support for Cypher can be found [here](https://graphscope.io/docs/latest/interactive_engine/neo4j/supported_cypher).

**3. Other enhancements and bug fixes**

- GAE Java
- Fix VertexSet's problem of supporting vertex_id in java long.
- Add Grape-GraphX performance report
- Fix the problem of installing grape-jdk locally.

Docker Image
--------------

shell
Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.23.0

Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.23.0

Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.23.0
Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.23.0

Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.23.0

GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.23.0

Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest


What's Changed

- [7dcf1950b]: Change the version to v0.23.0 (2969) (Dongze Li)
- [731629732]: Introduce flexbuild and a readme to flex. (2954) (luoxiaojian)
- [d8ad6ce6c]: Install jdk 11 on MacOS CI and fixes the GAE link failure in nightly CI on MacOS (2965) (Tao He)
- [c27f45902]: Fixes two compliation warnings (2968) (Tao He)
- [bb1921eca]: Update the elementMap-step's doc of GIE (2966) (Dongze Li)
- [cb27df481]: Fixes nightly CI error for Arm Linux wheels (2963) (Tao He)
- [69258d995]: Fixes the "illegal thread local variable reference to regular symbol" error on macOS for GLE (2962) (Tao He)
- [1e2c8f62a]: [GIE Compiler] Unify Gremlin Timeout Configurations (2953) (Xiaoli Zhou)
- [46949638e]: Add setuptools to client requirement in order to use pyproject based build (2958) (Siyuan Zhang)
- [7fc14b5af]: [GIE Compiler] Support Case When Expression in Logical and Physical Plan (2918) (Xiaoli Zhou)
- [b82b74883]: Avoid implicit int to enum conversion, causing the -fpermissive error (2957) (Siyuan Zhang)
- [ad4aea19d]: Use the correct delete in free_graph_handle (2955) (Tao He)
- [21fd64430]: Sort the server list before iterate to keep consistent between workers (2956) (Tao He)
- [502e41030]: Parallelize the PrepareToRun implementation in projected fragment (2949) (Tao He)
- [5ffad661f]: Add two more pagerank variants (2950) (Siyuan Zhang)
- [a22247999]: Use pagerank_local_parallel rather than pagerank_local for better performance (2945) (Tao He)
- [92da63887]: Refine CI process (2948) (Siyuan Zhang)
- [555b13de8]: add figures for GraphScope Flex. (2946) (Jingbo Xu)
- [7035c0a44]: ElementMap step support (2942) (Dongze Li)
- [b6402e708]: cluster_builder.py: add missing copyright header and remove unused import lines (2944) (Tao He)
- [5b852ffe1]: Docs preview is only available for pull requests from the orignal repo, not forks (2943) (Tao He)
- [cab8cde5e]: Add GLE quick start example in the overview getting started page (2939) (LiSu)
- [badf3f3be]: hotfix: pull_request_target issue. (2941) (Jingbo Xu)
- [08f8d0f24]: Disable JAVA SDK (with a warning) when JNI not found (2937) (Tao He)
- [f543cd29e]: Improve the `deploy_with_existing_vineyard_cluster` doc for easy understanding (2930) (Ye Cao)
- [ae8d96c7d]: Show explicit error message for SSSP run over a graph with no edge weight (2929) (Lei Wang)
- [50c7a1e21]: Pin vineyard python version to 0.15.0 (2933) (Siyuan Zhang)
- [25496f8d5]: Connect to frontend directly, bypass coordinator for GIE query (2923) (Siyuan Zhang)
- [96d50939d]: [GIE Compiler] Parse Label Id to Name in `label` Operator (2928) (Xiaoli Zhou)
- [34b365028]: [GAE-Java] Use `FFI-bitset` rather than java BitSet for VertexSet backend (2926) (Zhang Lei)
- [f0a779e88]: [good-first-issue]Move `save_to/load_from` from client side to coordinator (2917) (Zhang Lei)
- [c1007cff5]: [GIE Compiler] Introduce cypher service to accept queries from neo4j ecosystem (2848) (Xiaoli Zhou)
- [7637aafb5]: [GIE] Refine FFI interface `build_physical_plan` (2911) (Xiaoli Zhou)
- [d0dc2b7c4]: Print metrics of application running time (2908) (Siyuan Zhang)
- [628773d80]: [Bug Fix] Fix bug of null pointer exception in `distinct` operator (2901) (Xiaoli Zhou)
- [0a44e711e]: Fixes the misuse of "continue-on-error" and "fail_ci_if_error". (2904) (Tao He)
- [612bd7f09]: Upload coverage: continue on error to avoid false negative on main branch (2903) (Tao He)
- [513155c30]: Upgrade the graphlearn submodule to fixes nightly CI, refactor the Makefile for early error exit (2902) (Tao He)
- [bd7680f2c]: Use CONFIG mode to find protobuf to address the absl dependency issue. (2899) (Tao He)
- [cf81c7b80]: [GAE-Java] Relax version enforce for some mvn dependencies (2894) (Zhang Lei)
- [b37ab9031]: [GIE Compiler] Add data type of algebra operator output to physical plan (2870) (Xiaoli Zhou)
- [79cd5770b]: Use local metadata backend to avoid the requirements of etcd for local sessions (2889) (Tao He)
- [7ef0e4d14]: fix munew line are not rendered correclty when multiple params are given (2891) (Siyuan Zhang)
- [fbf40f6dd]: Update the pre-genereated unload_op when the op itself been deepcopied (2888) (Tao He)
- [32089f285]: [Grape-Java] Add `grape-graphx` performance.md (2886) (Zhang Lei)
- [e4f1f23a3]: Allow pass params to gie instance (2885) (Siyuan Zhang)
- [17819c81f]: [BugFix] Fix bug in endV() after PathExpand (2881) (BingqingLyu)
- [a18836d5c]: [GIE Compiler] Support dynamic query params in cypher queries (2874) (Xiaoli Zhou)
- [ce3e16b2e]: [GAE/tests] Add GAE tests for FLASH algorithms of bfs and cc (2869) (Longbin Lai)
- [5a63fafa0]: upload log should not fail the CI (2879) (Siyuan Zhang)
- [559b084b6]: Direct the content of log files to stdout of pods in kubernetes deployment (2871) (Siyuan Zhang)
- [4af5689be]: add initial version of command-line tool `gsctl`. (2868) (Jingbo Xu)
- [7a3e06b56]: [GIE Doc] Update Doc for `PathExpand` Step Usage in GIE (2877) (BingqingLyu)
- [387f915f6]: Fixes the docs generation for tagged releases (2878) (Tao He)
- [c489c6381]: [BugFix] Make the error message truncated not so aggressive (2876) (Weibin Zeng)
- [e9af91a91]: [GIE] Support `PathExpand` with `OPT=ALL_V_E` in GIE (2841) (BingqingLyu)
- [37738891e]: Add a initializeCommand to `devcontainer.json`: always pull newer image before create container (2873) (Weibin Zeng)
- [086fa6362]: [GIE/docs] reorganize gremlin docs (2862) (Longbin Lai)
- [bd8952093]: Allow triggering workflow dispatch event for CIs (2865) (Ke Meng)
- [a0b36eeef]: [GIE Doc] Add examples to collect gremlin results in a streaming way (2842) (Xiaoli Zhou)
- [5fd6cc173]: Fixes the inconsistent usage of msgpack/json in graph reporter and avoid protobuf message creation in __del__ (2843) (Tao He)
- [434acdfea]: Fix dns search error in groot (2850) (Siyuan Zhang)
- [9bf071cf3]: auto-gen docs of flex (2851) (ds-ssj)
- [6851cd9fc]: allow null value in data loading (2845) (Siyuan Zhang)
- [1ba61029e]: [GIE Compiler] Introduce `GraphPlanner` to unify logical and physical plan building (2838) (Xiaoli Zhou)
- [a8b9dce78]: [GAE-Java] Use maven-flatten-plugin to replace revision property for locally build&install (2840) (Zhang Lei)
- [3c4cd9d98]: Fix compile error for the flash bc algorithm on ubuntu (2839) (lixueclaire)
- [0de3063f1]: use -O3 instead of -O0 in customized algo compilation (2837) (Siyuan Zhang)
- [9a265dcd1]: [GIE/Runtime] Redesign `PartitionerInfo`, `ClusterInfo`, and `Router` trait to better support parallel processing in Runtime (2744) (BingqingLyu)
- [653823b88]: [GIE/tests] Add pattern matching test cases for GIE (2831) (lixueclaire)
- [30e45e0ed]: [GIE Compiler] Unify execution client which is used to send request to remote engine service (2818) (Xiaoli Zhou)
- [f46941564]: Remove unused ports in Groot deployment (2832) (Siyuan Zhang)
- [d75ba7b26]: Add coordinator as the Ownerreferces of frontend deployment. (2830) (Ye Cao)
- [ec218cf3d]: [GIE Compiler] Fix query params of getV base in path expand operator (2773) (Xiaoli Zhou)
- [e28faa991]: Change version to 0.23.0 for daily release (2828) (Dongze Li)
- [4535bc34b]: Fixes errors during macos M1 CI (2827) (Tao He)
- [9f2334c68]: [GIE/docs] Add gie glossary and quick start (2821) (Longbin Lai)

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.22.0 release. This release contains many important features and enhancements to Interactive Engine (GIE) and Deployment. Now the user can try GraphScope under the Linux Arm64 platform by `pip3 install graphscope`.

We highlight the following improvements included in this release:

**1. Enhancements for GIE**
- We have provided a utility for users to easily develop and test GIE on a local machine. Basically, users can run three parts of integration tests using the `gs` command line tool, including the official gremlin test, auxiliary pattern match test, and ldbc complex interactive workload on ldbc social network scale factor 1. In addition, users are allowed to locally start a frontend and an executor, where the former exposes an endpoint to easily integrate with the official gremlin console, and the latter is responsible for executing queries. Details can be found in the [documentation](https://graphscope.io/docs/latest/interactive_engine/dev_and_test).

**2. Enhancements for Deployment**
- User can try GraphScope under the **Linux Arm64** platform by `pip3 install graphscope`
- All images are now switched to ubuntu-based.
- Add a `devcontainer.json` for users to quickly get started.
- Use vineyardctl to install vineyard as the sidecar container.
- Support to deploy the engines of GraphScope on demand in the `lazy` mode.

**3. Other enhancements and bug fixes**
- Support to store and restore graphs from/to persistent storage on the kubernetes cluster. Details can be found in the guide: https://graphscope.io/docs/latest/deployment/persistent_storage_of_graphs_on_k8s.
- Add an end-to-end example about training over vineyard graphs using GLE
- Fixes the incorrect problem of results in python algorithm with PIE model under the session's parameter `num_workers=1`
- We have fixed a bug of GIE related to using `match` step with a `not`-existed edge, for which it should be executed via `anti join`, but it returned unexpected error "the first sentence of `MergedSentence` must have InnerJoin" instead.
groovy
g.V().match(
__.as("a").in().as("b"),
__.as("b").out().as("c"),
__.not(__.as("c").out().as("a")) `not`-exsited edge
)

Now such kind of query can be executed without issue.
- Fixes the bug of incorrect results when dedup edges on a partitioned graph in GIE, e.g.,
groovy
g.V().outE().dedup()

Now it can output expected results.

Docker Image
--------------

shell
Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.22.0

Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.22.0

Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.22.0
Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.22.0

Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.22.0

GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.22.0

Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest


What's Changed

- [99c9d15d2]: Fixes the GAE fork issue with openmpi (2820) (Tao He)
- [ba359967a]: [GLE Doc] complete the build and install details of learning engine (2815) (LiSu)
- [7af925532]: Fixes some failures in nightly CI (2817) (Tao He)
- [0b220536d]: Support to install deps for Alibaba Cloud Linux in the gs bash utility (2816) (Ye Cao)
- [28a360593]: Cleanup space in CI (2814) (Siyuan Zhang)
- [3f50d7d3e]: Update dev_guide.md: change the link to learning engine to a correct one. (2813) (Taoshu)
- [035036b2c]: disable debug-assert in groot (2811) (Siyuan Zhang)
- [57b22febf]: Groot schema now supports add comment (2802) (Siyuan Zhang)
- [43493ce11]: Fixes bug when building docs from pull requests (2810) (Tao He)
- [302b02877]: Support to backup and recover the specific graphs via vineyardctl (2771) (Ye Cao)
- [d49d0edb7]: docs: update rst-versions style, defaults default version to latest, and fixes the zh/en indexing (2803) (Tao He)
- [9efd4f672]: Add local python script path to PATH (2804) (Tao He)
- [9d5d968e7]: [Docs/GIE] add tinkerpop ecosystem (2784) (Longbin Lai)
- [278a24c79]: Adapt to latest vineyard for varint encoding in CSR (2791) (Tao He)
- [f1b216cc7]: Support users to only input the kubeconfig path for connecting the k8s cluster (2799) (Ye Cao)
- [33e300721]: Correct the version of built wheel (2798) (Siyuan Zhang)
- [191949b18]: Fixes the graphdef value when constructing graphs from vineyard_id. (2793) (Tao He)
- [699ffbe57]: [k8s]Support deploying vineyard cluster independently and deploy the engines on called (2710) (Ye Cao)
- [c1ef88582]: refine docs latest tag to make it ready as default. (2788) (Jingbo Xu)
- [e9090db36]: Refine doc for `dev and test` of analytical engine. (2787) (Jingbo Xu)
- [1e57650a0]: Read version number from the package in setup.cfg (2785) (Dongze Li)
- [09ead30e5]: [GLE docs] minor fixes (2783) (LiSu)
- [b014dad1f]: use kubectl cp --retries (2781) (Siyuan Zhang)
- [d2c7d2edb]: Fix typo in Makefile (2782) (Siyuan Zhang)
- [3606b857b]: Fixes the building failure of client wheel on Linux arm64 platform (2780) (Dongze Li)
- [fdc626ee4]: Use vineyardctl API to inject the vineyard sidecar (2612) (Ye Cao)
- [7fd430b3d]: [GraphProxy] refine implementation of LazyDetails (2766) (BingqingLyu)
- [4b24146d1]: [Bug Fix] fix the bug in implementation of `PartialEq` for `Edge` in GIE Runtime (2740) (BingqingLyu)
- [5b46a68bc]: Improve GetInnerVertex() by querying local fragment only (2772) (Songqing Zhang)
- [983a41e33]: Enable deploy groot on local environment (2769) (Siyuan Zhang)
- [2509958c4]: Make grape_engine help message more friendly (2774) (Siyuan Zhang)
- [e2943778e]: [GIE Dev Test] Refine GIE Dev and Test (2767) (Xiaoli Zhou)
- [bf8d7b096]: Generate pyi stubs for protobuf modules for better developer experience (2768) (Tao He)
- [6fc7db32f]: [Bug Fix] Fix the alias of `GetV` of `ExpandBase` in `PathExpand`, an… (2763) (BingqingLyu)
- [a119c23b4]: Correct the python sssp result by setting generate_eid to false in load_p2p_network dataset (2762) (Dongze Li)
- [e4bb06903]: Make sure grape-engine exit when CTRL-C without crash (2761) (Tao He)
- [df1628425]: Store address offset of string properties with 48-bit unsigned intege… (2760) (luoxiaojian)
- [747df41cf]: Drop the environment switch in dev and test, and update gs (2754) (Siyuan Zhang)
- [46d7ed8bb]: Fixes duplicated run and upload actions for k8s-ci (2751) (Tao He)
- [3cf079817]: Propogate the k8s exception to avoid waiting forever (2747) (Tao He)
- [e6af69be1]: Ensure use string as log-level, even for logging.XXX (2745) (Tao He)
- [e969d0e4e]: [GIE Dev Test] Add docs for interactive dev test (2733) (Xiaoli Zhou)
- [ac7308e87]: [BugFix] add Metadata for `PathExpand` (2720) (BingqingLyu)
- [94ec2d895]: Updated license and notice. (2725) (Jingbo Xu)
- [8ab7b1b84]: Overhaul data-load-tools (2715) (Siyuan Zhang)
- [b03bc60ea]: Publish wheel packages on Linux arm64 platform nightly (2732) (Dongze Li)
- [1e63e8731]: [GIE/bug] fix a bug of matching with anti join (2730) (Longbin Lai)
- [1a8162090]: use javac to find the version of java (2728) (Siyuan Zhang)
- [51f92ac04]: Support to build wheel package on Linux aarch64 platform (2723) (Dongze Li)
- [1d3cd4a44]: refine retry logic to exponential backoff (2729) (Siyuan Zhang)
- [06f876920]: Fixes the installation prefix for nightly linux CI (2727) (Tao He)
- [662014fcc]: Opt-out zstd in features when requiring rocksdb (2722) (Tao He)
- [0a5c14410]: set timeout of rt_admin to 1 minute. (2719) (luoxiaojian)
- [fb79c6211]: Bump up graph-learn submodule version (2716) (Tao He)
- [7a5ed1344]: add docs for using dev container and fix typo (2711) (Siyuan Zhang)
- [5bd90ba10]: Requires correct version of graphscope_client in coordinator to avoid diverage (2708) (Tao He)
- [10a6488de]: Introduce the new version of kubernetes log exporter for dumping logs during a period of time such as pytest (2706) (Ye Cao)
- [e447cd5cd]: Fixes the linux wheel install prefix (2707) (Tao He)
- [f476c5736]: Fixes the 'sudo' in dockerfile for gss image (2704) (Tao He)
- [3b11b77ff]: Build wheels for macOS m1 (2701) (Tao He)
- [446b6601d]: [LICENSE] add dependency licenses for calcite and antlr (2699) (Xiaoli Zhou)
- [52b1ee29e]: Fixes the initialization of paramters in load_from() (2698) (Tao He)
- [d4b72fd97]: Display stderr logs as expected on K8s (2696) (Tao He)
- [6da5c3112]: Add version info for making interactive (2685) (Siyuan Zhang)
- [6dd3af29b]: Implemented 3-retries, added detailed logs for insert queries, and unified bulk-load and recovery. (2691) (luoxiaojian)
- [03165eb94]: [FEAT][GraphAr] Add API to python client to enable loading graph from gar files or archive graph to gar files (2588) (Weibin Zeng)
- [c2cb28678]: Export the k8s logs of graphscope helm test and upload to artifact when the helm test failed (2683) (Ye Cao)
- [81230c586]: Add prepare-commit-msg (2688) (Siyuan Zhang)
- [90b7e0049]: Update install-hook.sh (Wenyuan Yu)
- [ff9582e37]: Create install-hook.sh (2687) (Wenyuan Yu)
- [909b3c4d0]: [GIE/Runtime] Remove the dependency of GIE Runtime on GlobalQueryStore to accelerate the compilation (2681) (BingqingLyu)
- [a46663e52]: Clearify the requirements for MacOS (2684) (Tao He)
- [25a70f127]: Preload searchindex.js in background to optimize the search experiences (2682) (Tao He)
- [b258999b0]: Bump up vineyard to v0.14.5 (2676) (Siyuan Zhang)
- [ff5809e06]: Use different default value for vineyard_shared_memory on k8s and local (2678) (Tao He)
- [a38f41439]: enable auth for realtime write (2674) (Siyuan Zhang)
- [4225cb1c7]: Use ubuntu-base to build out production images. Add another `wheel` tag dedicated for building wheels. (2607) (Siyuan Zhang)
- [8168275e5]: Improve the error message when passing unexpected arguments to `g.project()` (2667) (Tao He)
- [5646ce2ea]: Document how to install scipy on M1 MacOS. (2666) (Tao He)
- [b4caf8f06]: [GAE] Fix overflow of MPI_Isend()'s data size (2665) (Songqing Zhang)
- [5c0ad71cc]: Upgrade protobuf/grpc, use newer macos builder, fixes bugs in local launcher where vineyardd may fail, use latest black as formatter (2650) (Tao He)
- [25593331c]: temporarily fix the urllib3 2.0.0 incompatible with openssl (2662) (Siyuan Zhang)
- [57dba68e6]: Tune kafka and grpc maximum message size (2661) (Siyuan Zhang)
- [1f0b8b91e]: Implemented the init version of Flex. (2660) (luoxiaojian)
- [35b148c0c]: Update the GLE submodule to include the latest updates (2643) (LiSu)
- [7fc0a1522]: Change version to 0.22.0 for daily release (2658) (DongZe Li)
- [21bdae33c]: add export CMAKE_PREFIX_PATH (2655) (Siyuan Zhang)

New Contributors
* wenyuanyu made their first contribution in https://github.com/alibaba/GraphScope/pull/2687
* TaoLbr1993 made their first contribution in https://github.com/alibaba/GraphScope/pull/2813

**Full Changelog**: https://github.com/alibaba/GraphScope/compare/v0.21.0...v0.22.0