Graphscope

Commits
---------
- [f30b95c7]: Fixes CI failure on macOS (1346) (DongZe Li)
- [d911efe5]: Stop coordinator gracefully by accepting the signal from notebook restarted/stopped event (1342) (DongZe Li)
- [ff33f5a6]: Keep the global symbol to avoid it been stripped on Mac. (1345) (Tao He)
- [51f680aa]: Upgrade required vineyard version to 0.3.19 (1339) (Siyuan Zhang)
- [569c04b2]: Fixes PATH to not include $LLVM/bin to make sure the correct ranlib be found. (1341) (Tao He)
- [0b083e54]: Revert "Make sure the $PATH could search /usr/bin first. (1340) (Tao He)
- [158f4525]: Make sure the $PATH could search /usr/bin first. (1338) (Tao He)
- [385accd1]: Extract a util dynamic library from grape-engine (1337) (Siyuan Zhang)
- [e3f055ec]: Local bench groot (1326) (tianliplus)
- [953cc3e0]: Add timeout after launch vineyard to ensure we catches the error code correctly (1332) (Siyuan Zhang)
- [5e33f669]: Improve the docs. (1324) (Tao He)
- [e13eea9c]: Adapt to mutable fragment. (1320) (luoxiaojian)
- [f14b6f65]: Bump up graph-learn package for checkpoint. (1323) (Tao He)
- [ec3f3ef7]: Use `LargeAttrValue` in modify operation and dumps json with orjson (1321) (Weibin Zeng)
- [1a94a810]: Enable download metadata for repo 'appstream' of centos-8 (1318) (DongZe Li)
- [ec0ac9de]: Stream RunStepResponse of grpc to support fetching result more than 2GB (1314) (DongZe Li)
- [156c1c83]: Replace `folly::dynamic` with `rapidjson::Value` and bump vineyard to 0.3.18 (1271) (Weibin Zeng)
- [f3113bac]: Bump hadoop-common from 2.8.4 to 2.10.1 in /interactive_engine (1312) (dependabot[bot])
- [6918b764]: Stream RunStepRequest of grpc to support loading graph from numpy/pandas more than 2GB (1309) (DongZe Li)
- [cdb0dcfa]: Use `boost::lexical_cast` to handle string id fragment (1310) (Siyuan Zhang)
- [3b607ec2]: Bump httpclient in /research/query_service/gremlin/compiler (1306) (dependabot[bot])
- [0200ceb8]: Bump commons-io in /research/query_service/gremlin/compiler (1307) (dependabot[bot])

Commits
---------
- [97d12bf5]: Fixes failure on macOS CI (1304) (DongZe Li)
- [10f25808]: Add 'large_attr' data structure in op definition for large chunk (data of pandas/numpy) (1303) (DongZe Li)
- [e554e936]: [networkx] binding session from graphscope graph and fix bug (1302) (Weibin Zeng)
- [3dc75f49]: Better schema representation (1299) (Siyuan Zhang)
- [66b74fd4]: Cython sdk rst: Context.superstep() change return type str -> int (1296) (wuyueandrew)
- [7eeb7259]: Print data type in graph schema (1294) (Siyuan Zhang)
- [66825203]: Check the nullptr of graph_info (1295) (Siyuan Zhang)
- [7fc69d99]: Fix: nightly CI was broken on test_error_on_selector case (1291) (DongZe Li)
- [fd676d77]: Fix the invalid identifier 'FULL-TEST-SUITE' in the environment variable (1289) (DongZe Li)
- [26f52147]: Support running udf app on property graph with string oid type (1287) (DongZe Li)
- [b683dcd7]: Raise an error and tell users what to do on Windows. (1285) (Tao He)
- [e8f7aaf4]: Improve the document of learning.Graph (1277) (Weibin Zeng)
- [976345bf]: Fixes building docs for graphlearning, and the scroll bar. (1279) (Tao He)
- [06e3392e]: Support running built-in app on the property graph by projecting to flattened fragment implicitly (1258) (DongZe Li)
- [0ba3d5e8]: Drop support for Python 3.6 (1268) (Weibin Zeng)
- [d3e85456]: Fix the broken networkx ci (1264) (Weibin Zeng)
- [1446cd6d]: Parallelize eigenvector_centrality and katz_centrality (1262) (Weibin Zeng)
- [38470b0d]: Revise built-in clustering and replace forward with built-in (1255) (Weibin Zeng)
- [334e6248]: Bump protobuf-java from 3.5.1 to 3.16.1 in interactive_engine (1260) (dependabot[bot])
- [83c68425]: Guard folly headers when NETWORKX=OFF (1259) (Siyuan Zhang)
- [a45a7b99]: Optimize modification APIs of graphscope.nx (1250) (Weibin Zeng)
- [1ae368f3]: Add metrics for benchmark groot (1238) (tianliplus)

Commits
---------
- [c49b1b21]: Format java source code of analytical engine (1252) (DongZe Li)
- [306da050]: Refactor GAE Java (1227) (Zhang Lei)
- [64afab7d]: Fixed the auditwheel version to 5.0.0 (1251) (DongZe Li)
- [847a6a29]: Quickfix: a syntax error. (1248) (Tao He)
- [38ad3def]: Fixes the long-standing "Event loop is closed" issue. (1247) (Tao He)
- [53316583]: Bring back mars support, re-enable mars test in CI, and fixes several related issues. (1246) (Tao He)
- [9b32ccfc]: Enable serialization test on K8s, and add a new case to test local serialization (1245) (Tao He)
- [819b9709]: Enable customization of storage class (1244) (Siyuan Zhang)
- [038e915a]: Print diffs when failed to format java source code. (1243) (Tao He)
- [529ecd22]: Consistent label checking in client side (1242) (Siyuan Zhang)
- [a7d1472f]: Remove unrelated file from gh-pages (1241) (Siyuan Zhang)
- [89f03da1]: Add documents about potential permission error when allocating PV in NFS. (1239) (wuyueandrew)
- [db9b427d]: Format java code with google style (1233) (DongZe Li)
- [e37f69c1]: Upgrade log4j2 (1235) (tianliplus)
- [e1e22a42]: Fix log not fetched to client (1236) (Siyuan Zhang)
- [2e7e6399]: Changes on property applications (1228) (DongZe Li)
- [ab75ad71]: Fix error when selecting from empty graph (1231) (Siyuan Zhang)
- [9f8e7f21]: etcd: Increate the ETCD pods period seconds (1213) (Hao Xin)
- [0c60e6de]: Bump up required vineyard version to v0.3.14 and reable HDFS IO tests. (1229) (Tao He)
- [84ca6cd8]: Fix some typo in persistent_graph_store.rst (1232) (wuyueandrew)
- [e26e5330]: Fixed a typo in `persistent_graph_store.rst` (1226) (wuyueandrew)
- [06de1f17]: Make forward nightly ci pass (1223) (Weibin Zeng)
- [5b724b7a]: Fix codecov coverage ratio (1224) (DongZe Li)
- [a8129084]: Make GraphScope support on Apple M1 chip (1221) (DongZe Li)
- [f2ed9a14]: Typo of conn (1222) (Siyuan Zhang)
- [9e912363]: Upload codecover report only in ubuntu20 platform (1220) (DongZe Li)
- [d7140119]: add edit_on_github button on docs. (1214) (Jingbo Xu)
- [1eacd877]: Make the definition of src in sssp and bfs accurate (1218) (Siyuan Zhang)
- [ed4a82b3]: Report codecover of nightly CI (1219) (DongZe Li)
- [62c0676c]: [networkx] Revise the add_node/edge with cache and upgrade NetworkX dependency to 2.6 (1208) (Weibin Zeng)
- [a6099b69]: Enhanced input validation and schema representation (1205) (Siyuan Zhang)
- [8cdc1b6b]: Rename occurances of "libvineyard" to "v6d". (1202) (Tao He)
- [ec000ba9]: Support compiling analytical engine on Apple M1 chip (1200) (DongZe Li)
- [76d4641e]: [networkx] Update the built-in app implementation to make the behavior consistent with NetworkX implementation (1176) (Weibin Zeng)
- [3049ef92]: Update codecov/codecov-action to v2. (1193) (Tao He)
- [1c62d2a4]: Increase the default resource in helm deployment (1189) (DongZe Li)

We highlight the following fixes included in this release:

- Fix compile error of analytical engine app on [google colab](https://colab.research.google.com/)
- Fix failure of import `graphscope.nx` module under python3.6
- Fix the problem that `openmpi` cannot be found during compiling app in macOS
- Fix failure during running `app_assets` twice.
- Upgrade log4j2

We are glad to announce the availability of GraphScope v0.9.

In this release, we revisit the Dev-infra to improve the productivity. Now, you can enjoy GraphScope with standalone mode both in our PlayGround and Google Colab.

We also aims to continuously make GraphScope more user-friendly, and revised the documents and [tutorials](https://github.com/alibaba/GraphScope/tree/main/tutorials) based on the latest version.

Besides, we alsh have a preliminary support for [Java]( https://graphscope.io/docs/analytics_engine.html#writing-your-own-algorithms-in-java.) in Graph Analytics Engine (GAE), and users can succinctly [develop](https://graphscope.io/docs/reference/gae_java/index.html) graph analytics applications with Java.

For Graph Interactive Engine (GIE), we have done a lot of work to improve the performance of the distributed GAIA engine, which includes but is not limited to 1) simplifying the communication protocol that reduces many useless tags in correlated subtask; 2) making early-stop mechanism more effective due to 1); 3) resolving a lot of bugs due to 1); 4) refining the engine apis. As a distributed cyclic data-parallel engine, GAIA engine is in de-facto generic to handle any data-intensive task. Now users can assess to more GAIA [APIs](https://github.com/alibaba/GraphScope/tree/main/research/engine/pegasus/pegasus/src/api) (will be well documented) to play with the engine.

Features
- Support GLE on MacOS
- Java SDK in GAE, and examples to write and run algorithms written in Java.
- Add several popular datasets and ships datasets by adding an extra container in Kubernetes mode.
- Supplement NetworkX generator, read, write, drawing, convert with more generators and test cases.
- Add make install support for lgraph.
- GIE Runtime refactoring based on the new GAIA engine.
- Reimplement persistent storage engine with zero-copy read.
- Add common labels following Kubernetes best practice to GraphScope cluster.
- Add transformation ability for directed/undirected graphs.
- Unify the logging output file location to `/var/log/graphscope` or `$HOME/.local/log/graphscope`

Bug Fixes
- Fixed a bug that caused a crash if duplicate property names were encountered when extracting subgraphs.
- Fixed MPI library not found problem on MacOS.
- Fixed a data-lost bug in realtime write caused by kafka retention policy.
- Fixed context processing on graphs generated by `add_column`.
- Fixed a failure when exposing gremlin service deployed with Helm Charts.
- Fixed many bugs related to subtasks while querying Gremlin.

We are glad to announce the GraphScope 0.8 release. The GraphScope 0.8 release is a major update on many aspects of the project including deployment, system speed, and APIs. For quickly getting started, this release supports enjoying GraphScope on standalone mode without Kubernetes. To improve the efficiency of operators and applications in the NetworkX module, an immutable graph is applied by default, while it is converted to a dynamic graph only if modification operators for graphs are triggered. In addition, a jupyter notebook is integrated into the helm charts.

We highlight the following improvements included in this release:

1. Standalone mode support for GraphScope
- User now can install graphscope by `pip install graphscope` or install from [wheel](https://github.com/alibaba/GraphScope/releases/tag/v0.8.0) manually to enjoy GraphScope on local with runtime dependencies
- **User now switch to pip install `graphscope_client` to install python client of GraphScope**
- MacOS support is added, and compatible with Apple clang 13.0.0

2. Enhancement on graphscope.nx module
- Support NetworkX operators over immutable graphs:
- Support holding immutable graph in `nx.Graph` and **copy-on-write** to dynamic graph

3. Enhancement of deployment, performance and APIs:
- Integrate a jupyterlab notebook into the helm charts

For more detailed improvements that have been made in this release, please refer to the complete changelog.

Commits
---------
- [[7cfe16b7]: Don't export `/usr/local/opt/llvm/lib` to linker search path on MacOS. (936) (Tao He)
- [[52ebbaae]: Fix slf4j StaticLoggerBinder loading error (941) (Zichao Zhang)
- [[e52b1e1d]: Fixes helm charts jupyterlab container: use release version as image tag (935) (DongZe Li)
- [[6e5404ed]: Add readiness probe to graphscope-store frontend. (937) (Siyuan Zhang)
- [[a68c83ca]: [networkx] Riase `NetworkXError` when gs graph is multigraph (924) (Weibin Zeng)
- [[c80a2df5]: Add maxgraph backup module (921) (Zichao Zhang)
- [[476f3aab]: Cancel previous wheel packaging actions in CI as well. (Tao He)
- [[a5eb44b3]: Support using GraphScope on standalone mode (MacOS) (923) (DongZe Li)
- [[5d4c84d6]: Cancel workflows defined in other yamls. (Tao He)
- [[688f73a3]: Install LLVM and fastFFI in runtime/gsvineyard docker image. (932) (Tao He)
- [[c7815b03]: Cancel previous runs in PRs. (Tao He)
- [[30658fd1]: Integrate a Jupyter-lab container in the graphscope helm chart. (928) (Siyuan Zhang)
- [[d5821a12]: Add a flattened wrapper on property graph to run apps defined on simple graph (888) (Weibin Zeng)
- [[a29a964f]: Unload unreachable op after run (916) (Siyuan Zhang)
- [[0bc8393c]: Remove unused parameters (912) (Siyuan Zhang)
- [[959cb731]: Schedule package CI as nightly run (908) (DongZe Li)
- [[cf00cf34]: Add unittest for global store ffi and fix an issue in global store ffi that uses a shared global client to access vineyard. (905) (zhiminch)
- [[e6bdb109]: Fixes permission denied in nightly CI (906) (DongZe Li)
- [[9291e5c3]: Fix groot ut (903) (tianliplus)
- [[d970e885]: Add backup module of graph store (901) (Zichao Zhang)
- [[8004e224]: [networkx] Implement average degree connectivity app (516) (Xinger)
- [[9ac63094]: Fixes nightly CI failure and symlink problem (899) (DongZe Li)
- [[3b1de8f3]: Add progress bar for loading graphs. (894) (Siyuan Zhang)
- [[dbe5d0b8]: reorg gie submodules (893) (tianliplus)
- [[4cdc4848]: Change rust-rocksdb dependency for store (892) (GoldenLeaves)
- [[e5fbefe2]: Separate download tests (895) (Siyuan Zhang)
- [[1c60ff1a]: Support using GraphScope on standalone mode (Linux) (860) (DongZe Li)
- [[308f35e0]: Reorg groot (887) (tianliplus)
- [[d91f55e7]: [networkx] Support reporter API on arrow fragment (866) (Weibin Zeng)
- [[2956abc1]: [networkx] Implement all simple paths app (574) (MJY-HUST)
- [[dd816c39]: Create and use default session when there is no default session found (880) (DongZe Li)
- [[26d320b0]: Fix unit tests under frontend module (883) (shirly121)
- [[5f25d518]: update kind download url (882) (Weibin Zeng)
- [[db3b5e2a]: Launch etcd in coordinator (878) (DongZe Li)
- [[1dac7f41]: [networkx] implement (weighted) betweenness centrality in graphscope. (666) (mengke)
- [[19245211]: [Bug Fix] fix bug in starting GAIA service in v2 and vineyard (870) (BingqingLyu)
- [[f47e0197]: Fix compilation warning. Remove utilities in vineyard store ffi. (861) (zhiminch)
- [[79818472]: [bug fix] support id-only vertex in EdgeV Step (871) (BingqingLyu)
- [[2cd20e58]: Require vineyard (for serialization test, etc.), but only on x86_64. (873) (Tao He)
- [[46e34e46]: Launch vineyard in a nicer way when vineyardd not found. (872) (Tao He)
- [[50946714]: address maven warnings by specifying the dependency versions. (864) (Jingbo Xu)
- [[51379546]: merge gie and groot schema api (825) (tianliplus)
- [[16d41fb5]: Revisit the bindings to mars. (863) (Tao He)
- [[d6616cd2]: Make the `add_edges` behavior more Pythonic (855) (Siyuan Zhang)
- [[bc5c430f]: Fix unit test in compiler of interactive engine (859) (shirly121)
- [[c972b605]: [networkx] Implement attribute assortativity app and numeric assortativity app (515) (Xinger)
- [[ce439b5c]: [networkx] Support label on DynamicFragment (822) (Weibin Zeng)
- [[13b6b4f8]: Support download dataset from origin URL in standalone mode (828) (DongZe Li)
- [[4bd128b4]: Fix an incorrect error initialization (827) (Siyuan Zhang)
- [[bf7204f9]: Fix: add data cache clean in gremlin_test.sh (823) (Weibin Zeng)
- [[9d795761]: Update local deploy script: upgrade jdk tool-chain and open macOS CI (758) (Weibin Zeng)
- [[1c99a983]: [networkx] Check if node is illegal and raise NetworkXError (817) (Weibin Zeng)
- [[febd633d]: Nightly: don't run on forks. (818) (Tao He)
- [[d21de371]: Recover log4rs.yml of graphscope-store and fix the log output location (814) (Weibin Zeng)
- [[1858a873]: GaiaX Runtime refactoring (812) (BingqingLyu)
- [[ee03c8fc]: Unified directory structure after installation (809) (DongZe Li)
- [[846dd4f5]: Enable math.h functions in UDF algorithms. (811) (Tao He)
- [[60ab3af1]: Gaia x (801) (Longbin Lai)
- [[7dc8086d]: Add parameters section to Store docs (802) (Siyuan Zhang)
- [[bc048d79]: fix bug in graph store doc (799) (tianliplus)
- [[3e2fd7dd]: Add methods to unload contexts. (794) (Siyuan Zhang)
- [[b5ba16ca]: clear duplicated and unused code (777) (tianliplus)
- [[4b6b665a]: `shutil.rmtree` cannot remove files (793) (Siyuan Zhang)
- [[fce08189]: rename 0.0.1-SNAPSHOT as maxgraph-assembly-0.0.1-SNAPSHOT and update reference (783) (shirly121)
- [[c6fd2780]: Enable manually trigger nightly ci (789) (Weibin Zeng)
- [[106078ff]: Increase the resource size of coordinator for running demo of README (791) (DongZe Li)
- [[b03e12ba]: Build app/graph frames: with "Release" profile by default. (790) (Tao He)