Thanks to a whole bunch of feedback on our Python alpha, over the last 6 months Raphtory has been through its second full make over with a total rewrite of the core engine in Rust! There are WAY too many changes to list here, but below you can see a couple of highlights.
If you would like to give this new version a go, you can check out the [docs](https://docs.raphtory.com/en/v0.3.1/) as well as our [examples](https://github.com/Pometry/Raphtory/tree/master/examples) - available in both Python and Rust. Good places to start are the Jupyter notebooks for the [Reddit](https://github.com/Pometry/Raphtory/blob/master/examples/py/reddit/demo.ipynb) snap dataset and [Enron](https://github.com/Pometry/Raphtory/blob/master/examples/py/enron/enron.ipynb) emails.
Highlights 🏆
Revamped API
The first thing you will notice about the new Raphtory is that the API is A LOT more friendly. The example notebooks above show this off nicely, but as a snippet this is how you could grab Gandalf from our Lord of the Rings Graph and plot his degree every 1000 time increments:
python
timestamps = []
degree = []
for gandalf in g.vertex("Gandalf").rolling(window=1000):
timestamps.append(gandalf.latest_time())
degree.append(gandalf.degree())
sns.set_context()
ax = plt.gca()
plt.xticks(rotation=45)
ax.set_xlabel("Time")
ax.set_ylabel("Interactions")
sns.lineplot(x = timestamps, y = degree,ax=ax)
<img width="580" alt="image" src="https://github.com/Pometry/Raphtory/assets/6665739/974d06a4-a432-4654-8ace-984ec73ff582">
Performance 📈
- The memory usage of Rust is over an order of magnitude less than Scala, with one test graph (~11 million nodes, ~33 million edges) previously requiring ~120GB of Ram now only needing 7GB! This means that datasets which required spinning up a EC2 instance or cluster can now easily be run on your laptop 💪🏻
- Similarly, complex algorithms like `Temporal Triadic Motifs` run in seconds instead of hours - even when running locally instead of on a beefy box 🍖
- Querying for specific vertices/edges runs 1000x faster and can be done directly on the graph via the new APIs instead of having to write a full algorithm.
Python 🐍
- Raphtory can now be easily installed with all dependencies via `pip install raphtory`.
- Via [PyO3](pyo3.rs) Raphtory can now be run and called directly in python without secretly running a Java Runtime Environment in the background 🥷. This also means it now works perfectly with all other python libraries!
- The vast majority of the heavy lifting is done in Rust, meaning you get all the ease of Python, with very little slow down!
Whats next ⏭️
We are currently tinkering away with several parts of the new APIs, adding more helper functions, and expanding the algorithmic suite. However, alongside this we have some larger pieces of work in the pipeline:
- We are building a GraphQL extension to Raphtory, codename [flux-capacitor](https://github.com/Pometry/flux-capacitor), alongside the functionality for compiling Raphtory to [WASM](https://webassembly.org) - making it possible to interact with Raphtory through Javascript/web UIs.
- We are working on out-of-core analysis, allowing Raphtory to work on Graphs many times the size of the memory of the machine it is running on.
- Finally, we are running a suite of standard benchmarks on this new version to give some concrete comparable numbers to other Graph systems.
If you have an other suggestions please do let us know via Github [issues](https://github.com/Pometry/Raphtory/issues), [discussions](https://github.com/Pometry/Raphtory/discussions) or on [Slack](https://join.slack.com/t/raphtory/shared_invite/zt-xbebws9j-VgPIFRleJFJBwmpf81tvxA)!
**Full Changelog**: https://github.com/Pometry/Raphtory/compare/v0.2.0...v0.3.1