Runhouse

Improved parallelism, clearer exceptions, and saving resources within Den orgs

Improvements
* Improve the thread, reference, and fault tolerance model for EnvServlet ray actors (735, 733, 736, 734, 737)
* Catch all non-deserializable exceptions client-side (730)
* Support for saving resources on behalf of an org (676, 732)

Bugfixes
* Dynamically set `API_SERVER_URL` (708)
* Move OMP_NUM_THREADS setting into servlet to avoid setting it on import by (731)

**Full Changelog**: https://github.com/run-house/runhouse/compare/v0.0.24...v0.0.25

Fast-follow bugfixes for CPU parallelism and log streaming

Bug fixes
* Fix ray persistently setting OMP_NUM_THREADS=1 (723)
* Fix method call log streaming by unbuffering stdout/err in call threadpool (724)

**Full Changelog**: https://github.com/run-house/runhouse/compare/v0.0.23...v0.0.24

Richer async support, performance improvements, and bugfixes

Improvements
* Client-side Async support (690, 696, 696, 689) - We've improved the way we handle async calls to remove modules. Now, you can properly unblock the event loop and await any remote call by passing `run_async` as an argument into the method call. If your method is already defined as async, this will be applied automatically without specifying `run_async` so your code can `await` the remote method just as it did the original. You can still explicitly set `run_async=False` in that case to make the local call sync.
* Improve Mapper ergonomics and docs (700, 709) - Now you can simply pass a function to the mapper and it will send over the module and create replicas on its own. We'll publish new mapper tutorials shortly.
* Cache rich signature for Module to improve method call performance (699)
* Don't serialize tracebacks in OutputType.EXCEPTION (721) - Sometimes exceptions can't be deserialized locally because they depend on remote libraries. In those cases, we now still print the traceback for better visibility.
* Unset OMP_NUM_THREADS when Ray automatically set it because it may break user parallelism expectations (719)

Bugfixes

* Fix stdout and logs streaming in various scenarios (716, 717)
* Remove unused `requests.Session` created in HTTPClient (694)
* Change Caddy installation to download from Github (702) (Sorry Caddy!)
* Inherit Cluster READ access for resources on the cluster (706)
* Set the cluster name in the HTTPClient upon rename (704)
* Fix some `runhouse login` bugs (717)
* Make errors from Den include status code and be more verbose (707)
* Fix SkySSHRunner tunnels and processes to be correctly cleaned up (718)

**Full Changelog**: https://github.com/run-house/runhouse/compare/v0.0.22...v0.0.23

Performance improvements + bug fixes

Improvements
* Add to open_ports when creating new on demand cluster (651)
* Updates to Sagemaker Cluster (654)
* Change `AuthCache` logic to request per keypair (684)

Performance Improvements
* Cache various module/function computations (661, 665, 662)
* Async daemon side components (656, 664, 673, 674, 670)
* Use ThreadPoolExecutor to synchronous function calls on server side (663)
* Decrease log wait time (685)

Bug Fixes
* Fix bug with json serialization for exceptions (655)
* Update returned exceptions to be json serializable.
* Use shell for running cmd in env servlet (667)
* Previously shell commands would not consistently work.
* Fix cluster autostop (672, 681, 683)
* Change to correctly set and update last activity time and do it in a background thread
* Fix multinode cluster ips (681)
* Cluster ips previously computed from cached ips and would incorporate stale ones. Update to use only current ips.

Examples
* Add Llama2 on Inferentia with TGI example (649)
* Update Inferentia examples to use the DL AMI (677)

Some performance and feature improvements, bug fixes, and new examples.

Improvements
* OpenAPI pages for cluster (579, 586, 587, 589, 590)
* Properly raise exceptions in Module's `load_config` when dependency is missing (595)
* Kill Ray actors by default during `runhouse stop` (596)
* `module.to(rh.here)` throws error if local server is not initialized (597)
* Send exceptions in `data` field (602)
* Run commands inside env servlet (603)
* Return exceptions instead of `None` in failed mapper replicas (605)
* Remove `sshtunnel` library dependency (625, 634, 640)
* Don't save cluster secret during cluster init (633)
* Remove creds from cluster's config file (637)
Performance
* Use `check_server` instead of `is_up` with refresh for ondemand cluster endpoint (614)
* Remove `register_activity` calls within env servlet (629)

Bug Fixes
* Install aws dependencies properly for `runhouse[aws]` (613)
* Fix env servlet name in `put_resource` (626)
* Env servlet was using conda env name instead of env resource name.
* Fix SkySSHRunner local and remote port ordering (630)

BC-Breaking
* Remove previously deprecated items (624)
* `reqs` and `setup_cmds` in `rh.function.to()` removed. Pass it into the `env` instead.
* `access_type` removed in `Resource` and `share`. Use `access_level` instead.
* global pinning methods removed. Use `rh.here.put/get/delete/keys/clear` instead.
* Deprecate and raise exception for passing system into function/module factories (625)
* Passing in `system` to `rh.function/module` does not send code to the system and can be misleading. Use `.to` or `get_or_to` to sync code to the cluster.

Examples
See rendered examples on https://www.run.house/examples
New Examples
* Mistral 7B Inference with TGI on AWS EC2 (585, 604)
* Mistral 7B Inference on AWS Inferentia (609)
* Langchain RAG App on AWS EC2, with Custom Domain (607, 621)
* Llama2 on EC2 A10G (608)
* Llama2 Inference with TGI on AWS EC2 A10G (610)
Updates
* Add READMEs to GitHub (612, 619)
* Avoid reinstall for envs and extra imports in examples (616, 618)

Highlights

Cluster Sharing
We’ve made it easier to share clusters across different environments and with other users. You can now share and load a cluster just as you would any other resource.


my_cluster = rh.cluster("rh-cluster", ips=[...], ...)
my_cluster.share(["user1email.com", "username2"])

load the box with
shared_cluster = rh.cluster("owner_username/rh-cluster")


Shared users will be able to seamlessly run shared apps on that cluster, or SSH directly onto the remote box. To enable this, we persist the SSH credentials for the cluster as a Runhouse Secret object, which can easily be reloaded when another user tries to connect.

Improved rh.Mapper
`rh.Mapper` was first introduced in runhouse v0.0.15, an extension of functions/modules to handle mapping, replicating, and load balancing. Further improvements and some bug fixes were included in this release, plus a BC-breaking variable name (see section below).


def local_sum(arg1, arg2, arg3):
return arg1 + arg2 + arg3

remote_fn = rh.function(local_sum).to(my_cluster)
mapper = rh.mapper(remote_fn, replicas=2)
mapper.map([1, 2], [1, 4], [2, 3])
output: [4, 9]


Improvements
* Use hashed subtoken for cluster requests (270)
* Simplify storage of SSH creds for more reliable cluster access across environments and users (479)
* Remove sky storage dependency (415)
* Replace subprocess check_call with run (503)
* Serialize exceptions properly (516)
* Improved Logging
* Only write out execution logs if stream_logs is set (490)
* Propagate logs from pip installs on cluster (505)
* Write some logs to sys.out (519)

Bug Fixes
* Mapper bug fixes (539)

Deprecation
* Renaming config_for_rns property to config function (553, 554, 555)

BC-Breaking
* `rh.mapper` factory function args renaming
* `num_replicas` -> `replicas`
* `replicas` -> `concurrency`

Docs
See updated tutorials on [Runhouse docs](https://www.run.house/docs)
* New quick start guides -- local, cloud, and Den versions
* Updated API tutorials -- clusters, functions & modules, envs, folders

Examples
See new Runhouse examples on [GitHub](https://github.com/run-house/runhouse/tree/main/examples) or [webpage](https://run.house/examples)
* Llama2 inference on AWS EC2
* Stable Diffusion XL 1.0 on AWS EC2
* Stable Diffusion XL 1.0 on AWS Inferentia

Other
* Remove paramiko as server connection type