Skyplane

What's Changed
* IBM Support by sarahwooders in https://github.com/skyplane-project/skyplane/pull/731
* Resolve redundant GCP logs in load_gcp_config by mech-a in https://github.com/skyplane-project/skyplane/pull/786
* [Fix] Azure init issues by zizhong in https://github.com/skyplane-project/skyplane/pull/787
* Delete test buckets during integration testing by parasj in https://github.com/skyplane-project/skyplane/pull/798
* Programmable Gateways by sarahwooders in https://github.com/skyplane-project/skyplane/pull/804
* [Fix] exit before transfer all the files. by killerdbob in https://github.com/skyplane-project/skyplane/pull/813
* Support private versus public IPs for gateways by sarahwooders in https://github.com/skyplane-project/skyplane/pull/817
* Add integration tests with multiple file sizes. by xutingl in https://github.com/skyplane-project/skyplane/pull/816
* Fix Azure Quota API call error message by simon-mo in https://github.com/skyplane-project/skyplane/pull/821
* Fix integration tests by disabling IBM for now by sarahwooders in https://github.com/skyplane-project/skyplane/pull/822
* Fix single region transfers (same source/destinations) by sarahwooders in https://github.com/skyplane-project/skyplane/pull/826
* Implement azure private ip function by sarahwooders in https://github.com/skyplane-project/skyplane/pull/828
* Update google cloud auth action to `v1` by sarahwooders in https://github.com/skyplane-project/skyplane/pull/830
* Add Nix flake for creating development environments by shadaj in https://github.com/skyplane-project/skyplane/pull/833
* Updated integration tests and local transfers fix by sarahwooders in https://github.com/skyplane-project/skyplane/pull/831
* Don't search host directories for private keys when connecting to a cloud instance by shadaj in https://github.com/skyplane-project/skyplane/pull/837
* Fix HTTP connection error for long running transfers by sarahwooders in https://github.com/skyplane-project/skyplane/pull/842
* added null check to azure vnet cleanup by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/851
* fall back to smaller vm if necessary by xTRam1 in https://github.com/skyplane-project/skyplane/pull/815
* Cloudflare R2 Integration by sarahwooders in https://github.com/skyplane-project/skyplane/pull/856

This PR fixes a bug in 0.3.0 where transfers would break due to an error in the Skyplane Gateway.

What's Changed
* Clean AWS provisioning logic to prepare for debugging concurrent Skyplane usage by parasj in https://github.com/skyplane-project/skyplane/pull/610
* Add flag before deprovisioning networks by parasj in https://github.com/skyplane-project/skyplane/pull/615
* Clean up leaked resources upon VM provisioning errors for AWS, GCP and Azure by parasj in https://github.com/skyplane-project/skyplane/pull/617
* Prefix AWS keys with UUID, tag VMs with UUID by parasj in https://github.com/skyplane-project/skyplane/pull/616
* Organize imports, run autoflake and clean up logic around gateway cleanup by parasj in https://github.com/skyplane-project/skyplane/pull/619
* Linked to paper by vdm in https://github.com/skyplane-project/skyplane/pull/620
* Simplify README by sarahwooders in https://github.com/skyplane-project/skyplane/pull/622
* Fix 596 and 598: error profiling throughput grid on aws and gcp by lynnliu030 in https://github.com/skyplane-project/skyplane/pull/605
* Run jobs on pull_request_target to support Github Actions for external contributors by parasj in https://github.com/skyplane-project/skyplane/pull/633
* Use questionary to allow selection of multiple Azure subscriptions by parasj in https://github.com/skyplane-project/skyplane/pull/632
* Run pytest when label is added or for contributors by parasj in https://github.com/skyplane-project/skyplane/pull/634
* azure: add 'tags' argument to provision_instance by phi-line in https://github.com/skyplane-project/skyplane/pull/630
* Add support for Python 3.11 by parasj in https://github.com/skyplane-project/skyplane/pull/639
* Add `skyplane azure check` command by parasj in https://github.com/skyplane-project/skyplane/pull/647
* Fix broken Azure path logic by parasj in https://github.com/skyplane-project/skyplane/pull/650
* [draft] Remove setup.py with editable support by parasj in https://github.com/skyplane-project/skyplane/pull/640
* Cleanup `__init__.py` by parasj in https://github.com/skyplane-project/skyplane/pull/658
* [API] Meta API tracking PR by parasj in https://github.com/skyplane-project/skyplane/pull/652
* Move cloud CLI tools to common file by parasj in https://github.com/skyplane-project/skyplane/pull/660
* Interface for on-prem and pre-signed urls by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/659
* Update exception chaining logic to remove long traceback by parasj in https://github.com/skyplane-project/skyplane/pull/661
* Copy mime types with transfers by parasj in https://github.com/skyplane-project/skyplane/pull/655
* Move compute requirements under single package by parasj in https://github.com/skyplane-project/skyplane/pull/665
* Move GCP networking + pricing logic out of cloud provider by parasj in https://github.com/skyplane-project/skyplane/pull/614
* Use short UUIDs to limit length of provisioned resources by parasj in https://github.com/skyplane-project/skyplane/pull/664
* Migrate chunk IDs from ints to UUIDs by parasj in https://github.com/skyplane-project/skyplane/pull/656
* [bug] Fix integration test by parasj in https://github.com/skyplane-project/skyplane/pull/670
* [API] Performance Logging by JasonDing0401 in https://github.com/skyplane-project/skyplane/pull/669
* [API] fix client id issue by JasonDing0401 in https://github.com/skyplane-project/skyplane/pull/671
* [API] ML training example by JasonDing0401 in https://github.com/skyplane-project/skyplane/pull/676
* Fix Python 3.7 for config_paths by parasj in https://github.com/skyplane-project/skyplane/pull/668
* Nit: update gateway_daemon.py for inflight chunks by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/682
* Clean up API definitions by parasj in https://github.com/skyplane-project/skyplane/pull/686
* Programmable gateways implemented with queue-only inter-process communication by sarahwooders in https://github.com/skyplane-project/skyplane/pull/678
* Small reformatting changes from automatic scripts by sarahwooders in https://github.com/skyplane-project/skyplane/pull/688
* Native cmd fallback by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/694
* [broadcast] Fix issues with gateway merge to main by parasj in https://github.com/skyplane-project/skyplane/pull/689
* Use PyPI release of Skyplane for CI to deprovision VMs by parasj in https://github.com/skyplane-project/skyplane/pull/696
* [ci] Fix deprovision bug on main by parasj in https://github.com/skyplane-project/skyplane/pull/699
* [api] Migrate CLI over the API by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/677
* Add user configurable option for default location for multi-region buckets by parasj in https://github.com/skyplane-project/skyplane/pull/711
* disable OS Login by default for GCP VMs by troycarlson in https://github.com/skyplane-project/skyplane/pull/708
* fix: mismatched console markup by troycarlson in https://github.com/skyplane-project/skyplane/pull/720
* Handle lost instance profiles during deprovisioning by parasj in https://github.com/skyplane-project/skyplane/pull/724
* Add quotes around pip install command for alternative shells like Zsh by parasj in https://github.com/skyplane-project/skyplane/pull/730
* [hotfix] Fix integration tests on main by parasj in https://github.com/skyplane-project/skyplane/pull/713
* [API] Documentations by JasonDing0401 in https://github.com/skyplane-project/skyplane/pull/712
* Restore ILP support branch by parasj in https://github.com/skyplane-project/skyplane/pull/739
* [bug] Azure auth config client id update by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/742
* Prompt users to login, while setting up azure by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/744
* [Fix] the azure auth issue by zizhong in https://github.com/skyplane-project/skyplane/pull/743
* [Fix] Client in CLI uses correct transfer config by zizhong in https://github.com/skyplane-project/skyplane/pull/747
* [Fix] gateway requirements by zizhong in https://github.com/skyplane-project/skyplane/pull/749
* [fix] minor get instance bug in azure by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/752
* [fix] add back copy_log by zizhong in https://github.com/skyplane-project/skyplane/pull/750
* [Fix] gateway auth config and azure download with offset by zizhong in https://github.com/skyplane-project/skyplane/pull/751
* [fix] Adjusting reinit by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/753
* [fix] Copy gateway logs on interrupt by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/754
* [755] Add support for SEV provisioning in Azure by zizhong in https://github.com/skyplane-project/skyplane/pull/756
* GCP multipart completion fix + multi-project service credentials by sarahwooders in https://github.com/skyplane-project/skyplane/pull/757
* Updated keyboard interrupt warning by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/768
* Reformat code and pass pytype by parasj in https://github.com/skyplane-project/skyplane/pull/770
* Clean up confusing variable in dataplane.py by parasj in https://github.com/skyplane-project/skyplane/pull/769
* Prototype Airflow operator by parasj in https://github.com/skyplane-project/skyplane/pull/740
* Update documentation with API information and Tutorials by sarahwooders in https://github.com/skyplane-project/skyplane/pull/771
* Preliminary support for on-prem transfers with HDFS and POSIX path support by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/735
* Add object store interface to API client + tests by sarahwooders in https://github.com/skyplane-project/skyplane/pull/773
* Add cost estimation for dataplane queued jobs by sarahwooders in https://github.com/skyplane-project/skyplane/pull/776
* Download gateway logs with `--debug` flag by sarahwooders in https://github.com/skyplane-project/skyplane/pull/778
* Persist quota information on disk during `skyplane init` by simon-mo in https://github.com/skyplane-project/skyplane/pull/772
* Bump poetry version to 1.4.0 for releases by parasj in https://github.com/skyplane-project/skyplane/pull/779

New Contributors
* vdm made their first contribution in https://github.com/skyplane-project/skyplane/pull/620
* phi-line made their first contribution in https://github.com/skyplane-project/skyplane/pull/630
* troycarlson made their first contribution in https://github.com/skyplane-project/skyplane/pull/708
* zizhong made their first contribution in https://github.com/skyplane-project/skyplane/pull/743
* simon-mo made their first contribution in https://github.com/skyplane-project/skyplane/pull/772

**Full Changelog**: https://github.com/skyplane-project/skyplane/compare/0.2.1...0.3.0

What's Changed
* Mitigate Azure object store stragglers by limiting gateway connections by parasj in https://github.com/skyplane-project/skyplane/pull/594
* Update documentation with link to benchmarks and architecture by parasj in https://github.com/skyplane-project/skyplane/pull/601
* Improve dependency management for Skyplane by parasj in https://github.com/skyplane-project/skyplane/pull/602
* Fix replicate random by printing statistics by parasj in https://github.com/skyplane-project/skyplane/pull/593
* 585: migrate to private ips by lynnliu030 in https://github.com/skyplane-project/skyplane/pull/607
* Fix Azure core import issue by parasj in https://github.com/skyplane-project/skyplane/pull/613

New Contributors
* lynnliu030 made their first contribution in https://github.com/skyplane-project/skyplane/pull/607

**Full Changelog**: https://github.com/skyplane-project/skyplane/compare/0.2.0...0.2.1

What's Changed

Performance improvements
* Multi instance parallelism by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/565
* Accelerate multipart transfer requests by parasj in https://github.com/skyplane-project/skyplane/pull/578

Bug fixes
* Fixes 560 - aws s3 sync no-follow-symlink by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/562
* Fix integration tests w/ bad method rename by parasj in https://github.com/skyplane-project/skyplane/pull/574
* Catch all errors during transfer, not just KeyboardInterrupt by parasj in https://github.com/skyplane-project/skyplane/pull/577
* Extract cleanup from 578 by parasj in https://github.com/skyplane-project/skyplane/pull/583

CLI
* Check for az CLI, update usage message by parasj in https://github.com/skyplane-project/skyplane/pull/567
* Partition dependencies by cloud provider by parasj in https://github.com/skyplane-project/skyplane/pull/573
* Increase python version for integration test to 3.10 by parasj in https://github.com/skyplane-project/skyplane/pull/580

Testing and CI
* [CI] Test multiple versions of Python by parasj in https://github.com/skyplane-project/skyplane/pull/576

**Full Changelog**: https://github.com/skyplane-project/skyplane/compare/0.1.3...0.2.0

Documentation
* Add logo to README by parasj in https://github.com/skyplane-project/skyplane/pull/488
* Compress logo to improve page load times by parasj in https://github.com/skyplane-project/skyplane/pull/489
* Add OpenGraph metadata to documentation website by parasj in https://github.com/skyplane-project/skyplane/pull/494
* Add benchmark documentation to website by JasonDing0401 in https://github.com/skyplane-project/skyplane/pull/496
* Update README.md by HaileyJang in https://github.com/skyplane-project/skyplane/pull/501
* Update faq.md with requester pays info by HaileyJang in https://github.com/skyplane-project/skyplane/pull/538
* Chunking size benchmark to docs by antonzabreyko in https://github.com/skyplane-project/skyplane/pull/539
* Clean up code and update contributor docs by parasj in https://github.com/skyplane-project/skyplane/pull/556

CI testing
* [CI] Run integration test nightly by parasj in https://github.com/skyplane-project/skyplane/pull/526
* [CI] Clean up interfaces with improved object store interface testing by parasj in https://github.com/skyplane-project/skyplane/pull/525
* [CI] Unit tests for Azure by parasj in https://github.com/skyplane-project/skyplane/pull/529
* [CI] Integration tests for Azure by parasj in https://github.com/skyplane-project/skyplane/pull/532
* Fix flaky S3 test by setting default boto3 region by parasj in https://github.com/skyplane-project/skyplane/pull/534
* Add delays to object store tests to mitigate flaky AWS test by parasj in https://github.com/skyplane-project/skyplane/pull/557

Features
* Use spot instances to reduce transfer costs by parasj in https://github.com/skyplane-project/skyplane/pull/513
* [UX] Print better error message when transfer validation fails by liuxt129 in https://github.com/skyplane-project/skyplane/pull/535
* Support for Requester Pays by HaileyJang in https://github.com/skyplane-project/skyplane/pull/520
* Dispatch multipart transfer requests in parallel by parasj in https://github.com/skyplane-project/skyplane/pull/515
* On-prem support for Skyplane by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/540
* Small transfer support by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/544
* Fix integration test on main, extract fallback CLI commands, add Azure fallback support and add sync fallback support by parasj in https://github.com/skyplane-project/skyplane/pull/555

Bug fixes
* Fix edge cases for whole bucket transfers by parasj in https://github.com/skyplane-project/skyplane/pull/487
* Fix Docker build issues by parasj in https://github.com/skyplane-project/skyplane/pull/498
* Change multipart_min_threshold_mb from 64 to 128 by JasonDing0401 in https://github.com/skyplane-project/skyplane/pull/500
* Removing dollar sign from code blocks to mitigate $ copying by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/506
* Azure Authentication 2.0 (User Managed Identities) by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/505
* reenabling azure verification by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/507
* Allow for zero-byte objects by parasj in https://github.com/skyplane-project/skyplane/pull/519
* Check if Skyplane config is correct by parasj in https://github.com/skyplane-project/skyplane/pull/518
* fixing on-demand bug by antonzabreyko in https://github.com/skyplane-project/skyplane/pull/533
* Misc fixes for experiment update by parasj in https://github.com/skyplane-project/skyplane/pull/516
* Move `enrich_dest_objs` to separate func by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/527
* Fix: cloud to cloud cli error by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/542
* fix:minor bugs by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/547
* Dev/shishir/fix by ShishirPatil in https://github.com/skyplane-project/skyplane/pull/549
* Remove unused Azure config items from SkyplaneConfig by abiswal2001 in https://github.com/skyplane-project/skyplane/pull/522

New Contributors
* HaileyJang made their first contribution in https://github.com/skyplane-project/skyplane/pull/501
* liuxt129 made their first contribution in https://github.com/skyplane-project/skyplane/pull/535

**Full Changelog**: https://github.com/skyplane-project/skyplane/compare/0.1.2...0.1.3