Welcome to Cog v0.5.0! It's been a while since we've made a release, and there's a lot of stuff going on here.
**If you're using the Redis queue worker, this is a breaking release.** If you're not using that worker, this release is backwards compatible with previous versions of Cog.
Almost all of this release is about the Redis queue worker. We've been rebuilding the infrastructure for Replicate – improving the reliability and performance – and in the process have been moving some of the features of our serving platform directly into Cog. If you're using Cog to run your own models in production, then these changes are for you.
We're not done yet though! We're currently working on the HTTP interface, bringing it up to scratch. And we've got some big plans to use that updated interface to make serving machine learning models even easier and safer.
Changes
First up, a couple of quality of life fixes:
* Stop container when `ctrl+c` is pressed during `cog predict` 751
* Fix builds with `buildx` 798
And now, the Redis queue worker:
* can send responses back as HTTP webhook calls, instead of via Redis (727)
* can filter the events it'll send webhooks for (807)
* has an updated response format, matching the API for Replicate (729 & 743)
* makes file uploads as a basic PUT request, so it can upload directly to cloud storage, without needing special endpoints (735)
* can be configured to exit after too many failures, to protect against workers getting into a bad state (800)
* sends status, timings and logs for setup to a URL, so you can monitor performance and react to failures (803)
* writes a file when setup is complete, so you can create readiness checks via tools like Kubernetes (805)
* can be started with keyword arguments, instead of an increasingly long list of positional arguments (775)
* can handle cancelations internally, by giving it a Redis key to watch for changes (746)
* has a completely rewritten core, to improve the reliability and prepare us for updating the HTTP server to support most of the features above (795)
Full changelog
* 0c3466c Accept list of events to send webhooks for
* 93367e4 Add CUDA 11.8 and Tensorflow 2.11.0
* 01e9f7f Add metrics.predict_time to response output
* a3b97e6 Add option to exit redis worker after too many failures
* 1e5435d Add support for --extra-index-url to PyTorch
* fe5696d Add tests for StreamRedirector, and make line buffering consistent
* d74c142 Add the ability to suppress teeing of subprocess output
* 5cf6d0a Add unit tests for WrappedStream
* a3ca53a Add user-agent to webhook calls
* e44593c Add write()/flush() to WrappedStream
* ae2cb44 Address timing issue with cancel signal
* 1f8fec1 Bodge in support for Python 3.11
* c8058fb Bump github.com/docker/cli
* e3df72f Bump github.com/docker/docker
* 1683b8b Bump github.com/getkin/kin-openapi from 0.109.0 to 0.110.0
* 72c2839 Bump github.com/getkin/kin-openapi from 0.98.0 to 0.109.0
* 6799530 Bump github.com/golangci/golangci-lint from 1.49.0 to 1.50.0
* 909a184 Bump github.com/golangci/golangci-lint from 1.50.0 to 1.50.1
* 3fa833d Bump github.com/spf13/cobra from 1.5.0 to 1.6.1
* 5c489f3 Bump golang.org/x/tools from 0.1.12 to 0.3.0
* 338db31 Bump pillow from 9.0.1 to 9.3.0
* cef47fa Caller is responsible for wrapping streams
* 436b12d Canceled is a terminal status
* 5ad419f Echo back request fields in queue worker responses
* 8470901 Extract Dockerfile temporary file logic
* 7afd940 Extract format_datetime method for queue worker
* 017ac89 Fetch latest PyTorch version from pip repository
* 3d43593 Fix a small logic bug
* 7003fec Fix deadlock for large outputs
* 12137aa Fix past PyTorch version parsing
* 8c4e84a Fix tests
* 51ed797 Fix typo
* 2314dc2 Flush tee-d output after each line
* a55ad5a Handle cancelations in Redis queue worker
* c2c8c83 Handle dead predictor process
* e151d1d Install Python requirements from file
* 8f1f419 Log cancel signals to stderr
* 4782420 Loosen attrs dependency as far as possible
* 523b8f1 Make "Status" a string forward ref to avoid test errors
* f511232 Make a start on a more realistic state-based fuzz test
* 7230dc8 Make optional types explicit
* d6fc5d6 Minor setup error handling tweaks for clarity
* e06bc35 Move handling of Done into _predict
* 415918a Move x-experimental-timestamps to response root
* 2587fc1 Mypy fixes
* 1676cf4 New Worker implementation (replaces Runner)
* 1ac5b92 Pass response_interval to ResponseThrottler
* 987ebb8 Pass through lines in requirements.txt that aren't pinned
* 508eabc Put pip --find-links in requirements.txt instead of inline
* 17ee469 Read python_requirements and resolve versions from it
* a2df984 Refactor redis_queue to extract response building
* d3b0592 Remove python_extra_index_urls and python_find_links
* 248a733 Rename IndexURL to correct "FindLinks"
* 2e596a2 Rename PythonPackagesForArch to PythonRequirementsForArch
* 4890fac Rename ValidateAndCompleteConfig to ValidateAndComplete
* b516768 Require protobuf<=3.20.3
* e822bc8 Retry webhook for terminal responses
* 8ceb8fd Return a string from PythonRequirementsForArch
* df3b04c Satisfy the mypy gods
* 2fe5872 Send file uploads as a basic PUT
* cf88ed4 Send logs lines as newline-separated string instead of list
* 7caa858 Send timings and logs for setup to a URL
* 8798eea Set HYPOTHESIS_PROFILE for GitHub Actions
* 74b05e0 Set default throttle interval to 500ms
* 936afd7 Shut down if the runner died
* 4b54bab Switch to keyword arguments for queue worker
* ec63f96 Throttle webhook responses
* ce41c84 Update python.md
* 2d80011 Update yaml.md
* 01600e1 Use ValidateAndCompleteConfig in config tests
* 0ed776c Use future annotations instead of string type hint
* 4adcb5d Work around STDIO buffering in tests
* efbe725 Write a file when setup is complete
* 2483538 Yet more protection around cancelation!
* 002c14e black/isort run
* 745f003 gitignore .DS_Store
* 469d36d load flag is required in buildx
* 31872fd python-openssl system package not used
* e5dafee rename installpython function to include scope - cuda
* 76e9289 stop container on ctrl-c during cog predict (751)
* 63ca65d update .all-contributorsrc [skip ci]
* 78f61e7 update README.md [skip ci]
* 163eef7 update docs to 1.18 version of Go (used in CI)