Cog

Changelog
* a872309 Abort prediction on a failed healthcheck event
* 5f51a96 Add --await-explicit-shutdown option to HTTP server
* 8aeb7f1 Add `--redis-url` argument for connecting to redis from URL Signed-off-by: Brandon Berhent <bbedwardgmail.com>
* 1f80fe8 Add a "starting" state for predictions
* bb2baf1 Add a healthcheck endpoint to cog HTTP server
* 3418f8a Add asynchronous create prediction endpoint
* 4753da9 Add cancel prediction endpoint
* 6c206c9 Add cog.director for coordinating predictions
* d86938f Add missing FileUploadError
* f90078f Add missing continue during health confirmation
* 047567a Add some more logging for cancels and time outs
* 3106496 Add webhook_caller_filtered
* 371a562 Add webhooks to HTTP server predictions
* 0a81808 Allow requesting health status explicitly
* b02b3f6 Attempt to improve cooperative shutdown
* 6fa42f5 Bind context for director logs while handling prediction
* 0068f06 Bump numpy to support Python 3.11
* c4e6920 Bump to 64 cores
* 4645c31 Create event handler in PredictionRunner
* 9b710a5 Dampen down the noise from library loggers
* 6e773bc Decouple webserver lifecycle from director
* fb8f377 Don't allow updates for the wrong prediction
* 8dc78e4 Don't duplicate tests in PRs
* b8cab4b Don't guess at a free port, let the OS pick one
* 3b316bb Don't trust model container's timestamps
* c25d007 Ensure backwards-compatible file uploads
* b3f0ef6 Extend CANCEL_WAIT to 30s
* 0a128c9 Extend prediction create timeout to 5s
* 80a1c30 Fix async endpoint test for start webhooks
* 3aad265 Fix openapi schema generation
* 2a10398 Fix some types and clean up some imports
* 85b0aa8 Fix tests for python 3.7
* 56fa616 Fix up setup run reporting
* 08668a1 Fixed a problem with OpenAPI spec generation
* d16f972 Flesh out director logging
* 9284718 Flesh out model container logging
* 975ad2e Fully invert control between QueueWorker and the webserver
* 12825f9 Handle failure/success reporting correctly
* a9b5c40 Handle file uploads to PUT endpoint from HTTP server
* 996a22f Handle prediction timeouts
* 498ae09 Handle validation errors and failures to cancel
* fa4b3fe Implement PredictionRunner.cancel()
* 85e6caf Invert control of webserver
* 90b5da1 Loosen OpenAPI spec assertion for FastAPI 0.89.0
* 0fe74e2 Make empty logs an empty string, not None
* 347bdd3 Make the abort message required
* 158d663 Move model healthchecking into own thread
* 228ea78 Only check if succeeded if terminal check passes
* 907be82 Only set content type headers if there is a content type to set
* 4bb172c Only set started_at after prediction has started
* 00a0b9e Preserve prediction status after slow cancelation
* ef02a2c Properly crash the container when predict thread errors
* 999f6c7 Reclassify FIXME and TODOs
* a1c588c Refactor redis queue tests
* 2a1cf58 Reimplement max_failure_count in Director
* d31d6e4 Remove Redis XAUTOCLAIM and ack messages whenever possible
* 39c8763 Remove extra response properties from webhook body
* f916be9 Remove unnecessary _last_result variable
* dc1449f Remove unused model_id and log_queue parameters
* 8805a5f Rename QueueWorker to Director
* b0a1e2a Rename shutdown to start_shutdown to avoid name collision
* 73a845e Report setup runs from director
* f17eeae Request and wait for an explicit healthcheck before each prediction
* 8e647e3 Retry file upload requests
* d9edff4 Return 0 runtime duration if prediction hasn't started
* 2372e28 Revert "Add `--redis-url` argument for connecting to redis from URL"
* 90e34eb Revert "Bump to 64 cores"
* ebc3a2a Revert "Revert "Add `--redis-url` argument for connecting to redis from URL""
* 9687042 Revert how COG_VERSION is set in cog Makefile target
* 606aac2 Run Python tests in parallel
* df9c3ab Run integration tests in parallel
* bc681c2 SIGKILL when http server receives shutdown call
* 4c2e419 Send webhooks from HTTP runner
* 85ec456 Set default timeouts for file uploads
* 0fab9a8 Shut down model container on graceful termination
* 5751cf7 Split director package
* cbf2088 Standardize on structlog logging
* ccba7c6 Terminate worker before waiting for thread exit
* c7d4f3f Tighten up error handling in PredictionRunner
* bf3649d Unconditionally SIGKILL when shutting down
* f75b695 Update HTTP API docs
* 7f5b598 Update README.md
* f5e5b5b Update instructions for installing cog for development
* d6963a7 Update integration tests to work with director
* 85c7d64 Use JSON logging in production
* ab79aee Use faster GitHub runners
* 8570995 cog: log prediction id
* f0b27cd update .all-contributorsrc [skip ci]
* 08f4c8f update .all-contributorsrc [skip ci]
* ff646be update README.md [skip ci]
* 7bfbb7c update README.md [skip ci]

Changelog
* 696d9ef Fix a broken assertion

Welcome to Cog v0.6.0! Three changes this time:

- Configure a new docker entrypoint. This makes sure the container respects SIGTERM. It also fixes an issue caused by Nvidia changing entrypoints in all CUDA base images. https://github.com/replicate/cog/pull/844
- Expose the version of Cog within the Python package, via `cog.__version__`. https://github.com/replicate/cog/pull/843
- Update the HTTP API to use the new `Worker` class introduced in v0.5.0. This is the first step towards an asynchronous API that matches Replicate's API. https://github.com/replicate/cog/pull/824

Full changelog
* 0bb3ace Add a busy test for PredictionRunner
* 79851e8 Add tests for cog.server.runner.predict()
* d2726e0 Configure Cog python package version from git
* 7f49116 Extract predictor fixtures for Cog HTTP tests
* 5303d8a Install tini as image entrypoint
* a9569db Lookup the schema for the predictions API by path
* 04c4215 Make File inputs survive pickling/unpickling
* 8b67617 Use Worker for Cog HTTP API and respond like Replicate
* bbe999e document `brew install cog`

Changelog
* bf08e07 Add an explicit handler to die on SIGTERM
* 1e13500 Add anchor for removed Update section
* 25ba72d Add cog target
* d934417 Add console language identifier to code blocks
* f447eec Add default all target
* 26aa17f Add instructions for building from source
* b161aca Add uninstall target
* 3e5d555 Bump golang.org/x/sys from 0.2.0 to 0.3.0
* e4131de Bump golang.org/x/tools from 0.3.0 to 0.4.0
* fa87b16 Consolidate instructions for updating
* d949133 Copy editing Install section
* 8cc4fe9 Define constants for go and python executables
* 3cc673a Don't do Python stuff when testing Go
* 3bb2169 Don't pin dependencies that are unpinned in setup.py
* d83f75b Fix CI
* 0ded3c3 Ignore built cog executable
* f2f5199 Make cog HTTP server port configurable
* 90c4ce4 Only ignore the built cog binary, not anything named cog :)
* a94662e Read go version from go.mod in CI workflow definition
* fd5f712 Remove dollar sign prefix in pastable commands
* 3800f83 Remove instruction to download Docker from Install section
* d83584d Set LDFLAGS in cog target
* f7a2cd8 Specify requirements for integration tests
* 6b68d03 Support staged installs
* 0adf04e Update install target
* 8685a32 Update tests and FastAPI pin for schema changes
* 2ed0360 Upgrade to Go 1.19
* 0d7bbad Use built executables for integration tests
* 2fdeefa Use dollar sign for command substitution
* b58bd63 empty commit to trigger CI when next tag is created
* 1772f69 update .all-contributorsrc [skip ci]
* 9593f7c update .all-contributorsrc [skip ci]
* 9f6ebcd update .all-contributorsrc [skip ci]
* fa6aeb8 update .all-contributorsrc [skip ci]
* 6a6d5c0 update .all-contributorsrc [skip ci]
* 3ccea04 update .all-contributorsrc [skip ci]
* 340d8fb update .all-contributorsrc [skip ci]
* b94f0df update .all-contributorsrc [skip ci]
* 268dec0 update README.md [skip ci]
* ba10aa5 update README.md [skip ci]
* af3cf46 update README.md [skip ci]
* 6572830 update README.md [skip ci]
* febe58f update README.md [skip ci]
* 9b35dcf update README.md [skip ci]
* 35110c9 update README.md [skip ci]
* c275af2 update README.md [skip ci]

Welcome to Cog v0.5.0! It's been a while since we've made a release, and there's a lot of stuff going on here.

**If you're using the Redis queue worker, this is a breaking release.** If you're not using that worker, this release is backwards compatible with previous versions of Cog.

Almost all of this release is about the Redis queue worker. We've been rebuilding the infrastructure for Replicate – improving the reliability and performance – and in the process have been moving some of the features of our serving platform directly into Cog. If you're using Cog to run your own models in production, then these changes are for you.

We're not done yet though! We're currently working on the HTTP interface, bringing it up to scratch. And we've got some big plans to use that updated interface to make serving machine learning models even easier and safer.

Changes

First up, a couple of quality of life fixes:

* Stop container when `ctrl+c` is pressed during `cog predict` 751
* Fix builds with `buildx` 798

And now, the Redis queue worker:

* can send responses back as HTTP webhook calls, instead of via Redis (727)
* can filter the events it'll send webhooks for (807)
* has an updated response format, matching the API for Replicate (729 & 743)
* makes file uploads as a basic PUT request, so it can upload directly to cloud storage, without needing special endpoints (735)
* can be configured to exit after too many failures, to protect against workers getting into a bad state (800)
* sends status, timings and logs for setup to a URL, so you can monitor performance and react to failures (803)
* writes a file when setup is complete, so you can create readiness checks via tools like Kubernetes (805)
* can be started with keyword arguments, instead of an increasingly long list of positional arguments (775)
* can handle cancelations internally, by giving it a Redis key to watch for changes (746)
* has a completely rewritten core, to improve the reliability and prepare us for updating the HTTP server to support most of the features above (795)

Full changelog
* 0c3466c Accept list of events to send webhooks for
* 93367e4 Add CUDA 11.8 and Tensorflow 2.11.0
* 01e9f7f Add metrics.predict_time to response output
* a3b97e6 Add option to exit redis worker after too many failures
* 1e5435d Add support for --extra-index-url to PyTorch
* fe5696d Add tests for StreamRedirector, and make line buffering consistent
* d74c142 Add the ability to suppress teeing of subprocess output
* 5cf6d0a Add unit tests for WrappedStream
* a3ca53a Add user-agent to webhook calls
* e44593c Add write()/flush() to WrappedStream
* ae2cb44 Address timing issue with cancel signal
* 1f8fec1 Bodge in support for Python 3.11
* c8058fb Bump github.com/docker/cli
* e3df72f Bump github.com/docker/docker
* 1683b8b Bump github.com/getkin/kin-openapi from 0.109.0 to 0.110.0
* 72c2839 Bump github.com/getkin/kin-openapi from 0.98.0 to 0.109.0
* 6799530 Bump github.com/golangci/golangci-lint from 1.49.0 to 1.50.0
* 909a184 Bump github.com/golangci/golangci-lint from 1.50.0 to 1.50.1
* 3fa833d Bump github.com/spf13/cobra from 1.5.0 to 1.6.1
* 5c489f3 Bump golang.org/x/tools from 0.1.12 to 0.3.0
* 338db31 Bump pillow from 9.0.1 to 9.3.0
* cef47fa Caller is responsible for wrapping streams
* 436b12d Canceled is a terminal status
* 5ad419f Echo back request fields in queue worker responses
* 8470901 Extract Dockerfile temporary file logic
* 7afd940 Extract format_datetime method for queue worker
* 017ac89 Fetch latest PyTorch version from pip repository
* 3d43593 Fix a small logic bug
* 7003fec Fix deadlock for large outputs
* 12137aa Fix past PyTorch version parsing
* 8c4e84a Fix tests
* 51ed797 Fix typo
* 2314dc2 Flush tee-d output after each line
* a55ad5a Handle cancelations in Redis queue worker
* c2c8c83 Handle dead predictor process
* e151d1d Install Python requirements from file
* 8f1f419 Log cancel signals to stderr
* 4782420 Loosen attrs dependency as far as possible
* 523b8f1 Make "Status" a string forward ref to avoid test errors
* f511232 Make a start on a more realistic state-based fuzz test
* 7230dc8 Make optional types explicit
* d6fc5d6 Minor setup error handling tweaks for clarity
* e06bc35 Move handling of Done into _predict
* 415918a Move x-experimental-timestamps to response root
* 2587fc1 Mypy fixes
* 1676cf4 New Worker implementation (replaces Runner)
* 1ac5b92 Pass response_interval to ResponseThrottler
* 987ebb8 Pass through lines in requirements.txt that aren't pinned
* 508eabc Put pip --find-links in requirements.txt instead of inline
* 17ee469 Read python_requirements and resolve versions from it
* a2df984 Refactor redis_queue to extract response building
* d3b0592 Remove python_extra_index_urls and python_find_links
* 248a733 Rename IndexURL to correct "FindLinks"
* 2e596a2 Rename PythonPackagesForArch to PythonRequirementsForArch
* 4890fac Rename ValidateAndCompleteConfig to ValidateAndComplete
* b516768 Require protobuf<=3.20.3
* e822bc8 Retry webhook for terminal responses
* 8ceb8fd Return a string from PythonRequirementsForArch
* df3b04c Satisfy the mypy gods
* 2fe5872 Send file uploads as a basic PUT
* cf88ed4 Send logs lines as newline-separated string instead of list
* 7caa858 Send timings and logs for setup to a URL
* 8798eea Set HYPOTHESIS_PROFILE for GitHub Actions
* 74b05e0 Set default throttle interval to 500ms
* 936afd7 Shut down if the runner died
* 4b54bab Switch to keyword arguments for queue worker
* ec63f96 Throttle webhook responses
* ce41c84 Update python.md
* 2d80011 Update yaml.md
* 01600e1 Use ValidateAndCompleteConfig in config tests
* 0ed776c Use future annotations instead of string type hint
* 4adcb5d Work around STDIO buffering in tests
* efbe725 Write a file when setup is complete
* 2483538 Yet more protection around cancelation!
* 002c14e black/isort run
* 745f003 gitignore .DS_Store
* 469d36d load flag is required in buildx
* 31872fd python-openssl system package not used
* e5dafee rename installpython function to include scope - cuda
* 76e9289 stop container on ctrl-c during cog predict (751)
* 63ca65d update .all-contributorsrc [skip ci]
* 78f61e7 update README.md [skip ci]
* 163eef7 update docs to 1.18 version of Go (used in CI)