Authors
- Ara Ghukasyan <38226926+araghukasusers.noreply.github.com>
- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]users.noreply.github.com>
- Andrew S. Rosen <asrosen93gmail.com>
- Co-authored-by: Will Cunningham <wjcunningham7users.noreply.github.com>
- Co-authored-by: Sankalp Sanand <sankalpagnostiq.ai>
- Kevin Taylor <tkdtaylorgmail.com>
- FilipBolt <filipboltgmail.com>
- Co-authored-by: dependabot[bot] <49699333+dependabot[bot]users.noreply.github.com>
- Co-authored-by: Will Cunningham <wjcunningham7gmail.com>
- Co-authored-by: Prasy12 <prasanna.venkateshpsiog.com>
- Aviral Katiyar <123640350+maskboyAviusers.noreply.github.com>
- Co-authored-by: ArunPsiog <arun.mukeshpsiog.com>
- Casey Jao <caseyagnostiq.ai>
- Arnav Kohli <95236897+THEGAMECHANGER416users.noreply.github.com>
- Kirill Pushkarev <71515921+kirill-pushusers.noreply.github.com>
- Aditya Raj Kashyap <95625520+AdityaRaj23users.noreply.github.com>
- ArunPsiog <106462226+ArunPsiogusers.noreply.github.com>
- mpvgithub <107603631+mpvgithubusers.noreply.github.com>
- Aravind <100823292+Aravind-psiogusers.noreply.github.com>
- Faiyaz Hasan <faiyazagnostiq.ai>
- Co-authored-by: Venkat Bala <balavk89gmail.com>
- Co-authored-by: kessler-frost <ssanandhawk.iit.edu>
- Co-authored-by: Aravind-psiog <aravind.prabaharanpsiog.com>
- Co-authored-by: Manjunath PV <manjunath.poilathpsiog.com>
- Co-authored-by: Ara Ghukasyan <araagnostiq.ai>
- Co-authored-by: Alejandro Esquivel <aealejandro.ltd>
- Co-authored-by: jackbaker1001 <jsbaker1001gmail.com>
- Co-authored-by: Santosh kumar <29346072+santoshkumarradhausers.noreply.github.com>
- Co-authored-by: Will Cunningham <willagnostiq.ai>
- Co-authored-by: sriranjani venkatesan <sriranjani.venkatesanpsiog.com>
- Co-authored-by: Prasanna Venkatesh <54540812+Prasy12users.noreply.github.com>
- WingCode <smallstar1234gmail.com>
- Nick Tyler <nicholas.s.tyler.4gmail.com>
- Co-authored-by: RaviPsiog <raviteja.gurrampsiog.com>
- dwelsch-esi <116022979+dwelsch-esiusers.noreply.github.com>
- Co-authored-by: dwelsch-memverge <david.welschmemverge.com>
- Madhur Tandon <20173739+madhur-tandonusers.noreply.github.com>
- Co-authored-by: kamalesh.suresh <kamalesh.sureshpsiog.com>
- Co-authored-by: santoshkumarradha <santoshagnostiq.ai>
- Janosh Riebesell <janosh.riebesellgmail.com>
- Madhur Tandon <madhurtandon23gmail.com>
- Rob de Wit <RCdeWitusers.noreply.github.com>
- Venkat Bala <15014089+venkatBalausers.noreply.github.com>
- Co-authored-by: Venkat Bala <venkatagnostiq.ai>
- Co-authored-by: Amalan Jenicious F <amalan.jeniciouspsiog.com>
- RaviPsiog <111348352+RaviPsiogusers.noreply.github.com>
- Co-authored-by: RaviPsiog <ravieja.gurrampsiog.com>
- Akalanka <8133713+boneyagusers.noreply.github.com>
- Co-authored-by: Scott Wyman Neagle <scottagnostiq.ai>
- Scott Wyman Neagle <wymneaprotonmail.com>
Added
- check for `/bin/bash` AND `/bin/sh` (in that order) to execute bash leptons
- Programmatic equivalents of CLI commands `covalent start` and `covalent stop`
- Documentation and test cases for database triggers.
- Added the `__pow__` method to the `Electron` class
- New Runner and executor API to bypass server-side memory when running tasks.
- Added qelectron_db as an asset to be transferred from executor's machine to covalent server
- New methods to qelectron_utils, replacing the old ones
- Covalent deploy CLI tool added - allows provisioning infras directly from covalent
- Added a `py.typed` file to support type-checking
- Corrected support from distributed Hamiltonian expval calculations
- Exposed qelectron db in sdk result object
- UI changes added for qelectrons and fix for related config file corruption
- UI fix regarding Qelectron not showing up
- Performance optimisation of UI for large Qelectrons
- File transfer strategy for GCP storage
- Add CLI status for zombie, stopped process.
- Fix for double locking file in configurations.
- Introduced new data access layer
- Introduced Shutil file transfer strategy for local file transfers
- File transfer strategy for Azure blob storage
- `executor` property to `Electron` class, allowing updation of executor after electron function definition
- Added ability to hide post-processing electrons on the UI.
- Added prettify of names for the graph screen on the UI.
- Ability to specify a `workdir` for `local` and `dask` executors along with `create_unique_workdir` option for each electron / node.
- Heartbeat file is created and updated when the server is running
- Added `SQLiteTrigger` class to the `triggers` module.
- Added the `CloudResourceManager` class
- A new tutorial for a dynamic quantum chemistry workflow
- Tutorial for hybrid neural network using Covalent, AWSBatch, and Qiskit Runtime.
- Environment variable that users can set to specify the location where Covalent can find their defined executors.
- Task group id in Electrons.
- Reconstruct postprocessing method.
- Postprocessing as electrons.
- Postprocessing class in `postprocessing.py` module for all the different postprocessing helper methods and algorithms.
- `disable_run` option added to enable "saving-only" option on covalent server and not executing the workflow
- `register_triggers`, `stop_triggers` functions added to `LocalDispatcher` class
- `triggers` parameter to the lattice metadata
- `BaseTrigger`, `DirTrigger`, `TimeTrigger` classes added available to be assigned to any lattice enabling the triggers feature
- `TriggerLoader` class added enabling loading of any kind of triggers including user defined ones without requiring installation
- CLI options to start covalent server in triggers only, and no triggers mode
- `is_pending` option added during redispatch to resume execution of a previously "saved-only", i.e pending workflow
- API routes added for Triggers server
- Added feature to support cancelling workflow dispatches
- Updating/adding new tests to improve code coverage
- API endpoint for redispatching.
- Unit and functional tests for redispatching.
- TransportGraphOps class for diffing operations on transport graphs.
- Added make derived dispatch method.
- Apply electron updates method to \_TransportGraph.
- Workflow re-dispatching functionality.
- Added `dispatcher_addr` argument to `ct.get_result` similar to `ct.dispatch` so that it doesn't always fallback to using the default configured address
- Front-end pending unit tests for the GUI.
- Adding support for PostgresQL DB backend
- Added check for `COVALENT_DATABASE_URL`, if exists connect sqlalchemy engine using that
- Adding `COVALENT_DATABASE_USER` and `COVALENT_DATABASE_PASSWORD` environment variables
- Adding `COVALENT_DATABASE_HOSTNAME` and `COVALENT_DATABASE_PORT` environment variables for easy configuration
- Added `TaskRuntimeError` exception for executor plugin implementations to signal to Covalent that a task raised an
unhandled exception while running in the executor backend.
- Added environment variable for a remote database backend
- Added support for mysql and postgresql
- Support for transferring the contents of folders to and from S3 buckets using the file transfer module.
- RTD for User Interface
- Minor GUI fixes
- Ability to use terminal on the GUI.
Changed
- Terraform output to use scrolling buffer.
- Terraform output handling to show errors.
- Bumped up babel/traverse version to 7.23.2
- Changed the axios version on the webapp side.
- Changed the azurebatch.rst banner from default covalent jpg to azure batch's svg file
- [Significant Changes] Improving memory management part 1/3
- Removed strict version pins on `lmdbm`, `mpire`, `orjson`, and `pennylane`
- Changed license to Apache
- Improved error handling in generate_docs.py
- [Significant Changes] Migrated core server-side code to new data access layer.
- Changed the way UI was accessing the qelectron database to access it directly from the mdb file in object store
- Update version of browserverify-sign
- Limiting cloudpickle version to less than 3.0 for now
- Raised the minimum version of Pydantic from 1.10.1 to 2.1.1 in `requirements.txt`
- Electron DAL to use Covalent server's data instead of QServer's data.
- Renamed QCluster's `selector_serialized` attribute so it gets propagated to the qserver.
- Removed `orm_mode = True` in `covalent_ui/api/v1/models/dispatch_model.py` as it is deprecated in Pydantic 2
- Removed the upper limit from `dask` and `distributed` packages' versions until we find a version which is incompatible with Covalent.
- When the server is stopped, any workflows in a non-terminal state are first cancelled
- Pinned sqlalchemy version with upper limit <2.0.0.
- Added rich support to cli for better printing statements.
- Performed minor modifications and rearrangementsto fix the broken tests in the `nightly` workflow.
- Reverting nightly frequency back to midnight basis
- Temporarily changing the nightly frequency to every hour
- Error messages are propagated to stdout when the server is not started. These changes are applied to `dispatch`, `redispatch`, and `get_result`.
- Implementation of `TransportableObject` property method to be backwards compatible with version 0.209.1.
- Moved TransportableObject from transport.py to a new file transportable_object.py.
- Postprocessing logic.
- Sublattice logic. Sublattices are now treated as electrons. Once the transport graph has been built, the status get changed to `DISPATCHING` at which point it is executed as another workflow.
- Modified `ct.get_result` to allow for status only requests
- Enhanced the Dockerfile to include builds from various sources and a differentiation between SDK and Server builds
- Location of function to load result from the database now moved to load module in covalent_dispatcher/\_db folde.
- Changed decode-uri-component package version on webapp yarn-lock file.
- Changed json5 package version on webapp yarn-lock file.
- Reverted nightly frequency back to once a day
- Updated `requirements.txt` to include `pyscopg2`
- Refactored execution.py into loosely coupled modular pieces
- Updated `directory` like default environment variable paths to avoid creating redundant nested directories when self-hosting
- Docs for Covalent's Slurm plugin updated with explanation for optional `srun` parameters.
- Electron errors are segregated by type; task runtime errors are
stored in `stderr` while the `error` attribute of a node is reserved
for exceptions raised by Covalent itself.
- When tasks fail in a workflow, the Lattice ErrorCard in the UI summarizes the failed tasks.
- Width of lattice name column on dispatch list GUI.
- Optimzing larger graphs for better performance.
- Running migrations automatically if none have run in the past (fresh installs, after purging)
- Single line call to join instead of a for loop
- Updated black, mirrors-prettier, and detect-secrets in pre-commit hooks
- Made `root_dispatch_id` nullable to circumvent migration issues with sqlite in certain platforms
Removed
- Removed no longer needed methods in qelectron_utils
- Removed `dispatch-id` from generate_node_result method
- Removed unused module `covalent._data_store`
- Stress test files of cpu and sublattice stress tests removed from functional tests suite.
- Duplicate mocks for `UI_SRVDIR`
- Duplicate `_get_cancel_requested` method from `covalent_dispatcher/_core/runner.py`
- Postprocessing from runners.
- References to specific IBMQ hub/group/project in tutorial 5
- Slurm executor reference from qaoa tutorial since it's not compatible with conda env at the moment.
Fixed
- Lattice-default metadata attributes are now applied correctly
- Sublattices are built using `workflow_executor`
- Added covalent version attribute to Remote Executors
- Removed unassigned variable names
- Contributing guidelines steps for installing for the first time
- Updated gitignore to ignore yarn files and folders for latest version of yarn
- Fixed the bug that caused ValueError error when using KEYWORD_ONLY parameter in electron func
- Changed code at line 218 in covalent/_shared_files/utils.py
- Fixed usage of deprecated pydantic validation methods
- Fixed qelectron_db retrieval in result object
- Fixed editability of Qelectron on settings page - UI changes
- Certain pydantic v2 related updates
- Fixed lattice's metadata propagation to electron's metadata in case no metadata was provided to the electron
- Formatted executor block under Qelectron job details to handle any class-type values
- Fixed test-cases to handle latest pydantic version changes
- Rsync command fixed to recursively copy files when using SSH
- Removed accidentally added migrations build files
- Updated migration script to add a default value for `qelectron_data_exists` in the `electrons` table since it cannot be nullable
- Reduced server memory consumption during workflow processing
- Fixed dispatcher address not showing when covalent server starts.
- Fixed the failing tests in the `nightly` workflow.
- Using `filelock` package now for platform independent file locking of config file. This should fix the failing tests as well as improve compatibility with Windows.
- When stopping the server, we send the proper `SIGINT` signal to uvicorn instead of `SIGKILL` which allows the second part of the FastAPI `lifespan` to execute properly.
- Fixed the outstanding incompatibities between front-end data layer and a postgres database
- Reverted file-lock changes
- Fixed dispatches list UI api caused by pydantic config.
- Fixed graph API.
- Fixed UI backend unit test case.
- Executor and workflow executor data dictionaries are passed to sublattices
- Read in `README.md` with `encoding="utf-8"` in `setup.py`
- Resolving correct python executable
- Error handling for random URLs/random dispatchId entered on the GUI
- Fixed support for dynamically assigning `executor` to `Electron` class
- Fixed nightly by activating environment before running tests in `tests.yml`
- only pin `aiohttp` downwards to fix install on Python 3.11 [1654](https://github.com/AgnostiqHQ/covalent/pulls/1654)
- DB path creation now takes place at import time so that the CLI commands don't fail
- Raise error on dispatching a non-lattice
- Helpful message when trying to dispatch a workflow when covalent server is not available
- Open UI preview of transport graph when `lattice.draw()` is invoked and print URL of the same
- Defer creation of server specific config entries until covalent is started
- Functional tests on CI
- Move dask worker space to covalent cache instead of `site-packages`
- Result status comparison
- Raise error on extra args/kwargs
- Fixed redispatching and trigger server address passing in base trigger
- Doubling of nodes that are added to the transport graph.
- Ensure postprocessing node end time is added as the workflow end time.
- Functional tests
- Custom executor how to guide, and its loading mechanism in covalent server.
- Broken postprocessing unit test.
- UI crashing if time values are null
- No longer adding "http://" every time a dispatcher address is provided in local dispatcher class in order to use the provided address exactly
- Redispatch bug involving copying reusable nodes from old transport graph to new transport graph.
- Pennylane tutorial notebook.
- Redispatch bug.
- Braket pip installation instructions.
- Removing the entrypoint for SDK-only install
- Updating client requirements to match server versions
- Load plugins only when COVALENT_PLUGIN_LOAD environment variable has been set to a Truthy value.
- MNIST tutorial now shows non-Null outputs and the classifier training log image has been updated.
- Minor changes to tutorials: autoencoder, quantum and classical svm, ensemble classification, iris classification with Pennylane, quantum chemistry, DNN tutorial, qaoa, spacetime tutorial etc.
- The range of `networkx` versions in requirements.txt weren't compatible with each other, thus it is pinned to `2.8.6` now
- SDK-only sdist and installation should now work as expected, not packaging the server
- Fixed naming of collection nodes (was breaking postprocessing)
- Restored compatibility with stable release of AWS executors
- Optimization of logs on the GUI for large log file sizes.
- Fixed UI pagination not working for more than 11 pages
- Runtime field counting down for select running dispatches
- Removed merge conflict symbols in changelog
- Build graph now sets all unset lattice constraints from defaults
- Fixed all failing functional tests
- Fixed local executor tests on MacOS by adding ProcessPoolExecutor
- Respecting specified AWS profile & region in remote executed S3 file transfers, defaulting to env vars of execution backend
- Electrons will inherit the lattice executors.
- Sublattices inherit the parent lattice executor.
- When several electrons are running concurrently, their stdout and stderr are stored in the correct graph nodes.
- Electron errors now appear in the Electron ErrorCard when one clicks on a failed task in the UI.
- When an electron raises an exception during execution, the local and dask executors now try to recover any output that was already
written.
- Fixed functional tests.
- Added `requirements-client.txt` to MANIFEST file
- Respecting specified AWS profile & region in remote executed S3 file transfers, defaulting to env vars of execution backend
- Fixed local executor tests on MacOS (second attempt)
- The `initialize_results_dir` method attempts to use an environment variable instead of the results directory in the payload
- Modified certain sqlalchemy commands for postgres compatibility
- Removed references to results_dir in the payload
- Folder transfer unit test.
- Folder transfer download bug
- Result objects now print correctly when nodes fail
- Enabling logging by default
- Removed debugging output
- Clarify cli output when `covalent db migrate` needs to be run
- Rendering newlines in ErrorCard on the UI for displaying error stacktraces
- VERSION incrementing logic in changelog
- Fixed v11 migration to use render as batch to make DROP operations compatible with sqlite
- `covalent restart` honors the `sdk.no_cluster` setting
- Re-applying default executor fix post config file reunification
- Exceptions when instantiating executors are handled
- Covalent start now waits for the server to settle before returning
Tests
- Temporarily skipping the sqlite and database trigger functional tests
- Updated tests to accommodate the new qelectron fixes
- Added new tests for the Database class and qelectron_utils
- Covalent deploy CLI tool tests.
- Skipping functional tests for azure blob storage and gcp storage how to guides since they require credentials to run.
- Added testcases for GUI backend.
- Changed the method for startup and shutdown events for pytest to work with fastapi version 0.93.0
- Fixed test cases to adapt changes to SQLAlchemy version 1.4.49
- Ignored remote file transfer how-to functional tests.
- Skipping a UI backend test for now
- Fixed `test_decorated_function` test case in functional tests
- Added tests for the `CloudResourceManager` class
- Re-enable `test_run_workflow_does_not_deserialize`
- Updated QA stress test execution time baseline.
- Move QA scripts from QA repo to Covalent functional tests.
- Updated and added tests to account for all of the above triggers related changes
- Updated `_get_result_from_dispatcher` test to verify whether using a link directly works or not
Docs
- Added federated learning showcase code
- Updated tutorial for redispatching workflows with Streamlit
- Fix autodoc for SSH, Slurm, AWS Braket, AWS Lambda, AWS EC2, AWS Batch, Google Batch
- Updated documentation links in README
- Added tutorial for redispatching workflows with Streamlit
- Added documentation for Azure Blob Storage file transfers
- Added documentation for Google Cloud Storage file transfers
- Enhanced the quickstart with a set of commonly used features
- Removed duplicate "stop server" warning in the First Experimemnt page
- Fixed typo in quickstart
- Fix autodoc for SSH, Slurm, AWS Braket, AWS Lambda, AWS EC2, AWS Batch, Google Batch
- Updated documentation links in README
- Updating and reorganizing RTD API documentation
- Adding example links in API documentation
- Fix `Lattice` docstring in RTD.
- Added a missing `,` to the Slurm docs.
- Fix typo in GCP Batch executor RTD.
- Add steps for setting up GUI for local development in contribution guidelines.
- Fix docstring for set_config
- Redispatch feature page in Read the Docs.
- Clarify installation instructions for SLURM plugin in Read the Docs (x2).
- Fix waiting order of electrons in docs inside snippet for adding a dependency when inputs and outputs are independent.
- Expose GCP Batch executor RTD.
- Add GCP Batch executor image in RTD.
- Updated Feature documentation for Triggers with an example and minor corrections
- Update requirements file for the tutorials: `1_QuantumMachineLearning/pennylane_kernel/source.ipynb` and `machine_learning/dnn_comparison.ipynb`.
- Add macOS 13 (Ventura) to [compatibility list](doc/source/getting_started/compatibility.rst).
- Fixed broken links and typos in the documentation.
- Updated How-to documents.
- Port of Pennylane's Univariate QVR tutorial using Covalent to this repo.
- Adding troubleshooting guide to RTD's
- Added a note to First Experiment offering initial intro to executors.
- Adding Google Batch executor plugin RTD
- Adding `cancellation` RTD text files
- Added documentation for "Triggers" and a separate section for similar "Features"
- Tutorial guidelines
- Redispatch API section.
- Add how to for redispatch.
- Mention redispatch in the concepts section.
- Update `AWS Lambda` executor RTD with steps to extend the base executor image for installing custom packages
- Updated self-deployment (server deployment).
- Added IBMQ tutorial
- Published Self-Deployment Guide
- Revised UI reference. Added Settings page documentation.
- Added broken UI links in README
- Updated compatibility matrix
- Adding `Deployment` section for self-hosting guide
- Rewrote Concepts section in docs
- Split Concepts into API, server, and UI sections
- Added new examples and graphics for Concepts
- Added DNN tutorial
- Updated AWS Plugins install instructions
- Updated AWS Plugins documentation (minor fixes)
- Rewrote intro material in README.
- Changed "Citation" in the README.
- Renamed "Release Notes" to "What's New?" in the README. Updated What's New with a description of the newest GUI functionality.
- Added "Quick Start" guide.
- Updated and reorganized doc landing page.
- Rewrote "Getting Started" page.
- Broke out "Installing from Source" instructions to separate page.
- Corrected some API class names in headers.
- Added an executors-and-UI graphic.
- Adding `Deployment` section for self-hosting guide
- Rewrote intro material in README.
- Changed "Citation" in the README.
- Renamed "Release Notes" to "What's New?" in the README. Updated What's New with a description of the newest GUI functionality.
- Updated AWS Lambda executor docs to address conflict with using public ecr registries
- Fixed missing RTD content under API section for covalent, cli, leptons, deps, data transfer
- Updated RTD with details about the new AWS lambda executor interface
Operations
- Ignore custom executor plugin in how-to's when running `test_deploy_status` CLI test.
- Change the strict version pin on `pennylane` from `==0.33.1` to `>=0.31.1,<0.33.0`
- Updated `black` , `prettier`, `pycln` package versions in pre-commit config
- Changed `actions/checkoutv3` to `actions/checkoutv4` in CI
- Dependabot update to npm in changelog action
- Update tough-cookie to 4.1.3 version
- Added rich support to cli for better printing statements.
- Changed semver from 5.7.1 to 5.7.2 in package.json
- Updated word-wrap to 1.2.4 version
- Updated the nightly workflow's schedule
- Temporarily making the nightly workflow more frequent to test the fix for the failing tests
- Fixed failing tests
- Respecting node version as specified in `.nvmrc` file for tests workflow
- Bumped versions in pre-commit config
- Added prettier for markdown files.
- Reduce the number of pinned version numbers in the `setup.py`, `requirements.txt`, and `requirements-client.txt`
- Updated the `wci.yml` file with new features
- Bumped pre-commit versions
- Temporarily running nightly hourly to test whether the fix worked
- Reverted to daily frequency for nightly
- Update the PR template.
- Nightly frequency set to midnight EST
- Removed "already released" check from stable changelog action
- Updating `nightly` frequency (temp)
- Lowering number of jest workers as an attempt to fix flaky UI functional tests
- Added exception for nightly to pass if conda release fails
- Updated pre-commit hook versions
- Updated codecov upload steps in tests workflow to fail if upload to codecov fails
- Added job in `nightly` to trigger base executor image builds after a Covalent `pre-release`
- Fixed stable-changelog action removed `.catch` added `.on('error')`
- Removed AWS base executor deployment from `release.yml`
- Removed experimental tests from nightly test matrix (will be brought back but in different workflow)
- Updated release workflow to continue if release tag already exists
- Reverted nightly schedule back to daily at 4:00am
- Added Alejandro to PAUL_BLART group to allow trigerring of releases
- Updated hotfix logic to run on a merge to a release branch
- CodeQL workflow uses a test matrix to scan all repos in the Covalent ecosystem
- Updated all CI Slack alerts to all go to the covalent-ci channel
- Removed PAUL_BLART check on build sdist step in release.yml
- Consolidated pre & stable build into one step in release.yml
- Fixing the validate distribution step given changes in -rc0 suffix to version
- updated hotfix logic to run on a merge to a release branch
- Fixing js github actions dist by re-building from develop
- Fixing syntax in describe action & compiled action manually