Calliope

|changed| Updated to Python 3.9, with compatibility testing continuing for versions 3.8 and 3.9. Multi-platform CI tests are run on Python 3.9 instead of Python 3.8. CI tests on a Linux machine are also run for versions 3.7 and 3.8. This has been explicitly mentioned in the documentation.

|changed| Updated to Click 8.0.

|changed| Updated CBC Windows binary link in documentation to version 2.10.8.

|fixed| SPORES mode scoring will ignore technologies with energy capacities that are equal to their minimum capacities (i.e., `energy_cap_min`) or which have fixed energy capacities (`energy_cap_equals`).

|fixed| SPORE number is retained when continuing a model run in SPORES mode when solutions already exist for SPORE >= 1. Previously, the SPORE number would be reset to zero.

|fixed| Malformed carrier-specific group constraints are skipped without skipping all subsequent group constraints.

|fixed| Spurious negative values in `storage_inital` in operate mode are ignored in subsequent optimisation runs (379). Negative values are a result of optimisation tolerances allowing a strictly positive decision variable to end up with (very small in magnitude) negative values. Forcing these to zero between operate mode runs ensures that Pyomo doesn't raise an exception that input values are outside the valid domain (NonNegativeReals).

|fixed| `om_annual` investment costs will be calculated for technologies with only an `om_annual` cost defined in their configuration (373). Previously, no investment costs would be calculated in this edge case.

|new| run configuration parameter to enable relaxation of the `demand_share_per_timestep_decision` constraint.

|new| `storage_cap_min/equals/max` group constraints added.

|changed| Updated to Pyomo 6.2, pandas 1.3, xarray 0.20, numpy 1.20.

|changed| |backwards-incompatible| parameters defaulting to False now default to None, to avoid confusion with zero. To 'switch off' a constraint, a user should now set it to 'null' rather than 'false' in their YAML configuration.

|changed| `INFO` logging level includes logs for dataset cleaning steps before saving to NetCDF and for instantiation of timeseries clustering/resampling (if taking place).

|fixed| `demand_share_per_timestep_decision` constraint set includes all expected (location, technology, carrier) items. In the previous version, not all expected items were captured.

|fixed| Mixed dtype xarray dataset variables, where one dtype is boolean, are converted to float if possible. This overcomes an error whereby the NetCDF file cannot be created due to a mixed dtype variable.

|new| `spores` run mode can skip the cost-optimal run, with the user providing initial conditions for `spores_score` and slack system cost.

|new| Support for Pyomo's `gurobi_persistent` solver interface, which enables a more memory- and time-efficient update and re-running of models. A new backend interface has been added to re-build constraints / the objective in the Gurobi persistent solver after updating Pyomo parameters.

|new| A scenario can now be a mix of overrides *and* other scenarios, not just overrides.

|new| `model.backend.rerun()` can work with both `spores` and `plan` run modes (previously only `plan` worked). In the `spores` case, this only works with a built backend that has not been previously run (i.e. `model.run(build_only=True)`), but allows a user to update constraints etc. before running the SPORES method.

|changed| |backwards-incompatible| Carrier-specific group constraints are only allowed in isolation (one constraint in the group).

|changed| If `ensure_feasibility` is set to `True`, `unmet_demand` will always be returned in the model results, even if the model is feasible. Fixes issue 355.

|changed| Updated to Pyomo 6.0, pandas 1.2, xarray 0.17.

|changed| Update CBC Windows binary link in documentation.

|fixed| AttrDict now has a `__name__` attribute, which makes pytest happy.

|fixed| CLI plotting command has been re-enabled. Fixes issue 341.

|fixed| Group constraints are more robust to variations in user inputs. This entails a trade-off whereby some previously accepted user configurations will no longer be possible, since we want to avoid the complexity of processing them.

|fixed| `demand_share_per_timestep_decision` now functions as expected, where it previously did not enforce the per-timestep share after having decided upon it.

|fixed| Various bugs squashed in running operate mode.

|fixed| Handle number of timesteps lower than the horizon length in `operate` mode (337).

|new| `spores` run mode now available, to find Spatially-explicit Practically Optimal REsultS (SPORES)

|new| New group constraints `carrier_con_min`, `carrier_con_max`, `carrier_con_equals` which restrict the total consumed energy of a subgroup of conversion and/or demand technologies.

|new| Add ability to pass timeseries as dataframes in `calliope.Model` instead of only as CSV files.

|new| Pyomo backend interfaces added to get names of all model objects (`get_all_model_attrs`) and to attach custom constraints to the backend model (`add_constraint`).

|changed| Parameters are assigned a domain in Pyomo based on their dtype in `model_data`

|changed| Internal code reorganisation.

|changed| Updated to Pyomo 5.7, pandas 1.1, and xarray 0.16

|fixed| One-way transmission technologies can have `om` costs

|fixed| Silent override of nested dicts when parsing YAML strings

|new| New group constraints `energy_cap_equals`, `resource_area_equals`, and `energy_cap_share_equals` to add the equality constraint to existing `min/max` group constraints.

|new| New group constraints `carrier_prod_min`, `carrier_prod_max`, and `carrier_prod_equals` which restrict the absolute energy produced by a subgroup of technologies and locations.

|new| Introduced a `storage_discharge_depth` constraint, which allows to set a minimum stored-energy level to be preserved by a storage technology.

|new| New group constraints `net_import_share_min`, `net_import_share_max`, and `net_import_share_equals` which restrict the net imported energy of a certain carrier into subgroups of locations.

|changed| |backwards-incompatible| Group constraints with the prefix `supply_share` are renamed to use the prefix `carrier_prod_share`. This ensures consistent naming for all group constraints.

|changed| Allowed 'energy_cap_min' for transmission technologies.

|changed| Minor additions made to troubleshooting and development documentation.

|changed| |backwards-incompatible| The backend interface to update a parameter value (`Model.backend.update_param()`) has been updated to allow multiple values in a parameter to be updated at once, using a dictionary.

|changed| Allowed `om_con` cost for demand technologies. This is conceived to allow better representing generic international exports as demand sinks with a given revenue (e.g. the average electricity price on a given bidding zone), not restricted to any particular type of technology.

|changed| |backwards-incompatible| `model.backend.rerun()` returns a calliope Model object instead of an xarray Dataset, allowing a user to access calliope Model methods, such as `get_formatted_array`.

|changed| Carrier ratios can be loaded from file, to allow timeseries carrier ratios to be defined, e.g. ``carrier_ratios.carrier_out_2.heat: file=ratios.csv``.

|changed| Objective function options turned into Pyomo parameters. This allows them to update through the `Model.backend.update_param()` functionality.

|changed| All model defaults have been moved to `defaults.yaml`, removing the need for `model.yaml`. A default location, link and group constraint have been added to `defaults.yaml` to validate input model keys.

|changed| |backwards-incompatible| Revised internal logging and warning structure. Less critical warnings during model checks are now logged directly to the INFO log level, which is displayed by default in the CLI, and can be enabled when running in Python by calling `calliope.set_log_verbosity()` without any options. The `calliope.set_log_level` function has been renamed to `calliope.set_log_verbosity` and includes the ability to easily turn on and off the display of solver output.

|changed| All group constraint values are parameters so they can be updated in the backend model

|fixed| Operate mode checks cleaned up to warn less frequently and to not be so aggressive at editing a users model to fit the operate mode requirements.

|fixed| Documentation distinctly renders inline Python, YAML, and shell code snippets.

|fixed| Tech groups are used to filter technologies to which group constraints can be applied. This ensures that transmission and storage technologies are included in cost and energy capacity group constraints. More comprehensive tests have been added accordingly.

|fixed| Models saved to NetCDF now include the fully built internal YAML model and debug data so that `Model.save_commented_model_yaml()` is available after loading a NetCDF model from disk

|fixed| Fix an issue preventing the deprecated `charge_rate` constraint from working in 0.6.4.

|fixed| Fix an issue that prevented 0.6.4 from loading NetCDF models saved with older versions of Calliope. It is still recommended to only load models with the same version of Calliope that they were saved with, as not all functionality will work when mixing versions.

|fixed| |backwards-incompatible| Updated to require pandas 0.25, xarray 0.14, and scikit-learn 0.22, and verified Python 3.8 compatibility. Because of a bugfix in scikit-learn 0.22, models using k-means clustering with a specified random seed may return different clusters from Calliope 0.6.5 on.

|new| New model-wide constraint that can be applied to all, or a subset of, locations and technologies in a model, covering:

* `demand_share`, `supply_share`, `demand_share_per_timestep`, `supply_share_per_timestep`, each of which can specify `min`, `max`, and `equals`, as well as `energy_cap_share_min` and `energy_cap_share_max`. These supersede the `group_share` constraints, which are now deprecated and will be removed in v0.7.0.
* `demand_share_per_timestep_decision`, allowing the model to make decisions on the per-timestep shares of carrier demand met from different technologies.
* `cost_max`, `cost_min`, `cost_equals`, `cost_var_max`, `cost_var_min`, `cost_var_equals`, `cost_investment_max`, `cost_investment_min`, `cost_investment_equals`, which allow a user to constrain costs, including those not used in the objective.
* `energy_cap_min`, `energy_cap_max`, `resource_area_min`, `resource_area_max` which allow to constrain installed capacities of groups of technologies in specific locations.

|new| `asynchronous_prod_con` parameter added to the constraints, to allow a user to fix a storage or transmission technology to only be able to produce or consume energy in a given timestep. This ensures that unphysical dissipation of energy cannot occur in these technologies, by activating a binary variable (`prod_con_switch`) in the backend.

|new| Multi-objective optimisation problems can be defined by linear scalarisation of cost classes, using `run.objective_options.cost_class` (e.g. `{'monetary': 1, 'emissions': 0.1}`, which models an emissions price of 0.1 units of currency per unit of emissions)

|new| Storage capacity can be tied to energy capacity with a new `energy_cap_per_storage_cap_equals` constraint.

|new| The ratio of energy capacity and storage capacity can be constrained with a new `energy_cap_per_storage_cap_min` constraint.

|new| Easier way to save an LP file with a ``--save_lp`` command-line option and a ``Model.to_lp`` method

|new| Documentation has a new layout, better search, and is restructured with various content additions, such as a section on troubleshooting.

|new| Documentation for developers has been improved to include an overview of the internal package structure and a guide to contributing code via a pull request.

|changed| |backwards-incompatible| Scenarios in YAML files defined as list of override names, not comma-separated strings: `fusion_scenario: cold_fusion,high_cost` becomes `fusion_scenario: ['cold_fusion', 'high_cost']`. No change to the command-line interface.

|changed| `charge_rate` has been renamed to `energy_cap_per_storage_cap_max`. `charge_rate` will be removed in Calliope 0.7.0.

|changed| Default value of resource_area_max now is ``inf`` instead of ``0``, deactivating the constraint by default.

|changed| Constraint files are auto-loaded in the pyomo backend and applied in the order set by 'ORDER' variables given in each constraint file (such that those constraints which depend on pyomo expressions existing are built after the expressions are built).

|changed| Error on defining a technology in both directions of the same link.

|changed| Any inexistent locations and / or technologies defined in model-wide (group) constraints will be caught and filtered out, raising a warning of their existence in the process.

|changed| Error on required column not existing in CSV is more explicit.

|changed| |backwards-incompatible| Exit code for infeasible problems now is 1 (no success). This is a breaking change when relying on the exit code.

|changed| `get_formatted_array` improved in both speed and memory consumption.

|changed| `model` and `run` configurations are now available as attributes of the Model object, specifically as editable dictionaries which automatically update a YAML string in the `model_data` xarray dataset attribute list (i.e. the information is stored when sending to the solver backend and when saving to and loading from NetCDF file)

|changed| All tests and example models have been updated to solve with Coin-CBC, instead of GLPK. Documentation has been updated to reflect this, and aid in installing CBC (which is not simple for Windows users).

|changed| Additional and improved pre-processing checks and errors for common model mistakes.

|fixed| Total levelised cost of energy considers all costs, but energy generation only from ``supply``, ``supply_plus``, ``conversion``, and ``conversion_plus``.

|fixed| If a space is left between two locations in a link (i.e. `A, B` instead of `A,B`), the space is stripped, instead of leading to the expectation of a location existing with the name ` B`.

|fixed| Timeseries efficiencies can be included in operate mode without failing on preprocessing checks.

|fixed| Name of data variables is retained when accessed through `model.get_formatted_array()`

|fixed| Systemwide constraints work in models without transmission systems.

|fixed| Updated documentation on amendments of abstract base technology groups.

|fixed| Models without time series data fail gracefully.

|fixed| Unknown technology parameters are detected and the user is warned.

|fixed| Loc::techs with empty cost classes (i.e. value == None) are handled by a warning and cost class deletion, instead of messy failure.