Duqtools

This release changes how we deal with variables. Duqtools now automatically looks up and loads the dimensions for the variables, so they do not need to be specified anymore. Both the CLI tools for merging and plotting and the dashboard have been updated, so it should be easier to do these steps.

What's changed
* Update variable names to match variables.yaml lookup by stefsmeets in https://github.com/duqtools/duqtools/pull/413
* Add button to download charts to dashboard by stefsmeets in https://github.com/duqtools/duqtools/pull/417
* Add citation.cff by stefsmeets in https://github.com/duqtools/duqtools/pull/412
* Implement proof-of-concept for large scale validation by stefsmeets in https://github.com/duqtools/duqtools/pull/423
* Squash placeholder variables when retrieving the data by stefsmeets in https://github.com/duqtools/duqtools/pull/433
* `duqtools merge --all` by v1kko in https://github.com/duqtools/duqtools/pull/435
* Plot enhancements by v1kko in https://github.com/duqtools/duqtools/pull/437
* Enable multiple IDS in dashboard and simplify variable selection by stefsmeets in https://github.com/duqtools/duqtools/pull/443
* Revise variable selection in merge dashboard by stefsmeets in https://github.com/duqtools/duqtools/pull/445
* Update plotting interface to handle variables better by stefsmeets in https://github.com/duqtools/duqtools/pull/448
* Moved cfg.merge to the cli interface by v1kko in https://github.com/duqtools/duqtools/pull/444

Bug fixes
* Fix indenting error bug by v1kko in https://github.com/duqtools/duqtools/pull/411
* Skip time interpolation if time dimension has less than 2 values by v1kko in https://github.com/duqtools/duqtools/pull/434
* Fix for overwriting chart names by stefsmeets in https://github.com/duqtools/duqtools/pull/451

Documentation
* Fix docs build and add examples for jetto output config by stefsmeets in https://github.com/duqtools/duqtools/pull/427
* Docs: added dissemination section by v1kko in https://github.com/duqtools/duqtools/pull/430

Maintanance
* Removed queue_delete by v1kko in https://github.com/duqtools/duqtools/pull/415
* Remove unused dependencies and conditionally depend on/import backports by stefsmeets in https://github.com/duqtools/duqtools/pull/421
* Update flake8 repo in pre-commit config by stefsmeets in https://github.com/duqtools/duqtools/pull/422
* Decouple runs_dir from cwd by v1kko in https://github.com/duqtools/duqtools/pull/419
* Refactor common options parser to make it re-usable by stefsmeets in https://github.com/duqtools/duqtools/pull/425
* Fix Github CI by v1kko in https://github.com/duqtools/duqtools/pull/431
* Test coverage badge on gist by stefsmeets in https://github.com/duqtools/duqtools/pull/429
* Implement VarLookup class by stefsmeets in https://github.com/duqtools/duqtools/pull/441

**Full Changelog**: https://github.com/duqtools/duqtools/compare/1.1.1...1.2.0

Version 1.1.1 is a patch release that updates the documentation and fixes the output user for the merge command in the dashboard.

What's changed

* List variables by root IDS and sort alphabetically by stefsmeets in https://github.com/duqtools/duqtools/pull/403

Bug fixes

* Fix user for output imas target when merging data in the dashboard by stefsmeets in https://github.com/duqtools/duqtools/pull/400

Documentation

* Update contributing guidelines by stefsmeets in https://github.com/duqtools/duqtools/pull/387
* Fix broken links in documentation by stefsmeets in https://github.com/duqtools/duqtools/pull/393
* Define landing pages for subdirectories by stefsmeets in https://github.com/duqtools/duqtools/pull/394
* added asciinema movies to documentation by v1kko in https://github.com/duqtools/duqtools/pull/401

Maintanance

* Fixing pipelines by v1kko in https://github.com/duqtools/duqtools/pull/388
* Adding coverage by v1kko in https://github.com/duqtools/duqtools/pull/396
* Test all supported python versions by v1kko in https://github.com/duqtools/duqtools/pull/402
* Do not test 3.8 and 3.9 twice by v1kko in https://github.com/duqtools/duqtools/pull/404


**Full Changelog**: https://github.com/duqtools/duqtools/compare/1.1.0...1.1.1

What's Changed

* Let jetto.sin be an optional file by v1kko in https://github.com/duqtools/duqtools/pull/321
* Move variables out of `duqtools.yaml` to central config location by stefsmeets in https://github.com/duqtools/duqtools/pull/320
* Copy pre-defined config file when duqtools init is called by stefsmeets in https://github.com/duqtools/duqtools/pull/329
* Remove defaults that are too specific by stefsmeets in https://github.com/duqtools/duqtools/pull/330
* Tell the user why create fails by v1kko in https://github.com/duqtools/duqtools/pull/348
* Update `duqtools clean` to remove slurm array files by stefsmeets in https://github.com/duqtools/duqtools/pull/355
* Revise CLI plot interface to make use of variable lookup table by stefsmeets in https://github.com/duqtools/duqtools/pull/367
* Simplify getting data from ImasHandle and update notebooks by stefsmeets in https://github.com/duqtools/duqtools/pull/382
* Variable enhancements by stefsmeets in https://github.com/duqtools/duqtools/pull/346
* Features for Submit by v1kko in https://github.com/duqtools/duqtools/pull/363
* Performance improvements for status --detailed by v1kko in https://github.com/duqtools/duqtools/pull/364
* clean: be more lenient by v1kko in https://github.com/duqtools/duqtools/pull/365
* Add reference time slice to altair plots by stefsmeets in https://github.com/duqtools/duqtools/pull/368
* Add yolo command by stefsmeets in https://github.com/duqtools/duqtools/pull/370
* Add cli command to recreate run from `runs.yaml` by stefsmeets in https://github.com/duqtools/duqtools/pull/372
* Remove example dir and collect test data in its own directory by stefsmeets in https://github.com/duqtools/duqtools/pull/378
* Allow empty variables to be skipped when loading xarray data by stefsmeets in https://github.com/duqtools/duqtools/pull/384
* Add more variables to lookup table by stefsmeets in https://github.com/duqtools/duqtools/pull/385

Under the hood

* Require jetto_tools version by stefsmeets in https://github.com/duqtools/duqtools/pull/324
* Use jettopythontools whenever possible by v1kko in https://github.com/duqtools/duqtools/pull/322
* Unified jetto systems into jettosystem.py by v1kko in https://github.com/duqtools/duqtools/pull/335
* Performance improvements for status --detailed by v1kko in https://github.com/duqtools/duqtools/pull/364
* Add explicit queued versions of `ImasHandle.delete()` and `merge_data()` by stefsmeets in https://github.com/duqtools/duqtools/pull/386

Bug fixes

* Fix tests by stefsmeets in https://github.com/duqtools/duqtools/pull/334
* Patch jetto.jset during create to always write IDS files by stefsmeets in https://github.com/duqtools/duqtools/pull/371
* Fix dashboard by stefsmeets in https://github.com/duqtools/duqtools/pull/352
* Check for required flag before showing detailed status by stefsmeets in https://github.com/duqtools/duqtools/pull/369
* Fix the regressions in create and operations by v1kko in https://github.com/duqtools/duqtools/pull/361
* T e var fix by v1kko in https://github.com/duqtools/duqtools/pull/362
* Apply small code fixes by stefsmeets in https://github.com/duqtools/duqtools/pull/380

Documentation

* Fix docs build for `JettoSystem` by stefsmeets in https://github.com/duqtools/duqtools/pull/350
* Update documentation for variables and fix internal links by stefsmeets in https://github.com/duqtools/duqtools/pull/351
* Add example scripts to documentation by stefsmeets in https://github.com/duqtools/duqtools/pull/383
* Mention samplers and update variables section by stefsmeets in https://github.com/duqtools/duqtools/pull/328

House keeping

* Update sonar.yaml by v1kko in https://github.com/duqtools/duqtools/pull/306
* Disable codecov annotations in pull requests by v1kko in https://github.com/duqtools/duqtools/pull/327
* Use markdown-include to re-use files in docs by stefsmeets in https://github.com/duqtools/duqtools/pull/331
* Update links in code to new repo location by stefsmeets in https://github.com/duqtools/duqtools/pull/376
* Move scripts out to their own repo by stefsmeets in https://github.com/duqtools/duqtools/pull/375

**Full Changelog**: https://github.com/duqtools/duqtools/compare/1.0.0...1.1.0

The first major release of *duqtools*. Duqtools is a tool for Dynamic Undertainty Quantification for Tokamak reactor simulations modelling.

Features:

- Set up 100s of simulation runs from a single template
- Launch stardard sets of sensitivy tests with minimal programming
- Batch job submission and status tracking
- Supports the Standardized Interface Data Structures (IDSs) data directory
- Compare and visualize 100s of simulations in one overview
- Display simulation results as confidence ranges and distributions

What's Changed

* Add job scheduler by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/267
* Implement option for max jobs to submit by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/265
* Add cli switch to limit standard output by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/261
* Allowing for multiple `*` selectors in the variable path for xarray by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/268
* Create `jettopythontools` backend for duqtools by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/276
* Allow for inline specification of Jetto Lookup by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/288
* Major modification to analysis script compare_im_runs.py by aaronkho in https://github.com/CarbonCollective/fusion-dUQtools/pull/291
* Update scripts by Marinmepfl in https://github.com/CarbonCollective/fusion-dUQtools/pull/245

Under the hood

* Added JettoOperation(Dim) by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/283
* Coupled dimensions + t_start, t_end lookup tables by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/292
* Don't expand variables in config by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/290
* Added small imas test by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/295
* Standardize time function by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/298

Bug fixes

* Fixes crash in merge subcommand by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/272
* Filter pydantic-yaml explicit root warning by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/273
* Fix documentation build by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/280
* Deterministic parsing order for common options for cli by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/281
* bugfixes found by sonarcloud by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/296
* Fix plot by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/299

Documentation

* Add community guidelines by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/274
* Update documentation by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/302

House keeping

* Depend on `jetto-pythontools` from pypi by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/284
* Remove dead code by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/270
* Add sonarcloud badge by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/275
* Upload to codecov.io by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/277
* Run codecov on main by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/278
* Added codecov.yml by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/279
* Only run sonarcloud on PRs ready for review by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/304
* Condensed tests, removed superfluous test by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/294
* More specific sonarcloud flow by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/297

**Full Changelog**: https://github.com/CarbonCollective/fusion-dUQtools/compare/0.3.0...1.0.0

What's Changed
* Add manager to couple settings between `jetto.jset`, `jetto.in` by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/230
* Add `quiet` flag to operations by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/244
* Add implementation for xarray as in-memory data format by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/242
* Revise how IDS data paths are defined in config by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/251
* Refactor merge subcommand using xarray datasets by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/252
* Enable plot command to output to png, pdf by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/222
* Specify run in rebase functions by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/224
* Move jintrac settings config vars to data subdir by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/236
* Revise merge command config by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/234
* Generalize IDS support in create command step by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/238
* Enable multiple time slices in create command by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/241
* Get llcmd parameters from parameters of template system by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/233
* Set all log handlers to specified level by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/243
* Script develop by Marinmepfl in https://github.com/CarbonCollective/fusion-dUQtools/pull/213

Bug fixes

* Ids mapping fixes by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/223
* Status fix by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/231
* Fix member checking and improve error messages by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/248
* Fix crash in create and write correct build name to llcmd by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/258
* Fix dashboard by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/256

Documentation

* Add download link for jupyter notebooks in docs by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/237
* Update documentation for scaling to lower/upper error by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/259
* Update example notebooks by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/253
* Add notebook demoing 2D interpolation by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/255

House keeping

* Add hook to check notebook execution order by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/260

New Contributors
* Marinmepfl made their first contribution in https://github.com/CarbonCollective/fusion-dUQtools/pull/213

Known issues
* Plot command is broken https://github.com/CarbonCollective/fusion-dUQtools/issues/257

**Full Changelog**: https://github.com/CarbonCollective/fusion-dUQtools/compare/0.2.0...0.3.0

What's Changed

* Specify values as range and add error scaling by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/187
* Split template location by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/189
* Combine multiple IMAS db entries into single one by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/164
* major scripts update with bug corrections and new functionalities by aaronkho in https://github.com/CarbonCollective/fusion-dUQtools/pull/192
* Operations queue by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/191
* Better text output for operations by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/197
* Remove comments from yaml by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/200
* Enable setting values in `IDSMapping` by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/204
* Finalize merge feature in dashboard by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/206
* Convenience fixes by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/208
* Add run directory path to `jetto.jset` by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/181
* Add flag to add comments to config by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/182

Bug fixes

* Set error band extent to 'stdev' to fix notches by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/184

Documentation

* Update usage and installation instructions on docs by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/186
* Update, document, and demonstrate Python API by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/212
* Fixed documentation formatting issue by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/193
* Fix documentation build on readthedocs by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/203

Refactoring

* Refactor `ImasLocation` by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/165
* Refactor matrix samplers by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/167
* Refactor CLI schema by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/168
* Refactor `System` and `WorkDirectory` schemas by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/169
* Refactor ops sampler by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/180
* Refactor dry_run, improved tests by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/171
* System refactor by v1kko in https://github.com/CarbonCollective/fusion-dUQtools/pull/166
* Update dashboard to use refactored API by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/183
* Fix remaining refactoring issues by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/185
* Refactor dashboard and plotting tool by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/195

House-keeping

* Upload to pypi by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/190
* Increase maximum line length to 96 by stefsmeets in https://github.com/CarbonCollective/fusion-dUQtools/pull/199


**Full Changelog**: https://github.com/CarbonCollective/fusion-dUQtools/compare/0.1.0...0.2.0