Cylc

Enhancements

[2373](https://github.com/cylc/cylc-flow/pull/2373) - refactored suite server code
(for efficiency, maintainability, etc.)

[2396](https://github.com/cylc/cylc-flow/pull/2396) - improved job polling and task
state reset:
* allow polling of finished tasks - to confirm they really succeeded or failed
* poll to confirm a task message that implies a state reversal - it could just
be a delayed message
* allow reset to submitted or running states
* removed the "enable resurrection" setting - any task can now return from the
dead

[2410](https://github.com/cylc/cylc-flow/pull/2410) - new CUSTOM severity task
messages that can trigger a custom event handler

[2420](https://github.com/cylc/cylc-flow/pull/2420) - `cylc monitor` now
reconnects automatically if its target suite gets restarted on a different port

[2430](https://github.com/cylc/cylc-flow/pull/2430) - `cylc gscan`,
`cylc gui` - significant reduction in impact on suite server
programs

[2433](https://github.com/cylc/cylc-flow/pull/2433) - "group" (used to group suites
in `cylc gscan`) is now defined under the suite "[[meta]]" section

[2424](https://github.com/cylc/cylc-flow/pull/2424) - task job scripts now run in
`bash -l` (login shell) instead of explicitly sourcing your
`.profile` file. *WARNING*: if you have a
`.bash_profile` and were using `.profile` as well just for
Cylc, the latter file will now be ignored because bash gives precendence to the
former. If so, just move your Cylc settings into
`.bash_profile` or consult the Cylc User Guide for
other ways to configure the task job environment.

[2441](https://github.com/cylc/cylc-flow/pull/2441) -
[2458](https://github.com/cylc/cylc-flow/pull/2458) - allow more event handler
arguments:
* batch system name and job ID
* submit time, start time, finish time
* userhost

[2455](https://github.com/cylc/cylc-flow/pull/2455) - network client improvements:
* on a failed connection, clients detect if the suite has stopped according to
the contact file, then report it stopped and remove the contact file
* on attempt run an already-running suite (contact file exists) print more
information on how old the suite is and how to shut it down
* clients running in plain HTTP protocol will no longer attempt to fetch a
non-existent SSL certificate
* if a contact file is loaded, always use the values in it to avoid
conflicting host strings in SSL certificate file, etc.

[2468](https://github.com/cylc/cylc-flow/pull/2468) - initialize task remotes
asynchronously via the multiprocessing pool, to avoid holding up suite start-up
unnecessarily. *WARNING* this introduces new remote commands: `cylc
remote-init` and `cylc remote-tidy` that will affect sites
using ssh whitelisting

[2449](https://github.com/cylc/cylc-flow/pull/2449) -
[2469](https://github.com/cylc/cylc-flow/pull/2469) -
[2480](https://github.com/cylc/cylc-flow/pull/2480) -
[2501](https://github.com/cylc/cylc-flow/pull/2501) -
[2547](https://github.com/cylc/cylc-flow/pull/2547) -
[2552](https://github.com/cylc/cylc-flow/pull/2552) -
[2564](https://github.com/cylc/cylc-flow/pull/2564) -
User Guide:
* rewrote the section on restart from state checkpoints
* rewrote the section on suite run databases
* new section on suite contact files
* new section on disaster recovery
* new section on remote monitoring and control
* improved terminology:
* "suite server program" instead of "suite daemon" (it's not always a daemon)
* "severity" instead of "priority" for logging and task messaging
* "task remote" to encompass the concept of "the account where a task job
runs" whether under the same user account or not, on another host or not
* documented requirements for remote access to suite-parsing and
file-retrieval commands, including via the GUI; and clarified the same for
suite client commands
* documented a known bug in use of parameters in complex graph syntax (and an
easy workaround) - see the task parameters section
* documented kill-to-hold behavior of tasks with retries configured

[2475](https://github.com/cylc/cylc-flow/pull/2475) - suite server program:
separate debug mode from daemonization

[2485](https://github.com/cylc/cylc-flow/pull/2485) - export task job environment
variables on definition and before assignment, to ensure they are available to
subshells immediately - even in expressions inside subsequent variable
definitions

[2489](https://github.com/cylc/cylc-flow/pull/2489) -
[2557](https://github.com/cylc/cylc-flow/pull/2557) -
`cylc gscan` -
* configurable menubar visibility at start-up
* grouped suites now retain their grouped status once stopped

[2515](https://github.com/cylc/cylc-flow/pull/2515) -
[2529](https://github.com/cylc/cylc-flow/pull/2529) -
[2517](https://github.com/cylc/cylc-flow/pull/2517) -
[2560](https://github.com/cylc/cylc-flow/pull/2560) -
`cylc gui`
* put prompt dialogss above all windows
* load new-suite log files after switching to another suite via the File menu
* graph view: reinstate the right-click menu for ghost nodes (lost at cylc-7.5.0)
* job log files:
* fix and document the "extra log files" setting
* add "view in editor" support for extra log files
* add text-editor functionality to `cylc jobscript`
* add "preview jobscript" functionality to the GUI

[2527](https://github.com/cylc/cylc-flow/pull/2527) -
[2431](https://github.com/cylc/cylc-flow/pull/2431) -
[2435](https://github.com/cylc/cylc-flow/pull/2435) -
[2445](https://github.com/cylc/cylc-flow/pull/2445) -
[2491](https://github.com/cylc/cylc-flow/pull/2491) -
[2484](https://github.com/cylc/cylc-flow/pull/2484) -
[2556](https://github.com/cylc/cylc-flow/pull/2556) -
improved parameter support:
* allow "%d" integer format in parameter templates
* allow out of range parameter on graph RHS
* allow positive offset for parameter index on graph
* allow negative integer parameters
* allow custom templating of parameter environment variables, in addition to
the built-in `CYLC_TASK_PARAM\_<param-name>`
* allow bare parameter values as task names
* allow explicit parameter values in "inherit" items under "[runtime]"
* fix parameters inside (as opposed to beginning or end) of family names
* fixed inheritance from multiple parameterized namespaces at once

[2553](https://github.com/cylc/cylc-flow/pull/2553) - upgraded the bundled Jinja2
version to 2.10. This fixes the block scope problem introduced in the previous
version

[2558](https://github.com/cylc/cylc-flow/pull/2558) - new options to print out JSON
format from `cylc show` and `cylc scan`

Fixes

[2381](https://github.com/cylc/cylc-flow/pull/2381) - validation: fail bad event
handler argument templates

[2416](https://github.com/cylc/cylc-flow/pull/2416) - validation: print the problem
namespace in case of bad multiple inheritance

[2426](https://github.com/cylc/cylc-flow/pull/2426) - validation: fail
non-predefined config item names (e.g. batch scheduler directives) that contain
multiple consecutive spaces (to ensure that hard-to-spot whitespace typos don't
prevent repeated items from overriding as intended)

[2432](https://github.com/cylc/cylc-flow/pull/2432) - fixed an issue that could
cause HTTPS client failure due to SSL certificate host name mismatch

[2434](https://github.com/cylc/cylc-flow/pull/2434) - correctly strip "at TIME"
from the end of multi-line task messages

[2440](https://github.com/cylc/cylc-flow/pull/2440) - `cylc suite-state`
- fixed DB query of tasks with custom outputs that have not been generated yet

[2444](https://github.com/cylc/cylc-flow/pull/2444) - added `cylc
report-timings` to main command help

[2449](https://github.com/cylc/cylc-flow/pull/2449):
* server suite and task URLs from suite server programs, rather than parsing
them from the suite definition - so browsing URLs from a remote GUI now
works
* allow proper string templating of suite and task names in URLs; retained the
old pseudo environment variables for backward compatibility

[2461](https://github.com/cylc/cylc-flow/pull/2461) - fixed manual task retrigger
after an aborted edit run - this was erroneously using the edited job file

[2462](https://github.com/cylc/cylc-flow/pull/2462) - fixed job polling for the SGE
batch scheduler

[2464](https://github.com/cylc/cylc-flow/pull/2464) - fixed the ssh+HTTPS task

Enhancements

[2387](https://github.com/cylc/cylc-flow/pull/2387),
[2330](https://github.com/cylc/cylc-flow/pull/2330): New suite.rc `[meta]` sections
for suite and task metadata. These hold the existing `title`, `description`,
and `URL` items, plus arbitrary user-defined items. Metadata items can be passed
to event handlers (e.g. a site-specific task "priority" or "importance" rating
could inform an event-handler's decision on whether or not to escalate task
failures).

[2298](https://github.com/cylc/cylc-flow/pull/2298),
[2401](https://github.com/cylc/cylc-flow/pull/2401): New shell function
`cylc__job_abort <message>` to abort task job scripts with a custom message
that can be passed to task failed event handlers.

[2204](https://github.com/cylc/cylc-flow/pull/2204): Remove auto-fallback to HTTP
communications, if HTTPS is not available. Now HTTP is only used if explicitly
configured.

[2332](https://github.com/cylc/cylc-flow/pull/2332),
[2325](https://github.com/cylc/cylc-flow/pull/2325),
[2321](https://github.com/cylc/cylc-flow/pull/2321),
[2312](https://github.com/cylc/cylc-flow/pull/2312): Validation efficiency
improvements.

[2291](https://github.com/cylc/cylc-flow/pull/2291),
[2303](https://github.com/cylc/cylc-flow/pull/2303),
[2322](https://github.com/cylc/cylc-flow/pull/2322): Runtime efficiency
improvements.

[2286](https://github.com/cylc/cylc-flow/pull/2286): New command `cylc
report-timings` to generate reports of task runtime statistics.

[2304](https://github.com/cylc/cylc-flow/pull/2304): New event handlers for general
CRITICAL events.

[2244](https://github.com/cylc/cylc-flow/pull/2244),
[2258](https://github.com/cylc/cylc-flow/pull/2258): Advanced syntax for excluding
multiple points from cycling sequences.

[2407](https://github.com/cylc/cylc-flow/pull/2407): Documented exactly how Cylc
uses ssh, scp, and rsync to interact with remote job hosts.

[2346](https://github.com/cylc/cylc-flow/pull/2346),
[2386](https://github.com/cylc/cylc-flow/pull/2386): `cylc graph` now plots
implicit dependences as grayed-out ghost nodes.

[2343](https://github.com/cylc/cylc-flow/pull/2343): Improved the "Running
Suites" section of the User Guide, including documentation of suite remote
control.

[2344](https://github.com/cylc/cylc-flow/pull/2344): Attempt to access suite
service files via the filesystem first, before ssh, for other accounts on the
suite host.

[2360](https://github.com/cylc/cylc-flow/pull/2360): Better validation of suite
parameter configuration.

[2314](https://github.com/cylc/cylc-flow/pull/2314): In debug mode, send bash job
script xtrace output (from `set -x`) to a separate log file.

Fixes

[2409](https://github.com/cylc/cylc-flow/pull/2409): Fixed the `cylc spawn` command
(it was killing tasks, since cylc-7).

[2378](https://github.com/cylc/cylc-flow/pull/2378): Fixed use of negative offsets
by the `cylc suite-state` command.

[2364](https://github.com/cylc/cylc-flow/pull/2364): Correctly load completed custom
task outputs on restart.

[2350](https://github.com/cylc/cylc-flow/pull/2350): Handle bad event handler
command line templates gracefully.

[2308](https://github.com/cylc/cylc-flow/pull/2308): The parameterized task
environment variable `$CYLC_TASK_PARAM_<param>` is now guaranteed to be defined
before any use of it in the user-defined task environment section.

[2296](https://github.com/cylc/cylc-flow/pull/2296): Prevent suites stalling after
a restart that closely follows a warm-start (now the restart, like the warm
start, ignores dependence on tasks from before the warm start point).

[2295](https://github.com/cylc/cylc-flow/pull/2295): Fixed `cylc cat-log` "open in
editor" functionality for remote job logs.

[2412](https://github.com/cylc/cylc-flow/pull/2412): Fixed duplication of log
messages to the old log after restart.

-------------------------------------------------------------------------------

Enhancements and fixes.

Highlighted Changes

[2260](https://github.com/cylc/cylc-flow/pull/2260): Open job logs in your text
editor, from CLI (`cylc cat-log`) or GUI.

[2259](https://github.com/cylc/cylc-flow/pull/2259): `cylc gscan` - various
improvements: right-click menu is now for suite operations only; other items
moved to a main menubar and toolbar (which can be hidden to retain gscan's
popular minimalist look); added all suite stop options (was just the default
clean stop); task-state colour-key popup updates in-place if theme changed; new
collapse/expand-all toobar buttons.

[2275](https://github.com/cylc/cylc-flow/pull/2275): Pass suite and task URLs to
event handlers.

[2272](https://github.com/cylc/cylc-flow/pull/2272): Efficiency - reduce memory
footprint.

[2157](https://github.com/cylc/cylc-flow/pull/2157):
* internal efficiency improvements
* allow reset of individual message outputs
* "cylc submit" can now submit families

[2244](https://github.com/cylc/cylc-flow/pull/2244): Graph cycling configuration:
multiple exclusion points.

[2240](https://github.com/cylc/cylc-flow/pull/2240): Stepped integer parameters.

Fixes

[2269](https://github.com/cylc/cylc-flow/pull/2269): Fix auto suite-polling tasks
(i.e. inter-suite dependence graph syntax) - Broken in 7.3.0.

[2282](https://github.com/cylc/cylc-flow/pull/2282): Fix global config processing
of boolean settings - users could not override a site True setting to False.

[2279](https://github.com/cylc/cylc-flow/pull/2279): Bundle Jinja2 2.9.6. (up from
2.8) - fixes a known issue with Jinja2 "import with context".

[2255](https://github.com/cylc/cylc-flow/pull/2255): Fix handling of suite script
items that contain nothing but comments.

[2247](https://github.com/cylc/cylc-flow/pull/2247): Allow `cylc graph --help`
in the absence of an X environment.

Other Changes

[2270](https://github.com/cylc/cylc-flow/pull/2270): Detect and fail null tasks in
graph.

[2257](https://github.com/cylc/cylc-flow/pull/2257): `cylc gscan` - graceful exit
via Ctrl-C.

[2252](https://github.com/cylc/cylc-flow/pull/2252): `ssh`: add `-Y` (X Forwarding)
only if necessary.

[2245](https://github.com/cylc/cylc-flow/pull/2245): SSL certficate: add serial
number (issue number). This allows curl, browsers, etc. to connect to
suite daemons.

[2265](https://github.com/cylc/cylc-flow/pull/2265): `cylc gpanel` - restored
sorting of items by suite name.

[2250](https://github.com/cylc/cylc-flow/issues/2250): Updated installation docs
for HTTPS-related requirements.

-------------------------------------------------------------------------------

New Suite Design Guide, plus other enhancements and fixes.

Highlighted Changes

[2211](https://github.com/cylc/cylc-flow/pull/2211): New comprehensive Suite Design
Guide document to replace the outdated Suite Design section in the User Guide.

[2232](https://github.com/cylc/cylc-flow/pull/2232): `cylc gscan` GUI: stop, hold,
and release suites or groups of suites.

[2220](https://github.com/cylc/cylc-flow/pull/2220): dummy and simulation mode improvements:
* new `dummy-local` mode runs dummy tasks as local background jobs (allows
dummy running other-site suites).
* proportional run length, if tasks configure an `execution time limit`
* single common `[simulation]` configuration section for dummy, dummy-local, and
simulation modes.
* dummy or simulated tasks can be made to fail at specific cycle points, and
for first-try only, or all tries.
* custom message outputs now work in simulation mode as well as the dummy modes.

[2218](https://github.com/cylc/cylc-flow/pull/2218): fix error trapping in job
scripts (degraded since job file refactoring in 7.1.1)

[2215](https://github.com/cylc/cylc-flow/pull/2215): SGE batch system support -
fixed formatting of directives with a space in the name.

Other Notable Changes

[2233](https://github.com/cylc/cylc-flow/pull/2233): Upgraded the built-in example
suites to cylc-7 syntax.

[2221](https://github.com/cylc/cylc-flow/pull/2221): `cylc gui` GUI dot view - maintain
user selection during update.

[2217](https://github.com/cylc/cylc-flow/pull/2217): `cylc gscan` GUI - fix
tracebacks emitted during suite initialization.

[2219](https://github.com/cylc/cylc-flow/pull/2219): add `userhost` option to
`cylc monitor` an `cylc gui`. Allows suite selection at startup using `cylc
scan` output.

[2222](https://github.com/cylc/cylc-flow/pull/2222): `cylc gui` GUI graph view -
fixed right-click "view prerequisites" sub-menu.

[2213](https://github.com/cylc/cylc-flow/pull/2213): Record family inheritance
structure in the run database.

-------------------------------------------------------------------------------

Minor enhancements and fixes.

Highlighted Changes

[2209](https://github.com/cylc/cylc-flow/pull/2209): Fixed the `cylc gui` graph
view, broken at cylc-7.2.0.

[2193](https://github.com/cylc/cylc-flow/pull/2193): Restored `cylc gscan`
suite-stopped status checkerboard icons, lost at cylc-7.1.1.


[2208](https://github.com/cylc/cylc-flow/pull/2208): Use suite host name instead
of suite name in the SSL certificate "common name".

[2206](https://github.com/cylc/cylc-flow/pull/2206): Updated User Guide
installation section.

Other Notable Changes

[2191](https://github.com/cylc/cylc-flow/pull/2191): Clearer task prerequisites
print-out.

[2197](https://github.com/cylc/cylc-flow/pull/2197): Removed the bundled external
OrderedDict package.

[2194](https://github.com/cylc/cylc-flow/pull/2194): `cylc gscan` - better handling
of suites that are still initializing.

-------------------------------------------------------------------------------

Minor enhancements and fixes (note mid-level version number bumped up to
reflect significant changes included in 7.1.1 - esp. job file refactoring).

Highlighted Changes

[2189](https://github.com/cylc/cylc-flow/pull/2184): New `assert` and
`raise` functions for handling Jinja2 errors in suites.

Other Changes

[2186](https://github.com/cylc/cylc-flow/pull/2186): Use lowercase local shell
variable names in new job script shell functions introduced in 7.1.1, to avoid
overriding shell built-ins such as `$HOSTNAME`.

[2187](https://github.com/cylc/cylc-flow/pull/2187): Fixed a bug causing restart
failure in the presence of an active broadcast of a submission timeout value.

[2183](https://github.com/cylc/cylc-flow/pull/2183): Use site-configured suite host
self-identification, if present, as hostname in the SSL certificate.

[2182](https://github.com/cylc/cylc-flow/pull/2182): Fixed failed User Guide build
in 7.1.1.

-------------------------------------------------------------------------------