Cylc

Maintenance and minor enhancement release, plus new-format User Guide.

Selected user-facing changes:

Enhancements

[2910](https://github.com/cylc/cylc-flow/pull/2910) - replace LaTeX-generated HTML
and PDF User Guide with Sphinx-generated HTML.

[2815](https://github.com/cylc/cylc-flow/pull/2815) - allow initial cycle point
relative to current time.

[2902](https://github.com/cylc/cylc-flow/pull/2902) - expose suite UUID to event
handlers.

Fixes

[2932](https://github.com/cylc/cylc-flow/pull/2932) - fix possible blocking pipe
due to chatty job submission (and other subprocess) commands.

[2921](https://github.com/cylc/cylc-flow/pull/2921) - better suite validation
warning for out-of-bounds cycling sequences.

Minor release with over 120 issues closed. Significant issues include:

Enhancements

[2693](https://github.com/cylc/cylc-flow/pull/2693) - __auto host selection__; and
[2809](https://github.com/cylc/cylc-flow/pull/2809) - __auto migration__.
`cylc run` and `cylc restart` can now select the best host (based on several
metrics) on which to launch suite server programs. And running suites
can be told (via global config) to self-migrate to another available host, e.g.
for server maintenance. (The pool of suite hosts should see a shared
filesystem).

[2614](https://github.com/cylc/cylc-flow/pull/2614) and
[2821](https://github.com/cylc/cylc-flow/pull/2821) - __web-based job log viewer__ -
`cylc review` (migration of "Rose Bush" from the Rose project).


[2339](https://github.com/cylc/cylc-flow/pull/2339) - __general external
triggering__: tasks can trigger off of arbitrary user-defined Python functions
called periodically by the suite server program, with built-in functions
for suite-state (inter-suite) triggering and clock triggering (these
deprecate the existing suite-state polling tasks and clock-triggered tasks).

[2734](https://github.com/cylc/cylc-flow/pull/2734) - __EmPy templating__
support, as an alternative to Jinja2. _"EmPy allows embedding plain Python code
within templates and doesn't enforce any particular templating philosophy."_

[2733](https://github.com/cylc/cylc-flow/pull/2733) - enhanced Jinja2 support:
- __import pure Python modules__ in the same way as template modules
- Jinja2Tests and Jinja2Globals, for custom "is" tests and global variables
(c.f. our existing Jinja2Filters for custom filters).

[2682](https://github.com/cylc/cylc-flow/pull/2682) - new built-in
Jinja2 filter to convert ISO8601 date-time durations to
decimal seconds or hours.

[2842](https://github.com/cylc/cylc-flow/pull/2842) - `cylc gui` and
`cylc graph` - better integration with system desktop themes, including dark
themes; and other minor graph visualization improvements.

[2807](https://github.com/cylc/cylc-flow/pull/2807) - task output events (event
handlers can now be triggered when a task reports a registered output message).

[2868](https://github.com/cylc/cylc-flow/pull/2868) - a new task
runtime config item `exit-script`, for scripting to be executed at the
last moment after successful job completion. (Companion of `err-script`).

[2781](https://github.com/cylc/cylc-flow/pull/2781) and
[2854](https://github.com/cylc/cylc-flow/pull/2854) - improved suite
server program logging (including: `log/suite/err` is no longer used).

[2849](https://github.com/cylc/cylc-flow/pull/2849) - record local
background jobs by host name rather than "localhost".

[2877](https://github.com/cylc/cylc-flow/pull/2877) - new batch system
handler `pbs_multi_cluster`, supports PBS 13 clients fronting
heterogeneous clusters with different home directories from the
cylc remote. (Not needed for PBS 14+.) (For Rose suites this requires a
corresponding change to `rose suite-run`:
[metomi/rose2252](https://github.com/metomi/rose/pull/2252).)

[2812](https://github.com/cylc/cylc-flow/pull/2812) - `cylc gscan`:
show application menu bar by default.

[2768](https://github.com/cylc/cylc-flow/pull/2768) - `cylc gscan`: display Cylc
version of running suites.

[2786](https://github.com/cylc/cylc-flow/pull/2786) - make task try number
available to event handlers (as for task job submit number).

[2771](https://github.com/cylc/cylc-flow/pull/2771) - bash command completion:
complete suite names for commands that take a suite name argument (see
`etc/cylc-bash-completion`).

[2769](https://github.com/cylc/cylc-flow/pull/2769) - `cylc check-software` now
takes arguments to check for availability of specific modules.

[2763](https://github.com/cylc/cylc-flow/pull/2763) - `cylc monitor` - clean exit
on Ctrl-C.

[2704](https://github.com/cylc/cylc-flow/pull/2704) - paginated `cylc help` output.

[2660](https://github.com/cylc/cylc-flow/pull/2660) - new `gcylc.rc` config item to
show grouped cyclepoint subgraphs by default.

[2766](https://github.com/cylc/cylc-flow/pull/2766) - (development) formal test
coverage reporting and integration with GitHub.

[2751](https://github.com/cylc/cylc-flow/pull/2751) - (development) new contributor
guidelines - see `CONTRIBUTING.md`.

Fixes

[2876](https://github.com/cylc/cylc-flow/pull/2876) - avoid subprocess hang when
executing commands that generate a lot of stdout (such as when submitting
hundreds of jobs at once).

[2828](https://github.com/cylc/cylc-flow/pull/2828) - `suite.rc` - fail validation
on detecting trailing whitespace after a line continuation character.

[2807](https://github.com/cylc/cylc-flow/pull/2807) - handle multiple events of
the same type with the same message (e.g. warnings) from the same task job.

[2803](https://github.com/cylc/cylc-flow/pull/2803) - reset job submit number
correctly after aborting (backing out of) a trigger edit-run.

[2727](https://github.com/cylc/cylc-flow/pull/2727) - `cylc gui`: fix dropdown list
of log numbers for re-inserted tasks or after suite
restart.

[2759](https://github.com/cylc/cylc-flow/pull/2759) and
[2816](https://github.com/cylc/cylc-flow/pull/2816) -
suite registration tweaks and fixes.

[2861](https://github.com/cylc/cylc-flow/pull/2861) - improved emacs
syntax highlighting.

[2892](https://github.com/cylc/cylc-flow/pull/2892) - print the bad host name along
with "Name or service not known" exceptions.


-------------------------------------------------------------------------------

Maintenance release.

(Some minor changes not relevant to normal users may be omitted.)

Fixes and minor enhancements

[2719](https://github.com/cylc/cylc-flow/pull/2719) - improved job poll logging

[2724](https://github.com/cylc/cylc-flow/pull/2724) - fix a rare error associated
with ithe use of final cycle point in multiple recurrence expressions

[2723](https://github.com/cylc/cylc-flow/pull/2723) - fix remote commands (executed
by remote task jobs) running in UTC mode when the suite is not running in UTC
mode

[2726](https://github.com/cylc/cylc-flow/pull/2726) - fix crash in suites with no
final cycle point that are reloaded following a restart

[2716](https://github.com/cylc/cylc-flow/pull/2716) - ensure that job polling
interval lists are not overridden

[2714](https://github.com/cylc/cylc-flow/pull/2714) - block irrelevant
`InsecureRequestWarning`s from urllib3 on anonymous suite server access by
`cylc scan` and `cylc ping`

[2715](https://github.com/cylc/cylc-flow/pull/2715) - fix a cross-version
incompatibility, if a cylc-7.6.x task job messages a cylc-7.7.1 suite

[2710](https://github.com/cylc/cylc-flow/pull/2710) - fix a GUI error on
right-clicking a "scissor node" in the graph view

-------------------------------------------------------------------------------

Minor maintenance release.

Fixes

(Several minor fixes have been omitted from this list.)

[2678](https://github.com/cylc/cylc-flow/pull/2678) - fix loading of job poll
timers on restart (bug introduced at last release)

[2683](https://github.com/cylc/cylc-flow/pull/2683) - fix potential error in
`cylc check-software` (which checks for installed software dependencies)

[2691](https://github.com/cylc/cylc-flow/pull/2691) PBS support - handle job poll
result correctly if qstat temporarily fails to connect to the server

[2703](https://github.com/cylc/cylc-flow/pull/2703) - fix an error
(inconsequential) that appears in the suite log at restart: `ValueError: No
JSON object could be decoded`

[2692](https://github.com/cylc/cylc-flow/pull/2692) - fix X11 forwarding for GUI
edit job log, with `cylc gui --host=HOST`

[2690](https://github.com/cylc/cylc-flow/pull/2690) - invoking Cylc command help
should not require `$DISPLAY` to be set

[2677](https://github.com/cylc/cylc-flow/pull/2677) - use random serial numbers in
the self-signed SSL certificates generated by suite server programs

[2688](https://github.com/cylc/cylc-flow/pull/2688)
[2705](https://github.com/cylc/cylc-flow/pull/2705) - block several security
warnings emitted by `urllib3` under old Python versions (2.6). *We are
aware of the security issues, but these warnings serve no purpose on affected
platforms except to confuse and annoy users.*

[2676](https://github.com/cylc/cylc-flow/pull/2676) - use `!/usr/bin/env python2`
(i.e. Python-2 specific) in Cylc source files, to avoid issues with default
Python 3 installations (note Cylc is going to Python 3 next year)

[2679](https://github.com/cylc/cylc-flow/pull/2679) - change bold font back to
normal in the GUI log viewer

-------------------------------------------------------------------------------

Enhancements

[2661](https://github.com/cylc/cylc-flow/pull/2661) -
* new User Guide section on Remote Job Management
* tidy the installation documentation
* standardise directory structures (all docs updated accordingly):
- deprecated `<cylc-dir>conf/` file locations:
- site config file `<cylc-dir>etc/global.rc`
- gcylc config example `<cylc-dir>/etc/gcylc.rc.eg`
- site job environment init `<cylc-dir/etc/job-init-env.sh`
- editor syntax files in `<cylc-dir>etc/syntax/`
- bash completion script `<cylc-dir>/etc/cylc-bash-completion`
- user `global.rc` can now go in `~/.cylc/<cylc-version>/` or `~/.cylc/`
(the version-specific location avoid forward compatibility problems - see
notes in `<cylc-dir>/etc/global.rc.eg`).
- moved central cylc wrapper template to `usr/bin/cylc`
- various developer scripts and notes moved from `<cylc-dir>/dev/` to
`<cylc-dir>/etc/dev-bin, dev-notes, dev-suites`.
- *the ancient `site.rc` and `user.rc` global config filename variants are
now obsolete.*

[2659](https://github.com/cylc/cylc-flow/pull/2659) - commands in the process pool
(event handlers, and job submit, poll and kill commands) will now be killed on
a configurable timeout if they hang, rather than tying up a member of the
finite process pool: `global.rc` default: `process pool timeout = PT10M`

[2582](https://github.com/cylc/cylc-flow/pull/2582) - improve client/server
interface, including: `cylc message` can send multiple messages at once, with
different severities; server ignores messages from superseded job submits;
running jobs detect that the suite has been cold-started under them and will
not attempt to connect; ssh-based indirect client-server communication now
works automatically for all clients, not just messaging.

[2582](https://github.com/cylc/cylc-flow/pull/2582),
[2624](https://github.com/cylc/cylc-flow/pull/2624), and earlier changes: all job
host actions are now done by remote `cylc` subcommands that are compatible with
ssh whitelisting.

[2590](https://github.com/cylc/cylc-flow/pull/2590) - replaced the fixed process
pool with direct management of individual sub-processes (for external commands
executed by the server program).

[2561](https://github.com/cylc/cylc-flow/pull/2561) - pass resolved triggering
dependencies to `$CYLC_TASK_DEPENDENCIES` in job environments.

[2639](https://github.com/cylc/cylc-flow/pull/2639) - date-time cycling: document
availability of 365-day (never a leap year) and 366-day (always a leap year)
calendars.

[2597](https://github.com/cylc/cylc-flow/pull/2597) - emit a "late" event if a task
has not triggered by a user-defined real-time offset relative to cycle point.

[2648](https://github.com/cylc/cylc-flow/pull/2648) - improve version reporting.
Note than non-standard lowercase `cylc -v` is now gone; use `cylc -V` or `cylc
--version`, with optional `--long` format to print as well as version.

[2620](https://github.com/cylc/cylc-flow/pull/2620) - re-document the long-dated
`[special tasks]sequential` and recommend using explicit dependencies in the
graph instead.

[2584](https://github.com/cylc/cylc-flow/pull/2584) - internal queues now release
task jobs on a FIFO (First In, First Out) basis, rather than randomly.

[2538](https://github.com/cylc/cylc-flow/pull/2538) - remove leading whitespace
from multi-line `script` items in task definitions, for cleaner job scripts.

[2503](https://github.com/cylc/cylc-flow/pull/2503),
[2624](https://github.com/cylc/cylc-flow/pull/2624) - `cylc cat-log`: all remote
host actions now done by a `cylc` sub-command; and simpler command options (see
`cylc cat-log --help`; **warning: the old command options are not supported**)

Fixes

[2666](https://github.com/cylc/cylc-flow/pull/2666) - `cylc scan`: make default
behavior consistent with `cylc gscan`: get suite information from `~/cylc-run/`
only for the current user; and no partial matches with `-n/--name=PATTERN`
(i.e. `--name=bar` will only match the suite `bar`, not `foobar` or `barbaz`).

[2593](https://github.com/cylc/cylc-flow/pull/2593) - fix polling after job
execution timeout (configured pre-poll delays were being ignored).

[2656](https://github.com/cylc/cylc-flow/pull/2656) - fix suicide triggers with
multiple prerequisites in the same graph line.

[2638](https://github.com/cylc/cylc-flow/pull/2618) - fix duplicate "failed" task
events after `cylc stop --kill`.

[2653](https://github.com/cylc/cylc-flow/pull/2653) - tidy and correct the main `cylc
help` documentation.

[2644](https://github.com/cylc/cylc-flow/pull/2644),
[2646](https://github.com/cylc/cylc-flow/pull/2646) - fix some graph parsing edge
cases.

[2649](https://github.com/cylc/cylc-flow/pull/2649) - respect suite
UTC mode when recording job submit time (database and GUI tree view).

[2631](https://github.com/cylc/cylc-flow/pull/2631) - fix "failed" task event after
bad host select.

[2626](https://github.com/cylc/cylc-flow/pull/2626) - `cylc gui`: fix error when
a task job completes just before "view task prerequisites" (menu) is actioned.

[2600](https://github.com/cylc/cylc-flow/pull/2600) - correct task prerequisite
manipulation on state changes.

[2596](https://github.com/cylc/cylc-flow/pull/2596) - fix reference to parameter
values containing `+` or `-` characters.

[2592](https://github.com/cylc/cylc-flow/pull/2592) - permit syntax errors in edit
runs, so that the job file still gets written.

[2579](https://github.com/cylc/cylc-flow/pull/2579) - `cylc gscan`: fix a
re-spawning error dialog.

[2674](https://github.com/cylc/cylc-flow/pull/2674) - `cylc cat-log`: avoid leaving
orphaned tail-follow processes on job hosts.


-------------------------------------------------------------------------------

A collection of bug fixes made since cylc-7.6.0.

Fixes

[2571](https://github.com/cylc/cylc-flow/pull/2571) - `cylc gui`: fix tailing of
remote job logs from the GUI (in 7.6.0 this failed with a Python traceback).

[2596](https://github.com/cylc/cylc-flow/pull/2596) - allow parameter values that
contain +/- characters

[2574](https://github.com/cylc/cylc-flow/pull/2574) - `cylc gscan HOST`: show just
the owner's suites by default (at 7.6.0 this showed all suites on HOST).

[2592](https://github.com/cylc/cylc-flow/pull/2592) - `cylc trigger --edit`:
disable job script syntax checking in edit runs (this prevented a new job
script from being written for editing).

[2579](https://github.com/cylc/cylc-flow/pull/2579) - `cylc gscan`: fix respawning
error dialog on giving a bad regular expression (on the command line) to match
owner or suite name.

[2606](https://github.com/cylc/cylc-flow/pull/2606) - jobs with batch system info
missing in a corrupted status file (thus not pollable) will now poll as failed
rather appear stuck as running.

[2603](https://github.com/cylc/cylc-flow/pull/2603) - `cylc gui` (graph view): fix
possible error on inserting a nested family.

[2602](https://github.com/cylc/cylc-flow/pull/2602) - `cylc gui` (tree view): fix
negative progress bar value (reportedly possible after manual task state
manipulations).

[2586](https://github.com/cylc/cylc-flow/pull/2586) - `cylc gui` (tree view): fix
possible division-by-zero error in elapsed time computation.

[2588](https://github.com/cylc/cylc-flow/pull/2588) - `cylc trigger --edit`: fix
edit runs for tasks with dynamic remote host selection.

[2585](https://github.com/cylc/cylc-flow/pull/2585) - fix recovery from a failed
host select command.

-------------------------------------------------------------------------------