Cumulusci

New features:

- `LoadData` now supports the key `action: update` to perform a Bulk
API update job
- `LoadData` now supports an `after: <step name>` on a lookup entry to
defer updating that lookup until a dependent sObject step is
completed.
- `GenerateMapping` now handles self-lookups and reference cycles by
generating `after:` markers wherever needed.

Issues closed:

- Patch selenium to convert `executeScript` to `executeAsyncScript`.
This is a workaround for the `executeScript` issue in chromedriver 77.
- A small issue in `QueryData` affecting mappings using
`oid_as_pk: False` has been fixed.

Breaking changes:

- The `retries`, `retry_interval`, and `retry_interval_add` options
have been removed from the `run_tests` task. These were misleading
as they did not actually retry failing tests.

New features:

- The `run_tests` task now supports a `retry_failures` parameter. This
is a list of regular expressions to match against each unit test
failure's message and stack trace; if all failing tests match, the
failing tests will be retried serially. Set `retry_always` to True
to trigger this behavior when any failure matches.
- There is now a default CumulusCI global connected app that can be
used to connect to persistent orgs (assuming you know the
credentials) without creating a new connected app. It's still
possible to configure a custom connected app using
`cci service connect connected_app` if more control over the
connected app settings is needed.
- When CumulusCI is being run in a non-interactive context it can now
obtain an access token for a persistent org using a JWT instead of a
refresh token. This approach is used if the SFDX_CLIENT_ID and
SFDX_HUB_KEY environment variables are set. This makes it easier to
manage persistent org connections in the context of a hosted service
because it's possible to replace the connected app's certificate
without needing to obtain new refresh tokens for each org.

Issues closed:

- Fixed a bug where showing the summary of flow steps would break with
sub-steps in MetaDeploy.
- Fixed a bug in the caching of preflight task results in MetaDeploy.

New features:

- We've changed how the output of some commands are displayed in
tables. For users that prefer simpler style tables we've added a
`--plain` option to approximate the previous behavior. To
permanently set this option, add this in
`~/.cumulusci/cumulusci.yml`:

cli:
plain_output: True

- Added additional info to the `cci version` command, including the
Python version, an upgrade check, and a warning on Python 2.

- Improved the summary of flow steps that is shown at the start of
running a flow.

- The `github_release_notes` task now has an `include_empty` option to
include links to pull requests that have no release notes. This is
enabled by default when this task is called as part of the
`release_beta` flow.

- Robot Framework:

- Added locators file to support the Winter '20 release of
Salesforce.
- New `robot_lint` task to check for common problems in Robot
Framework test suites.
- The `Open Test Browser` keyword will now log details about the
browser.
- Added a new keyword to the CumulusCI library,
`Get Community Info`. It can be used to get information about a
Community by name via the Salesforce API.

Issues closed:

- Added workarounds for some intermittent 401 errors when
authenticating to the GitHub API as a GitHub App.
- `cci org info` shouldn't show traceback if the org isn't found
(1023)

New features:

- Add the `cci org shell` command, which opens a Python shell
pre-populated with a simple_salesforce session on the selected org
(as `sf`).
- The `cci flow info` command now shows nested subflows.
- Added the `create_community` task allowing for API-based Community
creation.
- Added the `generate_dataset_mapping` task to generate a Bulk Data
mapping file for a package.
- CumulusCI can now authenticate for GitHub API calls as either a user
or an app. The latter is for use when CumulusCI is used as part of a
hosted service.
- The `OrgConfig` object now provides access to the Organization
SObject data via the `organization_sobject` attribute.

Issues closed:

- The `install_regression` flow now upgrades to the latest beta from
the most recent final release instead of from the previous final
release.
- Made sure that an `errorMessage` returned from a metadata API deploy
will be reported.
- The `load_dataset` task will now stop with an exception if any
records fail during the load operation.

- Updated the default API version for new projects to 46.0
- Fixed a bug in reporting scratch org creation errors encountered
while running a flow.
- Fixed the `snapshot_changes` and `list_changes` tasks to avoid
breaking when the last revision number of a component is null.

Breaking changes:

- Added two new options to the UpdateDependencies task:

- `allow_newer`: If the org already has a newer release, use it.
Defaults to True.
- `allow_uninstalls`: Allow uninstalling a beta release or newer
final release if needed in order to install the requested
version. Defaults to False.

These defaults are a change from prior behavior since uninstalling
packages is not commonly needed when working with scratch orgs, and
it is potentially destructive.

New features:

- Added support for defining and evaluating preflight checks for
MetaDeploy plans.
- The tasks for bulk data extract and load are now configured by
default as `extract_data` and `load_data`.
- Updated the project template created by `cci project init`:
- Added `.gitignore`, `README.md`, and a template for GitHub pull
requests
- Added an option to store metadata in DX source format
- Added a sample `mapping.yml` for the bulk data tasks
- Specify the currently installed CumulusCI version as the
project's `minimum_cumulusci_version`
- Check to make sure the project name only contains supported
characters
- The `robot_libdoc` task can now generate documentation for Robot
Framework page objects.

Issues fixed:

- Colors in terminal output are now displayed correctly in Windows.
(813)
- `cci` no longer prints tracebacks when a flow or task is not found.
Additionally, it will suggest a name if a close enough match can be
found. (960)
- Fixed UnicodeDecodeError when reading output from subprocesses if
the console encoding is different from Python's preferred file
encoding.
- Fixes related to source tracking:
- Track the max revision retrieved for each component instead of
the overall max revision. This way components can be retrieved
in stages into different paths.
- If `snapshot_changes` doesn't find any changes, wait 5 seconds
and try again. There can be a delay after a deployment before
source tracking is updated.