Cumulusci

Changes:

- Updated the default Salesforce Metadata API version to 45.0
- The scratch org definition files generated by `cci project init` now
use `orgPreferenceSettings` instead of the deprecated
`orgPreferences`.
- The `metadeploy_publish` task now defaults to describing tasks based
on `Deploy` as "metadata" steps instead of "other".

Issues Fixed:

- Fixed a couple problems with generating passwords for new scratch
orgs:
- A project's predefined scratch org configs now default to
`set_password: True` (which was already the case for orgs
created explicitly using cci org scratch).
- A scratch org config's `set_password` flag is now retained when
recreating an expired org. (Fixes 670)
- Fixed the logic for finding the most recent GitHub release so that
it now only considers tags that start with the project's git
`prefix_release`.
- Fixed the `install_prod_no_config` flow. The `deploy_post` task was
not injecting namespace tokens correctly.
- Fixed the `connected_app` task to work with version 7 of the
sfdx CLI. (Fixes 1013)
- Robot Framework: Fixed the `Populate Field` keyword to work around
intermittent problems clearing existing field values.

Critical changes:

- If you are publishing installation plans to MetaDeploy, there have
been some significant changes:

> - Plan options are now read from a new `plans` section of
> `cumulusci.yml` instead of from task options. This means that
> a single run of the task can now handle publishing multiple
> plans, and there is now a generic `metadeploy_publish` task
> which can be used instead of setting up different tasks for
> each project.
> - Plan steps are now defined inline in the plan configuration
> rather than by naming a flow. This makes it easier to
> configure a plan that is like an existing flow with one or two
> adjustments.
> - There is now a way to customize MetaDeploy step settings such
> as `name` and `is_required` on a step-by-step basis, using
> `ui_options` in the plan config.
> - The task will now find or create a `PlanTemplate` as
> necessary, matching existing PlanTemplates on the product and
> plan name. This means the plan config no longer needs to
> reference a plan template by id, which makes it easier to
> publish to multiple instances of MetaDeploy.

- The `install_upgrade` flow was renamed to `install_regression` to
better reflect the use case it is focused on. There are also a few
updates to what it does:

> - It will now install the latest beta release of managed
> packages instead of the latest final release.
> - It now runs the `config_managed` flow after upgrading the
> managed package, so that it will work if this flow has
> references to newly added components.

Changes:

- Added support for deploying Lightning Web Components.
- Fixed the bulk data load task to handle null values in a datetime
column.
- The [](ci-master) flow now explicitly avoids trying to
install beta releases of dependencies (since it's meant for use
with non-scratch orgs and we block installing betas there since they
can't be upgraded).

- Added a new flow, `install_upgrade`, which can be used for testing
package upgrades. It installs and configures the _previous_
release of the package, then installs the latest release.
- Fixed an error when using `cci org info --json` (fixes 1013).

- Fixed a bug where flow options specified on the command line were
not passed to tasks correctly.
- `cci service connect` now shows a more helpful error message if you
call it with a service name that CumulusCI doesn't know about.
Fixes 752.
- Deleted scratch orgs will no longer show the number of days since
they were created in `cci org list`. Thanks to \21aslade for the
fix.
- Updates to the MetaDeploy publish task:
- It is now possible to publish a new plan for an existing
version.
- It is now possible to specify the AllowedList to which a plan is
visible.
- Updates to Robot Framework support:
- Fixed a bug in the `robot` task: it now accepts an option called
`test` rather than `tests`, since the latter was ignored by
Robot Framework.
- Fixed some stability problems with the `Populate Field` keyword.
- The `robot_libdoc` task has been replaced with a new task of the
same name that can generate a single HTML document for multiple
keyword files by passing a comma-separated list of files to the
`path` option.

- Mapping enhancements for bulk `QueryData` and `LoadData` tasks
- The mapping yaml file no longer requires using `Id: sf_id` as a
field mapping. If not provided, `QueryData` and `LoadData` will
use local database ids instead of Saleforce OIDs for storing
lookup relationships. Previous mappings which specify the
`Id: sf_id` mapping will continue to work as before using the
Salesforce OID as the mapping value.
- The mapping yaml file's `lookups:` section now handles defaults
to allow simpler lookup mappings. The only key required is now
`table`. If the `key_field` is provided it will be used.
- The `sql_path` option on `QueryData` can be used to provide the file
path where a SQL script should be written. If this option is used, a
sqlite in-memory database is used and discarded. This is useful for
storing data sets in a Github repository and allowing diffs of the
dataset to be visible when reviewing Pull Requests
- When using this option, it is best to make sure your mapping
yaml file does not provide a field mapping for the `Id` field.
This will help avoid merge conflicts if querying data from
different orgs such as scratch orgs.
- The [sql_path]{.title-ref} option on `LoadData` can be used to
provide the file path where a SQL script file should be read and
used to load an in-memory sqlite database for the load operation.

- Fixed a bug that caused the `cci` command to check for a newer
version on every run, rather than occasionally. Also we now detect
whether CumulusCI was installed using Homebrew and recommend an
upgrade command accordingly.
- CumulusCI now automatically generates its own keychain key and
stores it in the system keychain (using the Python
[keyring]{.title-ref} library). This means that it is no longer
necessary to specify a CUMULUSCI_KEY as an environment variable.
(However, the environment variable will still be preferred if it is
there, and it will be migrated to the system keychain.)
- New task `connected_app` makes it easier to deploy and configure the
Connected App needed for CumulusCI's keychain to work with
persistent orgs. The connected app is deployed using `sfdx` to an
org in the `sfdx` keychain and defaults to the
`defaultdevhubusername`.
- The `robot` task gives a more helpful error message if you forget to
specify an org.
- Updates to the task for publishing to MetaDeploy:
- Dependency installation steps are now named using the package
name and version.
- The task options have been revised to match changes in the
MetaDeploy API. An optional `plan_template_id` is now accepted.
`preflight_message` is now named `preflight_message_additional`
and is optional. `post_install_message` is now named
`post_install_message_additional` and is optional.