=====
The following change list is only interim. A followup change will revise this entry with better information
covering what changed in 4.0, which is considerably more.
* Some new modules. The scripts modules came from other repositories, for centralization reasons. The other modules
are originally refactorings to make functionality more broadly available at various stages of bootstrapping
this library.
* ``ecr_scripts`` has support for command line scripts related to ECR repositories.
* ``env_base`` has support for bits of environmental foothold needed before ``env_utils`` or ``s3_utils`` are ready.
* ``env_manager`` is a higher-level environmental abstraction built after ``env_utils`` is available.
* ``env_scripts`` has support for command line scripts related to configurable environments and the global env bucket.
* New ``make`` targets:
* ``make test-all`` runs all tests
* ``make test-most`` runs all unit and integration tests (marked ``unit``, ``integration`` or ``integrationx``),
but not things likely to fail (marked ``beanstalk failure`` or ``direct_es_query``).
* ``make test-integrations`` runs all integration tests (marked ``integration`` or ``integrationx``),
but not things likely to fail (marked ``beanstalk failure`` or ``direct_es_query``).
* ``make test-direct-es-query`` runs any test marked ``direct_es_query.
* ``test-units-with-coverage`` runs unit tests with the ``coverage`` feature.
* ``test-for-ga`` is an indirect way to call ``test-units-with-coverage``, and will be what the GithubActions
workflow calls.
* Configurable environmental support for orchestrated C4 applications (Fourfront and CGAP) in ``env_utils``
(`C4-689 <https://hms-dbmi.atlassian.net/browse/C4-689>`_).
* Extend that support to allow mirroring to be enabled
(`C4-734 <https://hms-dbmi.atlassian.net/browse/C4-734>`_).
The net result is a configurable environment in which the env descriptor in the global env bucket can contain
these new items:
=============================== ===============================================================================
Key Notes
=============================== ===============================================================================
``"dev_data_set_table"`` Dictionary mapping envnames to their preferred data set
``"dev_env_domain_suffix"`` e.g., .abc123def456ghi789.us-east-1.rds.amazonaws.com
``"foursight_bucket_table"`` A table mapping environments to another table mapping chalice stages to buckets
``"foursight_url_prefix"`` A prefix string for use by foursight.
``"full_env_prefix"`` A string like "cgap-" that precedes all env names
``"hotseat_envs"`` A list of environments that are for testing with hot data
``"indexer_env_name"`` The environment name used for indexing (being phased out)
``"is_legacy"`` Should be ``"true"`` if legacy effect is desired, otherwise omitted.
``"stage_mirroring_enabled"`` Should be ``"true"`` if mirroring is desired, otherwise omitted.
``"orchestrated_app"`` This allows us to tell 'cgap' from 'fourfront', in case there ever is one.
``"prd_env_name"`` The name of the prod env
``"public_url_table"`` Dictionary mapping envnames & pseudo_envnames to public urls
``"stg_env_name"`` The name of the stage env (or None)
``"test_envs"`` A list of environments that are for testing
``"webprod_pseudo_env"`` The pseudo-env that is a token name to use in place of the prd env for shared
stg/prd situations, replacing ``fourfront-webprod`` in the legacy system.
(In orchestrations, this should usually be the same as the ``prd_env_name``.
It may or may not need to be different if we orchestrate the legacy system.)
=============================== ===============================================================================
* In ``base``:
* ``compute_prd_env_for_project``
* ``compute_stg_env_for_project``
* ``get_env_info`` (replaces ``beanstalk_utils.get_beanstalk_info``)
* ``get_env_real_url`` (replaces ``beanstalk_utils.get_beanstalk_real_url``)
* In ``beanstalk_utils``:
* Removed:
* ``swap_cname``
NOTE: This was never invoked by automatic programs, so we didn't do a deprecation stage.
* Deprecated:
* ``get_beanstalk_info`` is deprecated. Use ``beanstalk_utils.get_env_info``.
* ``get_beanstalk_real_url`` is deprecated. Use ``env_utils.get_env_real_url``.
NOTE: These continue to work for now, but will be removed in the future.
Please update code to use recommended replacement.
* In ``cloudformation_utils``:
* Added function``discover_foursight_check_runner_name``.
* Added function ``tokenify``.
* Moved ``DEFAULT_ECOSYSTEM`` to ``cloudformation_utils``. Importing it from this library is now deprecated.
* In ``common``:
* New variables:
* ``CHALICE_STAGE_DEV``
* ``CHALICE_STAGE_PROD``
* ``CHALICE_STAGES``
* ``DEFAULT_ECOSYSTEM`` (moved from ``cloudformation_utils``)
* ``LEGACY_CGAP_GLOBAL_ENV_BUCKET``
* ``LEGACY_GLOBAL_ENV_BUCKET``
* New type hint (variable):
* ``ChaliceStage``
* In ``ecr_utils``:
* Removed ``CGAP_ECR_LAYOUT``. Use ``ECRUtils.ECR_LAYOUT`` instead.
* Deprecated ``CGAP_ECR_REGION``. Use ``ECRUtils.REGION`` or ``common.REGION`` instead.
* In ``ecs_utils``:
* Added ``ECSUtils.REGION``.
* In ``env_base``:
* Moved ``EnvBase`` to here from ``s3_utils``.
* Added ``s3_utils.s3Base`` (factored out of ``s3_utils.s3Utils``)
* In ``env_utils``:
* Removed:
* ``guess_mirror_env``
* ``make_env_name_cfn_compatible``
NOTE: This was not believed to be used anywhere so is presumably no great hardship.
(Kent also didn't like the naming, which used a confusing abbreviation.)
* New functions:
* ``blue_green_mirror_env``
* ``compute_prd_env_for_project``
* ``data_set_for_env``
* ``ecr_repository_for_env``
* ``full_cgap_env_name``
* ``full_fourfront_env_name``
* ``get_env_from_context``
* ``get_env_real_url`` (replaces ``beanstalk_utils.get_beanstalk_real_url``)
* ``get_foursight_bucket``
* ``get_foursight_bucket_prefix``
* ``get_standard_mirror_env``
* ``has_declared_stg_env``
* ``indexer_env_for_env`` (introduced _and_ deprecated during beta)
* ``infer_foursight_from_env``
* ``infer_foursight_url_from_env``
* ``is_indexer_env`` (introduced _and_ deprecated during beta)
* ``is_orchestrated``
* ``maybe_get_declared_prd_env_name``
* ``permit_load_data``
* New classes:
* ``ClassificationParts``
* ``EnvNames``
* ``EnvUtils``
* ``PublicUrlParts``
* Always erring:
* ``indexer_env_for_env``
* ``is_indexer_env``
NOTE: These functions unconditionally raise an error indicating that the functionality is no longer available.
Their callers must be rewritten, probably in a way that is not a simple substitution.
* Removed all top-level variables from ``env_utils`` variables, moving them to ``env_utils_legacy``.
This includes but is not limited to variables with names starting with ``CGAP_``, ``FF_`` or ``BEANSTALK_``.
These are deprecated and should not be used outside of ``dcicutils``.
Within ``dcicutils``, they may be used only for testing.
All ``env_utils`` functionality should be accessed through functions, not variables.
* In ``exceptions``:
* ``BeanstalkOperationNotImplemented``
* ``EnvUtilsLoadError``
* ``IncompleteFoursightBucketTable``
* ``LegacyDispatchDisabled``
* ``MissingFoursightBucketTable``
* ``NotUsingBeanstalksAnyMore``
* Added tech debt by disabling certain tests or marking them for later scrutiny.
Three new pytest markers were added in ``pytest.ini``:
* ``beanstalk_failure`` - An obsolete beanstalk-related test that needs fixing
* ``direct_es_query`` - A test of direct ES _search that is disabled for now
and needs to move inside the firewall
* ``stg_or_prd_testing_needs_repair`` - Some or all of a test that was failing on stg/prd
has been temporarily disabled
* ``recordable`` declares a test to use "recorded" technology so that if ``RECORDING_ENABLED=TRUE``,
a new test recording is made