Aws-parallelcluster

------

**CHANGES**
- Upgrade Slurm to version 20.02.7.

------

**ENHANCEMENTS**
- Enable support for ARM instances in China and GovCloud regions when using Ubuntu 18.04 or Amazon Linux 2.
- Add validation for `cluster_type` configuration parameter in `cluster` section.
- Add validation for `compute_type` configuration parameter in `queue` section.

**CHANGES**
- Upgrade EFA installer to version 1.11.2
- EFA configuration: efa-config-1.7 (no change)
- EFA profile: efa-profile-1.4 (from efa-profile-1.3)
- EFA kernel module: efa-1.10.2 (no change)
- RDMA core: rdma-core-31.2amzn (no change)
- Libfabric: libfabric-1.11.1amzn1.0 (no change)
- Open MPI: openmpi40-aws-4.1.0 (no change)

**BUG FIXES**
- Fix issue with ``awsbsub`` command when setting environment variables for the job submission

------

**ENHANCEMENTS**
- Improve cluster config validation by using cluster target AMI when invoking RunInstances in dryrun mode
- Improve configuration procedure for the Munge service.

**CHANGES**
- Update Python version used in ParallelCluster virtualenvs from version 3.6.9 to version 3.6.13.

**BUG FIXES**
- Fix sanity checks with ARM instance types by using cluster AMI when performing validation
- Fix `enable_efa` parameter validation when using Centos8 and Slurm or ARM instances.
- Use non interactive `apt update` command when building custom Ubuntu AMIs.
- Fix `encrypted_ephemeral = true` when using Alinux2 or CentOS8

------

**ENHANCEMENTS**
- Add support for me-south-1 region (Bahrein), af-south-1 region (Cape Town) and eu-south-1 region (Milan)
- At the time of this version launch:
- Amazon FSx for Lustre and ARM instance types are not supported in me-south-1, af-south-1 and eu-south-1
- AWS Batch is not supported in af-south-1
- EBS io2 is not supported in af-south-1 and eu-south-1
- Install Arm Performance Libraries (APL) 20.2.1 on ARM AMIs (CentOS8, Alinux2, Ubuntu1804).
- Install EFA kernel module on ARM instances with `alinux2` and `ubuntu1804`. This enables support for `c6gn` instances.
- Add support for io2 and gp3 EBS volume type.
- Add `iam_lambda_role` parameter under `cluster` section to enable the possibility to specify an existing IAM role to
be used by AWS Lambda functions in CloudFormation.
When using `sge`, `torque`, or `slurm` as the scheduler,
`pcluster` will not create any IAM role if both `ec2_iam_role` and `iam_lambda_role` are provided.
- Improve robustness of a Slurm cluster when clustermgtd is down.
- Configure NFS threads to be max(8, num_cores) for performance. This enhancement will not take effect on Ubuntu 16.04.
- Optimize calls to DescribeInstanceTypes EC2 API when validating cluster configuration.

**CHANGES**
- Upgrade EFA installer to version 1.11.1.
- EFA configuration: ``efa-config-1.7`` (from efa-config-1.5)
- EFA profile: ``efa-profile-1.3`` (from efa-profile-1.1)
- EFA kernel module: ``efa-1.10.2`` (no change)
- RDMA core: ``rdma-core-31.2amzn`` (from rdma-core-31.amzn0)
- Libfabric: ``libfabric-1.11.1amzn1.0`` (from libfabric-1.11.1amzn1.1)
- Open MPI: ``openmpi40-aws-4.1.0`` (from openmpi40-aws-4.0.5)
- Upgrade Intel MPI to version U8.
- Upgrade NICE DCV to version 2020.2-9662.
- Set default systemd runlevel to multi-user.target on all OSes during ParallelCluster official AMI creation.
The runlevel is set to graphical.target on head node only when DCV is enabled. This prevents the execution of
graphical services, such as x/gdm, when they are not required.
- Download Intel MPI and HPC packages from S3 rather than Intel yum repos.
- Change the default of instance types from the hardcoded `t2.micro` to the free tier instance type
(`t2.micro` or `t3.micro` dependent on region). In regions without free tier, the default is `t3.micro`.
- Enable support for p4d as head node instance type (p4d was already supported as compute node in 2.10.0).
- Pull Amazon Linux Docker images from public ECR when building docker image for `awsbatch` scheduler.
- Increase max retry attempts when registering Slurm nodes in Route53.

**BUG FIXES**
- Fix pcluster createami for Ubuntu 1804 by downloading SGE sources from Debian repository and not from the EOL
Ubuntu 19.10.
- Remove CloudFormation DescribeStacks API call from AWS Batch Docker entrypoint. This removes the risk of job
failures due to CloudFormation throttling.
- Mandate the presence of `vpc_settings`, `vpc_id`, `master_subnet_id` in the config file to avoid unhandled exceptions.
- Set the default EBS volume size to 500 GiB when volume type is `st1` or `sc1`.
- Fix installation of Intel PSXE package on CentOS 7 by using yum4.
- Fix routing issues with multiple Network Interfaces on Ubuntu 18.04.

------

**ENHANCEMENTS**
- Add support for CentOS 8 in all Commercial regions.
- Add support for P4d instance type as compute node.
- Add the possibility to enable NVIDIA GPUDirect RDMA support on EFA by using the new `enable_efa_gdr` configuration
parameter.
- Enable support for NICE DCV in GovCloud regions.
- Enable support for AWS Batch scheduler in GovCloud regions.
- FSx Lustre:
- Add possibility to configure Auto Import policy through the new `auto_import_policy` parameter.
- Add support to HDD storage type and the new `storage_type` and `drive_cache_type` configuration parameters.
- Create a CloudWatch Dashboard for the cluster, named `<clustername>-<region>`, including head node EC2 metrics and
cluster logs. It can be disabled by configuring the `enable` parameter in the `dashboard` section.
- Add `-r/-region` arg to `pcluster configure` command. If this arg is provided, configuration will
skip region selection.
- Add `-r/-region` arg to`ssh` and `dcv connect` commands.
- Add `cluster_resource_bucket` parameter under `cluster` section to allow the user to specify an existing S3 bucket.
- `createami`:
- Add validation step to fail when using a base AMI created by a different version of ParallelCluster.
- Add validation step for AMI creation process to fail if the selected OS and the base AMI OS are not consistent.
- Add `--post-install` parameter to use a post installation script when building an AMI.
- Add the possibility to use a ParallelCluster base AMI.
- Add possibility to change tags when performing a `pcluster update`.
- Add new `all_or_nothing_batch` configuration parameter for `slurm_resume` script. When `True`, `slurm_resume` will
succeed only if all the instances required by all the pending jobs in Slurm will be available.
- Enable queue resizing on update without requiring to stop the compute fleet. Stopping the compute fleet is only
necessary when existing instances risk to be terminated.
- Add validator for EBS volume size, type and IOPS.
- Add validators for `shared_dir` parameter when used in both `cluster` and `ebs` sections.
- Add validator `cfn_scheduler_slots` key in the `extra_json` parameter.

**CHANGES**
- CentOS 6 is no longer supported.
- Upgrade EFA installer to version 1.10.1
- EFA configuration: `efa-config-1.5` (from efa-config-1.4)
- EFA profile: `efa-profile-1.1` (from efa-profile-1.0.0)
- EFA kernel module: `efa-1.10.2` (from efa-1.6.0)
- RDMA core: `rdma-core-31.amzn0` (from rdma-core-28.amzn0)
- Libfabric: `libfabric-1.11.1amzn1.1` (from libfabric-1.10.1amzn1.1)
- Open MPI: `openmpi40-aws-4.0.5` (from openmpi40-aws-4.0.3)
- Unifies installer runtime options across x86 and aarch64
- Introduces `-g/--enable-gdr` switch to install packages with GPUDirect RDMA support
- Updates to OMPI collectives decision file packaging, migrated from efa-config to efa-profile
- Introduces CentOS 8 support
- Upgrade NVIDIA driver to version 450.80.02.
- Install NVIDIA Fabric manager to enable NVIDIA NVSwitch on supported platforms.
- Remove default region `us-east-1`. After the change, `pcluster` will adhere to the following lookup order for region:
1. `-r/--region` arg.
2. `AWS_DEFAULT_REGION` environment variable.
3. `aws_region_name` in ParallelCluster configuration file.
4. `region` in AWScli configuration file.
- Slurm: change `SlurmctldPort` to 6820-6829 to not overlap with default `slurmdbd` port (6819).
- Slurm: add `compute_resource` name and `efa` as node features.
- Remove validation on `ec2_iam_role` parameter.
- Improve retrieval of instance type info by using `DescribeInstanceType` API.
- Remove `custom_awsbatch_template_url` configuration parameter.
- Upgrade `pip` to latest version in virtual environments.
- Upgrade image used by CodeBuild environment when building container images for AWS Batch clusters, from
`aws/codebuild/amazonlinux2-x86_64-standard:1.0` to `aws/codebuild/amazonlinux2-x86_64-standard:3.0`.

**BUG FIXES**
- Retrieve the right number of compute instance slots when instance type is updated.
- Include user tags in compute nodes and EBS volumes.
- Fix `pcluster status` output when head node is stopped.
- `pcluster update`:
- Fix issue when tags are specified but not changed.
- Fix issue when the `cluster` section label changed.
- Fix issue when `shared_dir` and `ebs_settings` are both configured in the `cluster` section.
- Fix `cluster` and `cfncluster` compatibility in `extra_json` parameter.
- Fix `pcluster configure` to avoid using default/initial values for internal parameter initialization.
- Fix pre/post install script arguments management when using double quotes.
- Fix a bug that was causing `clustermgtd` and `computemgtd` sleep interval to be incorrectly computed when
system timezone is not set to UTC.
- Fix queue name validator to properly check for capital letters.
- Fix `enable_efa` parameter validation for `queue` section.
- Fix CloudWatch Log Group creation for AWS Lambda functions handling CloudFormation Custom Resources.

-----

**BUG FIXES**
- Fix cluster creation with the head node in a private subnet when it doesn't get a public IP.