Sagemaker-pytorch-training

Features

* add support for native PyTorch DDP distribution

Bug Fixes and Other Changes

* deriver master node from training environment

Features

* Add Heterogeneous Cluster support

Bug Fixes and Other Changes

* CI

Features

* add data parallelism support (11) (12)

Bug Fixes and Other Changes

* use ubuntu 18.04 base image in dlc gpu image
* remove TODOs in 1.6.0 dlc gpu dockerfile and reduce parameters for data parallel integ test
* use base cuda 11 image for test dlc gpu image
* use 1.6.0 for gpu tests and disable horovod tests
* remove local data parallel integ test
* use sagemaker-training 3.7.0 and enable data parallel integ tests
* patch socket call and update flake8 violations

Features

* Use MPIRunnerType

Bug Fixes and Other Changes

* Update main buildspec to only perform CPU integration tests
* Add GPU and unit test buildspecs
* Pin SageMaker version to less than v2

Documentation Changes

* improve training.py doc style