Gradient-accumulator

New API feature

Custom Batch Normalization layer

from gradient_accumulator.layers import AccumBatchNormalization

model = Sequential()
model.add(AccumBatchNormalization())

New API!

Model wrapper:

from gradient_accumulator import GradientAccumulateModel

model = Model(...)
model = GradientAccumulateModel(accum_steps=4, inputs=model.input, outputs=model.output)


Optimizer wrapper:

from gradient_accumulator import GradientAccumulateModel

opt = tf.keras.optimizers.SGD(1e-2)
opt = GradientAccumulateOptimizer(accum_steps=4, optimizer=opt)

How to use?

| Method | Usage |
| - | - |
| `GAModelWrapper` | `model = GAModelWrapper(accum_steps=4, inputs=model.input, outputs=model.output)` |
| `GAOptimizerWrapper` | `opt = GAOptimizerWrapper(accum_steps=4, optimizer=tf.keras.optimizers.Adam(1e-3))` |

**Full Changelog**: https://github.com/andreped/GradientAccumulator/compare/v0.2.2...v0.3.0

This is a minor patch release.

What's changed:
- Added support for tensorflow-metal, enabling GA on macOS with GPUs
- Bug fix related to undefined gradients: https://github.com/andreped/GradientAccumulator/issues/24

**Full Changelog**: https://github.com/andreped/GradientAccumulator/compare/v0.2.1...v0.2.2

This is a minor patch release.

**What's changed:**
- Fixed typo by renaming `use_acg` to `use_agc`.

What's Changed
* AGC and mixed precision are now compatible
* Support for both float16 and bfloat16 on GPU and TPU, respectively
* Set default accum_steps in GAModelWrapper to 1
* Improved documentation regarding usage of AGC, GPU/TPU, and recommended model format (SavedModel)
* Support for 3D operations in AGC, such as Conv3D, by dbouget in https://github.com/andreped/GradientAccumulator/pull/17
* Corrected mean reduction to before gradient computation dbouget in https://github.com/andreped/GradientAccumulator/pull/2

New Contributors
* dbouget made their first contribution in https://github.com/andreped/GradientAccumulator/pull/17

**Full Changelog**: https://github.com/andreped/GradientAccumulator/compare/v0.1.5...v0.2.0