Autoclip

Latest version: v0.2.1

Safety actively analyzes 624665 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

0.2.1

Patch v0.2.1 fixes minor issues concerning the optimizer wrapping behaviour.
- Fixed an error which occurred when pickling the `OptimizerWithClipping` class generated from the pytorch wrapper API.
- Added support for torch hooks for the `OptimizerWithClipping` class, to allow more seamless integration into existing code.
- Added support for returning the loss results from an `OptimizerWithClipping` step when using a closure
- Minor updates to docstrings of the `OptimizerWithClipping` class

0.2

Only the PyTorch API is updated in this version.
PyTorch API
There are three main improvements to the PyTorch AutoClip implementation. They are as follows:
- New Optimizer Wrapping Pattern
- Parameter Value and Type Checking
- Testing with Pytest
Optimizer Wrapping
There were a couple of cases where the previous AutoClip API could not be used, primarily when the training code was imported from an outside library. To address this friction, the optimizer wrapping pattern was added, along with the new `OptimizerWithClipper` class. This class is a true wrapper for the optimizer in question, meaning it will function almost identically, even if the optimizer has special fields and functions. This allows us to shim in clipping behavior anywhere we create an optimizer. A marked improvement!
Parameter Value and Type Checking
Clipper classes now must implement the `verify_parameter_settings` function. This function will be called during `__init__` for the clipper defaults, and during `add_param_group`. This should help give users much more accurate and useful error messages about invalid parameter inputs.
Testing
We now have tests! Hate to write them, love to have them. Not much more to say about this other than the reliability this should impose.

0.2.0

0.1

First release of AutoClip as a package! The package is now available to install from pypi, and base features are all completed.

Features
PyTorch API
The torch implementation of AutoClip has received a lot of attention due to some contributor preferences \**ahem*\*. As such it has a host of features beyond the basic implementation of the AutoClip algorithm.

Additional Clipping Method
In addition to the standard QuantileClip method, there is also StandardClip method, which will clip based on the standard deviation of previous gradients. Whether this is a beneficial or meaningful difference has yet to be seen, as no significant testing has been done.

Local Clipping
The torch clippers (both QuantileClip and StandardClip) support both global and local clipping modes. This allows for parameter level clipping adaptation if desired.

Checkpointing
Torch clippers also now support checkpointing, using the pytorch `state_dict` system. This is very important for using AutoClip in any sort of at scale training environment.

Tensorflow API
Only minor changes have been made to the tensorflow API. These aim to maintain the same parameter and naming schemes between the two versions. As such, the tensorflow implementation does not have feature parity with the pytorch implementation.

0.1.0

Links

Releases

Has known vulnerabilities

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.