Highlights
:rocket: Ensure equal importance of loss penalties in terms of grad norms in a specific layer :rocket:
> Other weights can then be applied, e.g. to have a loss term that influences the weights 50% more than the other.
python
Given two losses
loss_a, loss_b
Compute the loss_b weight
loss_b_weight = calculate_adaptive_weight(loss_a, loss_b, layer)
loss_b_weighted = loss_b_weight * loss_b
So that loss_a and loss_b have equal importance in the specified layer grads
loss_a_gradnorm = torch.norm(torch.autograd.grad(loss_a, layer, retain_graph=True)[0])
loss_b_weighted_gradnorm = torch.norm(torch.autograd.grad(loss_b_weighted, layer, retain_graph=True)[0])
assert torch.allclose(loss_a_gradnorm, loss_b_weighted_gradnorm)
What's Changed
* Disable pivot sorting in latex df by lucmos in https://github.com/lucmos/anypy/pull/12
* Add function to compute adaptive loss weights by lucmos in https://github.com/lucmos/anypy/pull/13
**Full Changelog**: https://github.com/lucmos/anypy/compare/0.0.5...0.0.6