Jaxopt

Bug fixes and enhancements

- Fix fragile tests in perturbation module, by Roy Frostig.
- Better verbose handling, by Vincent Roulet.
- Remove boston dataset, by Vincent Roulet.
- Control variate support in perturbation module, by Quentin Berthet.

Contributors

Roy Frostig, Vincent Roulet, Quentin Berthet.

Bug fixes and enhancements

- Added SPS+ variant to PolyakSGD, by Fabian Pedregosa.
- Fixed jax.config import, by Sergei Lebedev.
- Fixed typos in the doc + upload missing image, by Fabian Pedregosa.

Contributors

Fabian Pedregosa, Sergei Lebedev.

Bug fixes and enhancements

- Improved Resnet examples in Flax and Haiku, by Fabian Pedregosa.
- Documentation is built as part of the continuous integration workflow, by Fabian Pedregosa.
- Jit update method, by Mathieu Blondel.
- Linesearch improvements, by Vincent Roulet.
- GaussNewton and LevenbergMarquardt improvements, by Amir Saadat.
- Verbose support even when jit=True, by Amir Saadat.
- Various doc improvements, by Fabian Pedregosa.

Contributors

Amir Saadat, Fabian Pedregosa, Mathieu Blondel, Peter Hawkins, Vincent Roulet.

New features

- Added Broyden algorithm to solve non linear root equations, by Zaccharie Ramzi.

Bug fixes and enhancements

- Fixed “LBFGSB produces NaN for certain conditions”, by Vincent Roulet.
- Better layout for notebook gallery, by Fabian Pedregosa.
- Added support for complex numbers in nonlinear conjugate gradient, by Gregor Thalhammer.
- Added support for complex numbers in LBFGS and zoom line search, by Gregor Thalhammer.
- Various LBFGS improvements, by Vincent Roulet.
- Completely revamped zoom linesearch, by Vincent Roulet.
- Added continuous integration for python 3.11, fixed typing extension issue, by Vincent Roulet.
- Added number of function/grad/prox/etc… evaluation for various solvers, by Zaccharie Ramzi.
- Fixed implicit differentiation in LBFGSB, by Nathan Simpson.
- Ensure weak-type consistency, by Mathieu Blondel.
- Extend convenience API for QP + doc, by Louis Bethune.
- Add interpolation explanations for Polyak/Armijo API doc, by Louis Bethune.
- Support has_aux in AndersonAcceleration, by Louis Bethune.
- Drop Python 3.7 support, by Mathieu Blondel.
- Fixed typos in perturbation docs, by Guillaume Dalle.
- Fixed pytree support in tree_inf_norm, by Emily Fertig.
- Fixed NaN handling in LBFGSB, by Srinivas Vasudevan.
- Use jnp.ndarray instead of jnp.array, by Peter Hawkins.

Contributors

Louis Bethune, Emily Fertig, Fabian Pedregosa, Gregor Thalhammer-Thurner, Guillaume Dalle, Mathieu Blondel, Nathan Simpson, Peter Hawkins, Srinivas Vasudevan, Vincent Roulet, Zaccharie Ramzi.

New features

- Added jaxopt.LBFGSB, by Emily Fertig.
- Added jaxopt.perturbations.make_perturbed_fun, by Quentin Berthet.

Bug fixes and enhancements

- Allow to pass a function as value_and_grad option, by Chansoo Lee.
- Fixed imaml tutorial (speed and correctness), by Zaccharie Ramzi.
- Misc improvements in resnet_flax example, by Fabian Pedregosa.
- Fixed prox to handle pytrees, by Vincent Roulet.
- Added control variate to make_perturbed_argmax, by Lawrence Stewart.
- Added inverse hessian approximation to the returned state, Aymeric Galan.
- Avoid closing over dynamic jax tracers in the bisection solver, by Roy Frostig.
- Follow pjit API changes, by Yash Katariya and Peter Hawkins.
- Added isotonic module to documentation, by Mathieu Blondel.

Contributors

Aymeric Galan, Chansoo Lee, Emily Fertig, Fabian Pedregosa, Lawrence Stewart, Mathieu Blondel, Peter Hawkins, Quentin Berthet, Roy Frostig, Vincent Roulet, Yash Katariya, Zaccharie Ramzi.

New features

- Added new Hager-Zhang linesearch in LBFGS, by Srinivas Vasudevan (code review by Emily Fertig).
- Added perceptron and hinge losses, by Quentin Berthet.
- Added binary sparsemax loss, sparse_plus and sparse_sigmoid, by Vincent Roulet.
- Added isotonic regression, by Michael Sander.

Bug fixes and enhancements

- Added TPU support to notebooks, by Ayush Shridhar.
- Allowed users to restart from a previous optimizer state in LBFGS, by Zaccharie Ramzi.
- Added faster error computation in gradient descent algorithm, by Zaccharie Ramzi.
- Got rid of extra function call in BFGS and LBFGS, by Zaccharie Ramzi.
- Improved dtype consistency between input and output of update method, by Mathieu Blondel.
- Added perturbed optimizers notebook and narrative documentation, by Quentin Berthet and Fabian Pedregosa.
- Enabled auxiliary value returned by linesearch methods, by Zaccharie Ramzi.
- Added distributed examples to the website, by Fabian Pedregosa.
- Added Custom loop pjit example, by Felipe Llinares.
- Fixed wrong latex in maml.ipynb, by Fabian Pedregosa.
- Fixed bug in backtracking line search, by Srinivas Vasudevan (code review by Emily Fertig).
- Added pylintrc to top level directory, by Fabian Pedregosa.
- Corrected the condition function in LBFGS, by Zaccharie Ramzi.
- Added custom loop pmap example, by Felipe Llinares.
- Fixed pytree support in IterativeRefinement, by Louis Béthune.
- Fixed has_aux support in ArmijoSGD, by Louis Béthune.
- Documentation improvements, by Fabian Pedregosa and Mathieu Blondel.

Contributors

Ayush Shridhar, Fabian Pedregosa, Felipe Llinares, Louis Bethune, Mathieu Blondel, Michael Sander, Quentin Berthet, Srinivas Vasudevan, Vincent Roulet, Zaccharie Ramzi.