Madminer

New features:
- `ParameterizedRatioEstimator` now optionally rescales parameters (`theta`) to zero mean and unit variance during training. Use the keyword `rescale_params` in `ParameterizedRatioEstimator.train()`.
- Batching of parameter points in the `AsymptoticLimits` functions now also when using weighted events.

API and breaking changes:
- In MET smearing, the relative term is now multiplied with HT, defined as the scalar sum over the pT of all *visible* particles (before it was all particles).

Bug fixes:
- Fixed critical bug in the MET calculation in `LHEReader`, which caused the y component of the MET object to be wrong.
- Fixed crashes when training a likelihood ratio estimator without providing joint score information.

New features:
- `plot_histograms()` can now also visualize observed data / Asimov data.
- In 2D parameter spaces, calculating limits with `mode="adaptive-sally"` now works as described in [https://arxiv.org/abs/1805.00020](1805.00020). In higher dimensions, it still just concatenates the scalar product of score and parameter vector with all score components to form a `(d+1)`-dimensional observable space.

Bug fixes:
- Fixed bug in `plot_histograms()`.

Tutorials and documentation:
- The MadMiner paper is out!
- Updated README and docs, including a new troubleshooting list.
- Cleaned up examples folder.

New features:
- In `AsymptoticLimits`, the adaptive histogram binning can now be based on the weights summed over the whole parameter grid instead of just a central point. This is now also the default option.
- New function `plot_histograms()` in `madminer.plotting` to visualize the histograms used by `AsymptoticLimits`.

Bug fixes:
- Substantially improved automatic histogram binning and fixed some numerical issues in `AsymptoticLimits` functions.

Tutorials and documentation:
- Updated tutorial with new histogram plots.

New features:
- More observables for `LHEReader.add_observable`: Users can use `"p_truth"` to access particles before smearing, and (at least with XML parsing) there are new global observables `"alpha_qcd", "alpha_qed", "scale"`. `LHEReader.add_observable_from_function()` now accepts functions that take unsmeared particles as first argument.

Bug fixes:
- Fixed bug in `sample_train_ratio()` with `return_individual_n_effective=True`
- Fixed bug in `DelphesReader` when no events survive cuts

Tutorials and documentation:
- Removed outdated Docker link from docs
- Changed morphing basis in particle physics tutorial to work around a weird bug inn the MG-Pythia interface, see 371

Internal changes:
- Refactored LHE parsing. LHE files are now not read into memory all at once, but sequentially.

New features:
- `AsymptoticLimits` now supports the SALLINO method, estimating the likelihood with one-dimensional histograms of the scalar product of `theta` and the estimated score.
- Improved default histogram binning in `AsymptoticLimits` and added more binning options, including fully manual specification of the binning.
- Histograms now calculate a rough approximate of statistical uncertainties in each bin and give out a warning if it’s large. (At DEBUG logging level they’ll also print the uncertainties always, and `Histogram.histo_uncertainties` lets the user access the uncertainties.)

Breaking / API changes:
- The `AsymptoticLimits` functions `expected_limits()` and `observed_limits()` now return `(theta_grid, p_values, i_ml, llr_kin, log_likelihood_rate, histos)`. `histos` is a list of histogram classes, the tutorial shows how they allow us to plot the histograms. The `returns` keyword to these functions is removed. The keywords `theta_ranges` and `resolutions` were renamed to `grid_ranges` and `grid_resolutions`.
- Changed some ML default settings: less hidden layers, smaller batch size.
- Changed function names in the `FisherInformation` class (the old names are still available as aliases for now, but deprecated).

Bug fixes:
- Various small bug fixes.

Tutorials and documentation:
- `AsymptoticLimits` is finally properly documented.
- All incomplete user-facing docstrings were updated.

Internal changes:
- Refactored histogram class.
- `AsymptoticLimits` is now much more memory efficient.

New features
- Histograms in `AsymptoticLimits` now by default use one set of weighted events for all parameter points, reweighting them appropriately

API / breaking changes
- `AsymptoticLimits` functions keyword `n_toys_per_theta ` renamed to `n_histo_toys`

Internal changes
- Refactored HDF5 file access