Hyperspy

===================

New features
------------

Core
^^^^

* New syntax to index the :py:class:`~.axes.AxesManager`.
* New Signal methods to transform between Signal subclasses. More information
:ref:`here <transforming_signal-label>`.

+ ``hyperspy.signal.Signal.set_signal_type``
+ ``hyperspy.signal.Signal.set_signal_origin``
+ ``hyperspy.signal.Signal.as_signal2D``
+ ``hyperspy.signal.Signal.as_signal1D``

* The string representation of the Signal class now prints the shape of the
data and includes a separator between the navigation and the signal axes e.g
(100, 10| 5) for a signal with two navigation axes of size 100 and 10 and one
signal axis of size 5.
* Add support for RGBA data. See :ref:`signal.change_dtype`.
* The default toolkit can now be saved in the preferences.
* Added full compatibility with the Qt toolkit that is now the default.
* Added compatibility witn the the GTK and TK toolkits, although with no GUI
features.
* It is now possible to run HyperSpy in a headless system.
* Added a CLI to ``hyperspy.signal.Signal1DTools.remove_background``.
* New ``hyperspy.signal.Signal1DTools.estimate_peak_width`` method to estimate
peak width.
* New methods to integrate over one axis:
``hyperspy.signal.Signal.integrate1D`` and
``hyperspy.signal.Signal1DTools.integrate_in_range``.
* New ``hyperspy.signal.Signal.metadata`` attribute, ``Signal.binned``. Several
methods behave differently on binned and unbinned signals.
See :ref:`signal.binned`.
* New ``hyperspy.signal.Signal.map`` method to easily transform the
data using a function that operates on individual signals. See
:ref:`signal.iterator`.
* New ``hyperspy.signal.Signal.get_histogram`` and
``hyperspy.signal.Signal.print_summary_statistics`` methods.
* The spikes removal tool has been moved to the :class:`~.api.signals.Signal1D`
class so that it is available for all its subclasses.
* The ``hyperspy.signal.Signal.split method now can automatically split back
stacked signals into its original part. See :ref:`signal.stack_split`.

IO
^^

* Improved support for FEI's emi and ser files.
* Improved support for Gatan's dm3 files.
* Add support for reading Gatan's dm4 files.

Plotting
^^^^^^^^

* Use the blitting capabilities of the different toolkits to
speed up the plotting of images.
* Added several extra options to the Signal ``hyperspy.signal.Signal.plot``
method to customize the navigator. See :ref:`visualization-label`.
* Add compatibility with IPython's matplotlib inline plotting.
* New function, :py:func:`~.api.plot.plot_spectra`, to plot several
spectra in the same figure. See :ref:`plot.spectra`.
* New function, :py:func:`~.api.plot.plot_signals`, to plot several
signals at the same time. See :ref:`plot.signals`.
* New function, :py:func:`~.api.plot.plot_histograms`, to plot the histrograms
of several signals at the same time. See :ref:`plot.signals`.

Curve fitting
^^^^^^^^^^^^^

* The chi-squared, reduced chi-squared and the degrees of freedom are
computed automatically when fitting. See :ref:`model.fitting`.
* New functionality to plot the individual components of a model. See
:ref:`model.visualization`.
* New method, ``hyperspy.model.Model.fit_component``, to help setting the
starting parameters. See :ref:`model.starting`.

Machine learning
^^^^^^^^^^^^^^^^

* The PCA scree plot can now be easily obtained as a Signal. See
:ref:`mva.scree_plot`.
* The decomposition and blind source separation components can now be obtained
as ``hyperspy.signal.Signal`` instances. See :ref:`mva.get_results`.
* New methods to plot the decomposition and blind source separation results
that support n-dimensional loadings. See :ref:`mva.visualization`.

Dielectric function
^^^^^^^^^^^^^^^^^^^

* New ``hyperspy.signal.Signal`` subclass,
``hyperspy._signals.dielectric_function.DielectricFunction``.

EELS
^^^^

* New method,
``hyperspy._signals.eels.EELSSpectrum.kramers_kronig_analysis`` to calculate
the dielectric function from low-loss electron energy-loss spectra based on
the Kramers-Kronig relations. See :external+exspy:ref:`eels.kk`.
* New method to align the zero-loss peak,
``hyperspy._signals.eels.EELSSpectrum.align_zero_loss_peak``.

EDS
^^^

* New signal, EDSSpectrum especialized in EDS data analysis, with subsignal
for EDS with SEM and with TEM: EDSSEMSpectrum and EDSTEMSpectrum. See
:external+exspy:ref:`eds-label`.
* New database of EDS lines available in the ``elements`` attribute of the
``hspy.utils.material`` module.
* Adapted methods to calibrate the spectrum, the detector and the microscope.
See :external+exspy:ref:`eds_calibration-label`.
* Specific methods to describe the sample,
``hyperspy._signals.eds.EDSSpectrum.add_elements`` and
``hyperspy._signals.eds.EDSSpectrum.add_lines``. See :external+exspy:ref:`eds_sample-label`
* New method to get the intensity of specific X-ray lines:
``hyperspy._signals.eds.EDSSpectrum.get_lines_intensity``. See
:external+exspy:ref:`eds_sample-label`

API changes
-----------

* hyperspy.misc has been reorganized. Most of the functions in misc.utils has
been rellocated to specialized modules. misc.utils is no longer imported in
``hyperspy.hspy``. A new ``hyperspy.utils`` module is imported instead.
* Objects that have been renamed

+ ``hspy.elements`` -> ``utils.material.elements``.
+ ``Signal.navigation_indexer`` -> ``inav``.
+ ``Signal.signal_indexer`` -> ``isig``.
+ ``Signal.mapped_parameters`` -> ``Signal.metadata``.
+ ``Signal.original_parameters`` -> ``Signal.original_metadata``.
* The metadata has been reorganized. See :ref:`metadata_structure`.
* The following signal methods now operate out-of-place:

+ ``hyperspy.signal.Signal.swap_axes``
+ ``hyperspy.signal.Signal.rebin``

.. _changes_0.6:

===================

New features
------------

* Signal now supports indexing and slicing. See :ref:`signal.indexing`.
* Most arithmetic and rich arithmetic operators work with signal.
See :ref:`signal.operations`.
* Much improved EELSSpectrum methods:
``hyperspy._signals.eels.EELSSpectrum.estimate_zero_loss_peak_centre``,
``hyperspy._signals.eels.EELSSpectrum.estimate_elastic_scattering_intensity`` and
``hyperspy._signals.eels.EELSSpectrum.estimate_elastic_scattering_threshold``.

* The axes can now be given using their name e.g. ``s.crop("x", 1,10)``
* New syntax to specify position over axes: an integer specifies the indexes
over the axis and a floating number specifies the position in the axis units
e.g. ``s.crop("x", 1, 10.)`` crops over the axis `x` (in meters) from index 1
to value 10 meters. Note that this may make your old scripts behave in
unexpected ways as just renaming the old \*_in_units and \*_in_values methods
won't work in most cases.
* Most methods now use the natural order i.e. X,Y,Z.. to index the axes.
* Add padding to fourier-log and fourier-ratio deconvolution to fix the
wrap-around problem and increase its performance.
* New
``hyperspy.components.eels_cl_edge.EELSCLEdge.get_fine_structure_as_spectrum``
EELSCLEdge method.
* New ``hyperspy.components.arctan.Arctan`` model component.
* New
``hyperspy.model.Model.enable_adjust_position``
and ``hyperspy.model.Model.disable_adjust_position``
to easily change the position of components using the mouse on the plot.
* New Model methods
``hyperspy.model.Model.set_parameters_value``,
``hyperspy.model.Model.set_parameters_free`` and
``hyperspy.model.Model.set_parameters_not_free``
to easily set several important component attributes of a list of components
at once.
* New
:py:func:`~.api.stack` function to stack signals.
* New Signal methods:
``hyperspy.signal.Signal.integrate_simpson``,
``hyperspy.signal.Signal.max``,
``hyperspy.signal.Signal.min``,
``hyperspy.signal.Signal.var``, and
``hyperspy.signal.Signal.std``.
* New sliders window to easily navigate signals with navigation_dimension > 2.
* The Ripple (rpl) reader can now read rpl files produced by INCA.

API changes
-----------
* The following functions has been renamed or removed:

* components.EELSCLEdge

* knots_factor -> fine_structure_smoothing
* edge_position -> onset_energy
* energy_shift removed

* components.Voigt.origin -> centre
* signals.Signal1D

* find_peaks_1D -> Signal.find_peaks1D_ohaver
* align_1D -> Signal.align1D
* shift_1D -> Signal.shift1D
* interpolate_1D -> Signal.interpolate1D

* signals.Signal2D.estimate_2D_translation -> Signal.estimate_shift2D
* Signal

* split_in -> split
* crop_in_units -> crop
* crop_in_pixels -> crop


* Change syntax to create Signal objects. Instead of a dictionary
Signal.__init__ takes keywords e.g with a new syntax .
``>>> s = signals.Signal1D(np.arange(10))`` instead of
``>>> s = signals.Signal1D({'data' : np.arange(10)})``



.. _changes_0.5.1:

===================

New features
------------
* New Signal method `get_current_signal` proposed by magnunor.
* New Signal `save` method keyword `extension` to easily change the saving format while keeping the same file name.
* New EELSSpectrum methods: estimate_elastic_scattering_intensity, fourier_ratio_deconvolution, richardson_lucy_deconvolution, power_law_extrapolation.
* New Signal1D method: hanning_taper.



Major bugs fixed
----------------
* The `print_current_values` Model method was raising errors when fine structure was enabled or when only_free = False.
* The `load` function `signal_type` keyword was not passed to the readers.
* The spikes removal tool was unable to find the next spikes when the spike was detected close to the limits of the spectrum.
* `load` was raising an UnicodeError when the title contained non-ASCII characters.
* In Windows `HyperSpy Here` was opening in the current folder, not in the selected folder.
* The fine structure coefficients were overwritten with their std when charging values from the model.
* Storing the parameters in the maps and all the related functionality was broken for 1D spectrum.
* Remove_background was broken for 1D spectrum.




API changes
-----------
* EELSSpectrum.find_low_loss_centre was renamed to estimate_zero_loss_peak_centre.
* EELSSpectrum.calculate_FWHM was renamed to estimate_FWHM.

.. _changes_0.5:

===================

New features
------------
* The documentation was thoroughly revised, courtesy of M. Walls.
* New user interface to remove spikes from EELS spectra.
* New align2D signals.Signal2D method to align image stacks.
* When loading image files, the data are now automatically converted to
grayscale when all the color channels are equal.
* Add the possibility to load a stack memory mapped (similar to ImageJ
virtual stack).
* Improved hyperspy starter script that now includes the possibility
to start HyperSpy in the new IPython notebook.
* Add "HyperSpy notebook here" to the Windows context menu.
* The information displayed in the plots produced by Signal.plot have
been enhanced.
* Added Egerton's sigmak3 and sigmal3 GOS calculations (translated
from matlab by I. Iyengar) to the EELS core loss component.
* A browsable dictionary containing the chemical elements and
their onset energies is now available in the user namespace under
the variable name `elements`.
* The ripple file format now supports storing the beam energy, the collection and the convergence angle.


Major bugs fixed
----------------
* The EELS core loss component had a bug in the calculation of the
relativistic gamma that produced a gamma that was always
approximately zero. As a consequence the GOS calculation was wrong,
especially for high beam energies.
* Loading msa files was broken when running on Python 2.7.2 and newer.
* Saving images to rpl format was broken.
* Performing BSS on data decomposed with poissonian noise normalization
was failing when some columns or rows of the unfolded data were zero,
what occurs often in EDX data for example.
* Importing some versions of scikits learn was broken
* The progress bar was not working properly in the new IPython notebook.
* The constrast of the image was not automatically updated.

API changes
-----------
* spatial_mask was renamed to navigation_mask.
* Signal1D and Signal2D are not loaded into the user namespace by default.
The signals module is loaded instead.
* Change the default BSS algorithm to sklearn fastica, that is now
distributed with HyperSpy and used in case that sklearn is not
installed e.g. when using EPDFree.
* _slicing_axes was renamed to signal_axes.
* _non_slicing_axes to navigation_axes.
* All the Model \*_in_pixels methods were renamed to to _*_in_pixel.
* EELSCLEdge.fs_state was renamed to fine_structure_active.
* EELSCLEdge.fslist was renamed to fine_structure_coeff.
* EELSCLEdge.fs_emax was renamed to fine_structure_width.
* EELSCLEdge.freedelta was renamed to free_energy_shift.
* EELSCLEdge.delta was renamed to energy_shift.
* A value of True in a mask now means that the item is masked all over
HyperSpy.


.. _changes_0.4.1:

===================

New features
------------

* Added TIFF 16, 32 and 64 bits support by using (and distributing) Christoph Gohlke's `tifffile library <https://pypi.org/project/tifffile/>`_.
* Improved UTF8 support.
* Reduce the number of required libraries by making mdp and hdf5 not mandatory.
* Improve the information returned by __repr__ of several objects.
* DictionaryBrowser now has an export method, i.e. mapped parameters and original_parameters can be exported.
* New _id_name attribute for Components and Parameters. Improvements in their __repr__ methods.
* Component.name can now be overwriten by the user.
* New Signal.__str__ method.
* Include HyperSpy in The Python Package Index.


Bugs fixed
----------
* Non-ascii characters breaking IO and print features fixed.
* Loading of multiple files at once using wildcards fixed.
* Remove broken hyperspy-gui script.
* Remove unmantained and broken 2D peak finding and analysis features.

Syntax changes
--------------
* In EELS automatic background feature creates a PowerLaw component, adds it to the model an add it to a variable in the user namespace. The variable has been renamed from `bg` to `background`.
* pes_gaussian Component renamed to pes_core_line_shape.

.. _changes_0.4:

===================

New features
------------
* Add a slider to the filter ui.
* Add auto_replot to sum.
* Add butterworth filter.
* Added centring and auto_transpose to the svd_pca algorithm.
* Keep the mva_results information when changing the signal type.
* Added sparse_pca and mini_batch_sparse_pca to decomposition algorithms.
* Added TV to the smoothing algorithms available in BSS.
* Added whitening to the mdp ICA preprocessing.
* Add explained_variance_ratio.
* Improvements in saving/loading mva data.
* Add option to perform ICA on the scores.
* Add orthomax FA algorithm.
* Add plot methods to Component and Parameter.
* Add plot_results to Model.
* Add possibility to export the decomposition and bss results to a folder.
* Add Signal method `change_dtype`.
* Add the possibility to pass extra parameters to the ICA algorithm.
* Add the possibility to reproject the data after a decomposition.
* Add warning when decomposing a non-float signal.
* adds a method to get the PCs as a Signal1D object and adds smoothing to the ICA preprocessing.
* Add the possibility to select the energy range in which to perform spike removal operations.
* the smoothings guis now offer differentiation and line color option. Smoothing now does not require a gui.
* Fix reverse_ic which was not reversing the scores and improve the autoreversing method.
* Avoid cropping when is not needed.
* Changed criteria to reverse the ICs.
* Changed nonans default to False for plotting.
* Change the whitening algorithm to a svd based one and add sklearn fastica algorithm.
* Clean the ummixing info after a new decomposition.
* Increase the chances that similar independent components will have the same indexes.
* Make savitzky-golay smoothing work without raising figures.
* Make plot_decomposition* plot only the number of factors/scores determined by output_dimension.
* make the Parameter __repr__ method print its name.
* New contrast adjustment tool.
* New export method for Model, Component and Parameter.
* New Model method: print_current_values.
* New signal, spectrum_simulation.
* New smoothing algorithm: total variance denoising.
* Plotting the components in the same or separate windows is now configurable in the preferences.
* Plotting the spikes is now optional.
* Return an error message when the decomposition algorithm is not recognised.
* Store the masks in mva_results.
* The free parameters are now automically updated on chaning the free attribute.

Bugs fixed
----------
* Added missing keywords to plot_pca_factors and plot_ica_factors.
* renamed incorrectly named exportPca and exportIca functions.
* an error was raised when calling generate_data_from_model.
* a signal with containing nans was failing to plot.
* attempting to use any decomposition plotting method after loading with mva_results.load was raising an error.
* a typo was causing in error in pca when normalize_variance = True.
* a typo was raising an error when cropping the decomposition dimension.
* commit 5ff3798105d6 made decomposition and other methods raise an error.
* BUG-FIXED: the decomposition centering index was wrong.
* ensure_directory was failing for the current directory.
* model data forced to be 3D unnecessarily.
* non declared variable was raising an error.
* plot naming for peak char factor plots were messed up.
* plot_RGB was broken.
* plot_scores_2D was using the transpose of the shape to reshape the scores.
* remove background was raising an error when the navigation dimension was 0.
* saving the scores was sometimes transposing the shape.
* selecting indexes while using the learning export functions was raising an error.
* the calibrate ui was calculating wrongly the calibration the first time that Apply was pressed.
* the offset estimation was summing instead of averaging.
* the plot_explained_variance_ratio was actually plotting the cumulative, renamed.
* the signal mask in decomposition and ica was not being raveled.
* the slice attribute was not correctly set at init in some scenarios.
* the smoothing and calibrabrion UIs were freezing when the plots where closed before closing the UI window.
* to_spectrum was transposing the navigation dimension.
* variance2one was operating in the wrong axis.
* when closing the plots of a model, the UI object was not being destroyed.
* when plotting an image the title was not displayed.
* when the axis size was changed (e.g. after cropping) the set_signal_dimension method was not being called.
* when using transform the data was being centered and the resulting scores were wrong.

Syntax changes
--------------

* in decomposition V rename to explained_variance.
* In FixedPattern, default interpolation changed to linear.
* Line and parabole components deleted + improvements in the docstrings.
* pca_V = variance.
* mva_result renamed to learning_results.
* pca renamed to decomposition.
* pca_v and mva_results.v renamed to scores pc renamed to factors .
pca_build_SI renamed to get_pca_model ica_build_SI renamed to get_ica_model.
* plot_explained_variance renamed to plot_explained_variance_ratio.
* principal_components_analysis renamed to decomposition.
* rename eels_simulation to eels_spectrum_simulation.
* Rename the output parameter of svd_pca and add scores.
* Replace plot_lev by plot_explained_variance_ratio.
* Scores renamed to loadings.
* slice_bool renamed to navigate to make its function more explicit.
* smoothing renamed to pretreatment and butter added.
* variance2one renamed to normalize_variance.
* w renamed to unmixing matrix and fixes a bug when loading a mva_result
in which output_dimension = None.
* ubshells are again availabe in the interactive session.
* Several changes to the interface.
* The documentation was updated to reflex the last changes.
* The microscopes.csv file was updated so it no longer contains the
Orsay VG parameters.