Pyteomics

---

- mzML and mzIdentML parsers can now create an index of element offsets.
This allows quick random access to elements by unique ID.

- mzML parsers now come in two flavors: :py:class:`pyteomics.mzml.MzML` and
:py:class:`pyteomics.mzml.PreIndexedMzML`. The latter uses the byte offsets
listed at the end of the file.

- New parameters `convert_arrays` and `read_charges` in :py:func:`mgf.read`
allow using it without :py:mod:`numpy` and possibly improve performance.
The default behavior is retained.

- Performance optimizations in :py:func:`mgf.read` and :py:func:`parser.cleave`.

- New decoy generation mode called "fused decoy", described in the `paper accepted to JASMS
<http://dx.doi.org/10.1007/s13361-016-1436-7>`_.

API changes
...........

- :py:func:`pyteomics.parser.cleave` no longer accepts the `labels` argument.
It is emphasized that the input sequences are expected to be in plain one-letter
notation, but no checks are performed.

- :py:func:`DataFrame` functions in :py:mod:`pepxml` and :py:mod:`tandem` now
extract more protein-related information. The list-like protein-related values
can be reported as lists or packed into strings, depending on the optional
paramter `sep`. Some column names have changed as a result.

- Call signatures of :py:func:`pyteomics.fasta.decoy_sequence` and the functions using it
are slightly changed. Standard modes are now also exposed as individual functions.

---

New submodule :py:mod:`pyteomics.mass.unimod` contains rewritten machinery
for handling of Unimod relational databases (contributed by Joshua Klein).
This is a substitution and extension for the old :py:class:`mass.Unimod` class.
:py:mod:`pyteomics.mass.unimod` requires SQLAlchemy.

Other changes:

- New function :py:func:`pyteomics.auxiliary.linear_regression_perpendicular`
provides a linear fit minimizing distances from data points to the fit line
(as opposed to :py:func:`pyteomics.auxiliary.linear_regression`, which
minimizes vertical distances).

- Both new and old linear regression functions now accept a single array of shape (N, 2).

- :py:func:`pyteomics.pylab_aux.scatter_trend` now has an optional parameter
`regression` which can be a callable performing the regression.
Also, the regression equation is now the label of the regression line, not
the scatter plot.

- Another two new parameters for :py:func:`pyteomics.pylab_aux.scatter_trend`
are `sigma_kwargs` and `sigma_values`.

- :py:mod:`pyteomics.pylab_aux` functions :py:func:`plot_line` and
:py:func:`scatter_trend` now return the objects they create.

- Writer functions (:py:func:`pyteomics.mgf.write`, :py:func:`pyteomics.fasta.write`,
:py:func:`pyteomics.fasta.write_decoy_db`) now accept a `file_mode` argument that
overrides the mode in which the file is opened.

- In :py:func:`pyteomics.mgf.write` one can now override the format spec for fragment m/z,
intensity and charge values using the optinal `fragment_format` argument. Key order and
key-value parameter formatters are now also handled via optional arguments.

- :py:func:`pyteomics.fasta.decoy_db` now supports `ignore_comments` and `parser` arguments.

-----

- Bugfix in :py:mod:`pyteomics.auxiliary`.

- New parameter `show_legend` in :py:func:`pyteomics.pylab_aux.scatter_trend`.

- Performance improvements in :py:mod:`pyteomics.parser`.

---

This release offers integration with the great :py:mod:`pandas` library.
Working with :py:func:`qvalues` and :py:func:`!filter` functions
is now much easier if you have your PSMs in a :py:class:`DataFrame`.
Many search engines use CSV as their output format, allowing direct
creation of :py:class:`DataFrame` objects. New functions
:py:func:`pyteomics.tandem.DataFrame` and :py:func:`pyteomics.pepxml.DataFrame`
faciliatate creation of DataFrames from corresponding formats.

Also, :py:func:`qvalues`, :py:func:`!filter` and :py:func:`fdr` functions can now use
posterior error probabilities (PEPs) instead of using decoys for q-value calculation.

- In :py:func:`qvalues` and :py:func:`!filter` functions,
`key` and `is_decoy` can now be array-like objects or strings
(as well as functions and iterators).
If a string is given, it is used as a field name in the PSM array
or :py:class:`DataFrame`. :py:func:`fdr` functions also support strings
and iterables as arguments.

- New parameter `pep` in :py:func:`qvalues`, :py:func:`!filter` and :py:func:`fdr` functions.
It can be callable, array-like, or iterator. Conflicts with decoy-related
parameters. Compatible with `key`, but makes it optional.

- Fixed the behavior of :py:func:`filter.chain` functions. They now treat
the `full_output` argument the same way as :py:func:`!filter` functions.

- Fixed the issue that caused exceptions when calling :py:func:`fasta.decoy_db`
and :py:func:`fasta.write_decoy_db` with explicitly given `mode` (signature
for creation of :py:class:`pyteomics.auxiliary.FileReader` objects slightly changed).

- Pyteomics now uses `setuptools` and is a namespace package.

- Minor fixes.


API changes
...........

- Default value of `remove_decoy` in :py:func:`qvalues` is now :py:const:`False`.

-----

- Added `legend_kwargs` as a keyword argument to
:py:func:`pyteomics.pylab_aux.scatter_trend`.

- Minor fixes.

-----

- XML parsers are now implemented as objects, each format has its own class.
Those classes can be instantiated using the same arguments as :py:func:`read`
functions accepted, and support direct iteration and the ``with`` syntax.
The :py:func:`read` functions are now simple aliases to the corresponding
constructors.

- As a result, functions :py:func:`iterfind`, :py:func:`version_info` and
:py:func:`get_by_id` functions are now deprecated in favor of methods
:py:meth:`iterfind` and :py:meth:`get_by_id` and attribute
:py:attr:`version_info` of corresponding instances.

- In :py:func:`pyteomics.mgf.write`, the order of keys and the format of values
are now controlled via module-level variables.

- In :py:mod:`pyteomics.electrochem`, correction for pK of terminal groups
depending on the terminal residue is implemented; example set of pK and
corrected pK added.

- Imports of external dependencies are delayed where possible, so that
unnecessary :py:exc:`ImportErrors` do not occur.

- :py:func:`local_fdr` renamed to :py:func:`qvalues` in :py:mod:`pepxml`,
:py:mod:`mzid`, :py:mod:`tandem` and :py:mod:`auxiliary`.
:py:func:`local_fdr` did not reflect the semantics of the function.
The algorithm has been also corrected so that the array of q-values
is always sorted (as it should be by definition).

- :py:func:`qvalues` now also accepts a parameter `full_output` which keeps the
PSMs alongside their scores and associated q-values.

- All :py:func:`fdr`, :py:func:`qvalues`, and :py:func:`!filter` functions
now accept a new parameter `correction`. It is used for more accurate
estimation of the number of false positives using TDA (`paper with explanation
<http://dx.doi.org/10.1021/acs.jproteome.6b00144>`_).

- :py:func:`!filter` functions now support both iterator protocol and context
manager protocol. They now also accept the `full_output` parameter, which has
the following meaning: if :py:const:`True` (default), then an array of PSMs
is directly returned by the function. Otherwise, an iterator is returned, as
before. The array takes some memory, but this way is usually around 2x faster.

- New function :py:func:`pyteomics.pylab_aux.plot_qvalue_curve`.

- :py:class:`pyteomics.mass.Composition` objects now have a :py:meth:`mass`
method (equivalent to :py:func:`pyteomics.mass.calculate_mass`.

- Also, :py:class:`Composition` and objects returned by
:py:func:`pyteomics.parser.amino_acid_composition` now inherit from
:py:class:`collections.defaultdict` **and** :py:class:`collections.Counter`.

- Decoy-related functions in :py:mod:`pyteomics.fasta` now accept a new parameter
`keep_nterm` that preserves the N-terminal residue in the generated decoy
sequences.

- Minor fixes.

API changes
...........

- In :py:func:`pyteomics.pylab_aux.scatter_trend`, keyword arguments for
:py:func:`pylab.scatter` and :py:func:`pylab.plot` are now accepted as dicts
`scatter_kwargs` and `plot_kwargs`. Keyword argument `alpha` is now not
accepted and should be put in the appropriate dict.
- In :py:func:`pyteomics.pylab_aux.plot_function_3d` and
:py:func:`pyteomics.pylab_aux.plot_function_contour`, arbitrary kwargs can
now also be passed to the plotting function.
- :py:func:`!filter` functions do not support context manager protocol by
default. To keep using them as iterators / context managers, specify
``full_output=False`` (see above for details).