Pysam

comes with additional capabilities for remote file access which depend
on the presence of optional system libraries. As a consequence, the
installation script :file:`setup.py` has become more complex. For an
overview, see :ref:`installation`. We have tested installation on
linux and OS X, but could not capture all variations. It is possible
that a 0.9.1 release might follow soon addressing installation issues.

The :py:class:`~.pysam.VariantFile` class provides access to
:term:`vcf` and :term:`bcf` formatted files. The class is certainly
usable and interface is reaching completion, but the API and the
functionality is subject to change.

Detailed release notes
----------------------

* upgrade to htslib 1.3
* python 3 compatibility tested throughout.
* added a first set of bcftools commands in the pysam.bcftools
submodule.
* samtools commands are now in the pysam.samtools module. For
backwards compatibility they are still imported into the pysam
namespace.
* samtools/bcftools return stdout as a single (byte) string. As output
can be binary (VCF.gz, BAM) this is necessary to ensure py2/py3
compatibility. To replicate the previous behaviour in py2.7, use::

pysam.samtools.view(self.filename).splitlines(True)

* get_tags() returns the tag type as a character, not an integer (214)
* TabixFile now raises ValueError on indices created by tabix <1.0 (206)
* improve OSX installation and develop mode
* FastxIterator now handles empty sequences (204)
* TabixFile.isremote is not TabixFile.is_remote in line with AlignmentFile
* AlignmentFile.count() has extra optional argument read_callback
* setup.py has been changed to:
* install a single builtin htslib library. Previously, each pysam
module contained its own version. This reduces compilation time
and code bloat.
* run configure for the builtin htslib library in order to detect
optional libraries such as libcurl. Configure behaviour can be
controlled by setting the environment variable
HTSLIB_CONFIGURE_OPTIONS.
* get_reference_sequence() now returns the reference sequence and not
something looking like it. This bug had effects on
get_aligned_pairs(with_seq=True), see 225. If you have relied on on
get_aligned_pairs(with_seq=True) in pysam-0.8.4, please check your
results.
* improved autodetection of file formats in AlignmentFile and VariantFile.

==============

.. rubric:: 5 October 2023

This pysam release wraps htslib/samtools/bcftools 1.18 (PR 1208).

It has been tested with Python versions 3.6 through 3.12, and wheels are
available via pypi_ for all of those Python versions. Python versions 3.6
and 3.7 are end-of-life; particularly if you use pysam with either of
these versions, please vote in the version survey at issue 1230.

The final pysam release that supported Python 2.7 was v0.20.0.

Bugs fixed:

* Remove Cython from runtime dependencies (PR 1186, thanks to Nicola Soranzo,
also reported by Arya Massarat in PR 1194)

* Miscellaneous dependency improvements (PR 1216, 1217, PR 1218, PR 1219,
thanks to Martin Larralde and Arthur Vigil)

* Suppress spurious "Could not retrieve index file" message when opening an
AlignmentFile (939, 1214, reported by ChengYong Tham and Sebastian Röner)

* Propagate SAM parsing errors encounted in :meth:`.AlignedSegment.fromstring`
(1196, reported by DV Klopfenstein)

* Accept invalid MD:A tagged fields produced by HTSeq instead of crashing
in :meth:`AlignedSegment.get_aligned_pairs(with_seq=True)
<.AlignedSegment.get_aligned_pairs>` (1226, reported by Isaac Vock)

* Fix multiarch macOS CI builds by removing brewed liblzma (1205, reported
by Till Hartmann)

* Fix :attr:`.VariantRecordSample.alleles` type hint (1179, reported by
David Seifert)

New functionality:

* Add optional :meth:`HTSFile.seek(..., whence) <.HTSFile.seek>` parameter
and clarify which functions use libc.SEEK_SET vs io.SEEK_SET
(1185, requested by luyulin)

* File handling improvements in samtools & bcftools commands (should improve
1193 and 1195, reported by Rob Bierman and Sam Chorlton)

* Improve :class:`.FastxFile` performance (PR 1227, thanks to Fabian Klötzl
and Valentyn Bezshapkin)

* Improve the accuracy of type hints for :class:`.AlignmentFile` iteration
(1184, PR 1189, reported by PikalaxALT)

Documentation improvements:

* Clarify that :meth:`.AlignedSegment.get_aligned_pairs` results are 0-based
(1180, reported by Nick Semenkovich)

* Clarify :meth:`.AlignedSegment.get_reference_positions` documentation
(836, 838, reported by Liang Ou and Nick Stoler)

* Clarify that installation via pip usually uses a wheel, and that configuring
the build via $HTSLIB_CONFIGURE_OPTIONS etc only applies when installing from
an sdist (1086, reported by Layne Sadler)

A message from pysam's founder, Andreas Heger:

As many of you will have noticed, John Marshall has been effectively
maintaining pysam and supporting users over the last few years.
I, Andreas, am very grateful for the countless hours he has contributed.
Unfortunately, I will not be able to contribute much in the near and
intermediate future. To keep pysam going, John has kindly agreed to
continue maintaining and supporting pysam as the principal developer
of pysam. I am very happy to know that pysam is in good hands and want
to thank again John and the wider pysam community for their suggestions,
bug reports, code contributions and general support.

Thank you Andreas for all your work over the years and the solid foundations
that pysam enjoys and the useful functionality it provides.

==============

.. rubric:: 2 April 2023

This release wraps htslib/samtools/bcftools version 1.17.

Pysam is now compatible with Python 3.11. We have removed Python 2.x
support. Pysam is tested with Python versions 3.6 to 3.11.

* [1175] VariantHeader.new_record: set start/stop before alleles
* [1173] Add multiple build improvements in htscodecs on multi-arch macOS
* [1148] Ignore CIGAR-less reads in find_introns.
* [1172] Add new `samtools cram-size` and `samtools reset` commands
* [1169] Fix CRAM index-related crash when using the musl C standard library.
* [1168] Add a minimal pyproject.toml for PEP517.
* [1158] Fix type hints and add FastqProxy type hints.
* [1147] Py3.11 compatibility, get shared object suffix from EXT_SUFFIX.
* [1143] Add mypy symbols for samtools and bcftools.
* [1155] Fix pysam.index() when using recent `samtools index` options.
* [1151] Test suite py3.11 compatibility, work around failing test case.
* [1149] MacOS universal build compatibility.
* [1146] Fix build when CFLAGS/etc environment variables are set.

==============

.. rubric:: 29 October 2022

This release wraps htslib/bcftools version 1.16 and samtools version 1.16.1.

* [1113] Full compatibility with setuptools v62.1.0's build directory name changes
* [1121] Build-time symbol check portability improved
* [1122] Fix setting sample genotype using .alleles property
* [1128] Fix test suite failure when using a libdeflate-enabled samtools

Many additional type hints have been provided by the community,
thanks!

==============

.. rubric:: 27 May 2022

This release wraps htslib/samtools/bcftools version 1.15.1.

* [1104] add an add_samples() method to quickly add multiple samples
to VCF.

==============

.. rubric:: 30 March 2022

This release wraps htslib/samtools/bcftools version 1.15.

* [1085] Improve getopt()/getopt_long() resetting when running samtools/bcftools commands

* [1078] Support BAM_CPAD in get_aligned_pairs

* [1063] Run flake8 and fix some linting issues

* [1088] Add AlignedSegment is_mapped/mate_is_mapped/is_forward/mate_is_forward properties

* Write an absent AlignedSegment.qual as all-bytes-0xff

* Fix BGZFile.read() behaviour near or at EOF

* First API for the htslib modified bases interface