v0.22.1

This bug fix release wraps htslib/samtools/bcftools 1.18.

Bugs fixed:

• Preserve all header field tags defined in the SAM specification (notably TP) in AlignmentHeader.from_dict() and AlignmentHeader.to_dict() (#1237, PR #1238, thanks to @tfenne and @nh13)

• Adjust HTSlib's Makefile so that make distclean no longer tries to rebuild the htscodecs configury (PR #1247, reported by @nsoranzo)

• Reinstate S3 support in pre-built Linux wheels: support for this protocol was inadvertently omitted from the pre-built 0.22.0 wheels on Linux (#1249, #1277, etc varying circumstances; likely it is this that was reported by @Mathew-B-SDGS, @BenjaminSargsyan, et al)

• Add missing AlignedSegment.is_mapped etc properties to type stubs (PR #1273, thanks to @msto)

• Fix off-by-one NamedTupleProxy, asBed, etc array bounds check (#1279, reported by @dbolser)

• Make pysam's klib headers compatible with C++ (reported by @martin-g)

v0.22.0

This pysam release wraps htslib/samtools/bcftools 1.18 (PR #1208).

It has been tested with Python versions 3.6 through 3.12, and wheels are available via PyPI for all of those Python versions. Python versions 3.6 and 3.7 are end-of-life; particularly if you use pysam with either of these versions, please vote in the version survey at #1230.

The final pysam release that supported Python 2.7 was v0.20.0.

Bugs fixed:

• Remove Cython from runtime dependencies (PR #1186, thanks to @nsoranzo, also reported by @aryarm in PR #1194)

• Miscellaneous dependency improvements (PR #1216, #1217, PR #1218, PR #1219, thanks to @althonos and @ahvigil)

• Suppress spurious “Could not retrieve index file” message when opening an AlignmentFile (#939, #1214, reported by @cytham and @sroener)

• Propagate SAM parsing errors encounted in AlignedSegment.fromstring() (#1196, reported by @dvklopfenstein)

• Accept invalid MD:A tagged fields produced by HTSeq instead of crashing in get_aligned_pairs(with_seq=True) (#1226, reported by @isaacvock)

• Fix multiarch macOS CI builds by removing brewed liblzma (#1205, reported by @tedil)

• Fix VariantRecordSample.alleles type hint (#1179, reported by @SoapGentoo)

New functionality:

• Add optional HTSFile.seek(..., whence) parameter and clarify which functions use libc.SEEK_SET vs io.SEEK_SET (#1185, requested by @ylluX)

• File handling improvements in samtools & bcftools commands (should improve #1193 and #1195, reported by @biermanr and @schorlton)

• Improve FastxFile performance (PR #1227, thanks to @kloetzl and @valentynbez)

• Improve the accuracy of type hints for AlignmentFile iteration (#1184, PR #1189, reported by @PikalaxALT)

Documentation improvements:

• Clarify that AlignedSegment.get_aligned_pairs() results are 0-based (#1180, reported by @semenko)

• Clarify AlignedSegment.get_reference_positions() documentation (#836, #838, reported by @weir12 and @NickSto)

• Clarify that installation via pip usually uses a wheel, and that configuring the build via $HTSLIB_CONFIGURE_OPTIONS etc only applies when installing from an sdist (#1086, reported by @aiqc)

A message from pysam's founder, @AndreasHeger:

As many of you will have noticed, John Marshall has been effectively maintaining pysam and supporting users over the last few years. I, Andreas, am very grateful for the countless hours he has contributed. Unfortunately, I will not be able to contribute much in the near and intermediate future. To keep pysam going, John has kindly agreed to continue maintaining and supporting pysam as the principal developer of pysam. I am very happy to know that pysam is in good hands and want to thank again John and the wider pysam community for their suggestions, bug reports, code contributions and general support.

Thank you Andreas for all your work over the years and the solid foundations that pysam enjoys and the useful functionality it provides.

pysam-v0.21.0

Release 0.21.0

This release wraps htslib/samtools/bcftools version 1.17.

Pysam is now compatible with Python 3.11. We have removed python 2.x
support. Pysam is tested with python versions 3.6 to 3.11.

• [#1175] VariantHeader.new_record: set start/stop before alleles
• [#1173] Add multiple build improvements in htscodecs on multi-arch macOS
• [#1148] Ignore CIGAR-less reads in find_introns.
• [#1172] Add new samtools cram-size and samtools reset commands
• [#1169] Fix CRAM index-related crash when using the musl C standard library.
• [#1168] Add a minimal pyproject.toml for PEP517.
• [#1158] Fix type hints and add FastqProxy type hints.
• [#1147] Py3.11 compatibility, get shared object suffix from EXT_SUFFIX.
• [#1143] Add mypy symbols for samtools and bcftools.
• [#1155] Fix pysam.index() when using recent samtools index options.
• [#1151] Test suite py3.11 compatibility, work around failing test case.
• [#1149] MacOS universal build compatibility.
• [#1146] Fix build when CFLAGS/etc environment variables are set.

Big thanks to @jmarshall who again has done most of the heavy lifting for this release.

pysam-v0.20.0

Release 0.20.0

This release wraps htslib/bcftools version 1.16 and samtools version 1.16.1.

• [#1113] Full compatibility with setuptools v62.1.0's build directory name changes
• [#1121] Build-time symbol check portability improved
• [#1122] Fix setting sample genotype using .alleles property
• [#1128] Fix test suite failure when using a libdeflate-enabled samtools

Many additional type hints have been provided by the community,
thanks!

This will be the last release with python2.7 support.

pysam-v0.19.1

Release 0.19.1

This release wraps htslib/samtools/bcftools version 1.15.1.

• [#1104] add an add_samples() method to quickly add multiple samples
  to VCF.

Wheels are now built using manylinux_2_24

pysam-v0.19.0

Release 0.19.0

This release wraps htslib/samtools/bcftools version 1.15.

• [#1085] Improve getopt()/getopt_long() resetting when running samtools/bcftools commands

• [#1078] Support BAM_CPAD in get_aligned_pairs

• [#1063] Run flake8 and fix some linting issues

• [#1088] Add AlignedSegment is_mapped/mate_is_mapped/is_forward/mate_is_forward properties

• Write an absent AlignedSegment.qual as all-bytes-0xff

• Fix BGZFile.read() behaviour near or at EOF

• First API for the htslib modified bases interface

pysam-v0.18.0

Release 0.18.0

This release wraps htslib/samtools/bcftools version 1.14.

• [#1048] and [#1060], clarify documentation of index statistics with CRAM files
• Prevent "retval may be used uninitialised" warning.
• Add new "samples" subcommand to pysam/samtools.py
• Introduce TupleProxyIterator iterator object class

pysam-v0.17.0

Release 0.17.0

This release wraps htslib/samtools/bcftools version 1.13. Corresponding to new samtools commands, pysam.samtools now has additional functions ampliconclip, ampliconstats, fqimport, and version.

Bugs fixed:

• [#447] The maximum QNAME length is fully restored to 254
• [#506, #958, #1000] Don't crash the Python interpreter on pysam.bcftools.*() errors
• [#603] count_coverage: ignore reads that have no SEQ field
• [#928] Fix pysam.bcftools.mpileup() segmentation fault
• [#983] Add win32/*.[ch] to MANIFEST.in
• [#994] Raise exception in get_tid() if header could not be parsed
• [#995] Choose TBI/CSI in tabix_index() via both min_shift and csi
• [#996] AlignmentFile.fetch() now works with large chromosomes longer than 2²⁹ bases
• [#1019] Fix Sphinx documentation generation by avoiding Python 2 ur'string' syntax
• [#1035] Improved handling of file iteration errors
• [#1038] tabix_index() no longer leaks file descriptors
• [#1040] print(aligned_segment) now prints the correct TLEN value (it also now prints RNAME/RNEXT more clearly and prints POS/PNEXT 1-based)
• setup.py longer uses setup(use_2to3) for compatibility with setuptools >= v58.0.0

New facilities:

• [PR #963] Additional VCF classes are exposed to pysam programmers (thanks to @nh13)
• [#998, PR #1001] Add get/set_encoding_error_handler() to control UTF-8 conversion
• [PR #1012] Running python setup.py sdist now automatically runs cythonize (thanks to @marcelm)
• Running tests with pytest now automatically runs make to generate test data

Documentation improvements:

• [#726] Clarify get_forward_sequence/get_forward_qualities documentation
• [#865] Improved example
• [#968] get_index_statstics parameters
• [#986] Clarify VariantFile.fetch start/stop region parameters are 0-based and half-open.
• [#990] Corrected PileupColumn.get_query_sequences documentation
• [#999] Fix documentation for AlignmentFile.get_reference_length()
• [#1002] Document the default min_base_quality for pileup()

Notes and comments:

Apologies for the hiatus in releases. A big thank you to @jmarshall who has been doing a lot of the work of maintaining pysam in the last year. The delay in getting out releases has been entirely down to me. We have now automated the release process using Github actions. With less overhead, we should be able to release more timely again.

The priorities in the near future are to:

1. track htslib releases and release pysam in a timely manner.
2. remove python2 compatibility from pysam. This will simplify quite a bit of code within pysam.
3. review and triage the many open issues and pull requests. Many thanks to the whole community for submissions and apologies in the delay on acting on them.

The medium term priority is to review the pysam API in order to arrive at a 1.0 release with a consistent API.

pysam-v0.16.0.1

Release is identical to v0.16.0.

This release was made to be able re-upload fixed wheels of v0.16.0 to pypi.

pysam-v0.16.0

This release wraps htslib/bcftools version 1.10.2 and samtools version
1.10. The following bugs reported against pysam are fixed due to this:

• [#447] Writing out QNAME longer than 251 characters corrupts BAM
• [#640, #734, #843] Setting VariantRecord pos or stop raises error
• [#738, #919] FastxFile truncates concatenated plain gzip compressed files

Additional bugfixes:

• [#840] Pileup doesn't work on python3 when index_filename is used
• [#886] FastqProxy raises ValueError when instantiated from python
• [#904] VariantFile.fetch() throws ValueError on files with no records
• [#909] Fix incorrect quoting in VariantFile contig records
• [#915, #916] Implement pileup() for unindexed files and/or SAM files

Backwards incompatible changes:

• The samtools import command was removed in samtools 1.10, so pysam
  no longer exports a samimport function. Use pysam.view() instead.

Many thanks to @jmarshall who has done most of the heavy lifting.