Gemmi

Library:
* SmallStructure: changed how the space group is [read and accessed](https://gemmi.readthedocs.io/en/latest/mol.html#smallstructure-spacegroup).
Relying on H-M space group names alone was not always sufficient. The new mechanism uses the list of operations and Hall symbol in preference to the H-M symbol – the order is configurable.
* symmetry triplets: parse decimal fractions (small molecule files may use notation such as x+0.25 instead of x+1/4)
* tabulated space groups: a few more settings: B 1 2 1, B 1 21 1, F 1 m 1, F 1 d 1, F 1 2 1
* X-ray scattering coefficients: changed the default value of `IT92::ignore_charge` to true (i.e. charges are now ignored by default; before version 0.6.3 they were always ignored)
* cif::Table: added method `ensure_loop()` that converts tag-value pairs into a loop; might be needed before calling `append_row()`
* place_hydrogens(): fix for NH3-like configurations
* improved gemmi->mmdb conversion
* Grid: tweaked good_grid_size() to ensure that when creating a grid up to a certain d_min, all reflections up to d_min are in the grid (it matters when no oversampling is applied)
* DensityCalculator: deprecated function `set_grid_cell_and_spacegroup()`, use `grid.setup_from()`
* fixed TNT-compatible reciprocal space ASU calculation for non-standard settings
* infer_polymer_end(): complicate the heuristic even more, to detect files that have HETATM incorrectly used for standard residues in a polymer (such files were reported, they are either a result of mutating from non-standard residues, or a buggy program)
* added function assign_het_flags() to re-set ATOM/HETATM flags
* Model: added funtions `calculate_b_iso_range()` and `calculate_b_aniso_range()`; the first one can be used to detect if pLDDT is in the range 0-100 (like from AlphaFold) or 0-1 (like from ESMFold)
* writing mmCIF: write _entity_poly_seq.hetero
* added flag `Entity::reflects_microhetero` that shows if sequences were read from SEQRES (and don't account for point mutations) or from _entity_poly_seq; new function `add_microhetero_to_sequences()` changes the former to the latter

Program:
* gemmi sfcalc: added a few more options
* gemmi convert: added options `--assign-records[=A|H]`, improved `--sifts-num`, adding microheterogeneities to _entity_poly_seq when converting from PDB
* gemmi cifdiff: added option `-t` for basic comparison of values for a single tag

Other:
* minimal WebAssembly port (C++ code compiled with emscripten) of Structure,
as a proof-of-concept and for reading mmCIF files in UglyMol
* examples/to_rdkit.py: example of conversion of gemmi ChemComp to RDKit Mol

and a number of less important changes

Library:
* gemmi can now be built with zlib-ng, a faster fork of zlib (good for working with large, compressed files)
* experimental: binary serialization of Structure (contained objects, such as Model, Chain or UnitCell, can also be serialized separately)
* finalized handling of 5-character monomer names; uses the tilde-hetnam extension (`ABCDE` ↔ `~DE`) for PDB files
* when atom names in the coordinate file match previous names (`_chem_comp_atom.alt_atom_id`) from the monomer library (the names in the CCD and therefore also in the ML change occasionally), print better diagnostic; added function `MonLib::update_old_atom_names()` to update the names in a Structure
* topology: fixed handling of two bonds between the same two residues
* options for handling mmCIF files with incorrect entities (modified `add_entity_ids()` when called with `overwrite=true`)
* added function `Intensities::prepare_merged_mtz()`
* a few bug fixes (for instance, in handling of negative residue numbers in the selection syntax)

Python bindings:
* generating type stubs - see 293
* python: `cif.Loop.val()` has been replaced with `__getitem__`/`__setitem__`
* fixed `Mtz.Batch.ints` and `Mtz.Batch.floats`

Program
* subcommand diff has been renamed to cifdiff
* subcommand prep has been renamed to crd
* validate: more options for checking monomer files
* gemmi-grep: added option --extended-regexp
* mtz2cif: added column names Iplus/Iminus (used by ccp4i2) to the default conversion spec

Note: this list is meant to show important changes only.

Library
* completely changed build system for Python module, from setuptools to scikit-build-core
* optimized electron density calculation: single-precision version is now about 2x faster and slightly less exact; some other grid-based calculations also got optimized in the process
* as part of the above optimizations, some of the grid computations require that the model is in the standard orientation (conventional axis directions); in other cases (which are very rare after the [remediation of non-standard coordinate frames](https://www.wwpdb.org/news/news?year=2023#6525a09ad78e004e766a96af) in the PDB) call standardize_crystal_frame()
* CIF output: more flexible formatting
* mmCIF writing: category _entity_poly is included by default, with pdbx_strand_id and pdbx_seq_one_letter_code
* minor changes in reading mmCIF coordinate files
* cif: added functions Loop::add_columns(), Loop::remove_column(), Column::erase()
* MRC map format: ORIGIN record is ignored (previously, if ORIGIN was non-zero, Ccp4::full_cell() returned false and some map properties were not set)
* new function Grid::symmetrize_avg()
* fixed bug in ReciprocalGrid::prepare_asu_data()
* added function read_pir_or_fasta() for reading sequences (previously it was undocumented and more limited)
* added function pdbx_one_letter_code() which returns a string like AA(MSE)H…, for _entity_poly.pdbx_seq_one_letter_code
* new functions expand_one_letter() and expand_one_letter_sequence() that take ResidueKind.AA/RNA/DNA as argument replaced expand_protein_one_letter*()
* adjusted weights in align_sequence_to_polymer()
* added function assign_best_sequences()
* PDB reading: added Structure::ter_status flag to indicate if TER records were: absent, present, clearly in wrong places
* experimental (not documented yet) new functions: Model::get_cra(), Model::get_parent_of()
* Topo::Bond stores a flag for bonds between different symmetry images
* ChemComp::Atom: store _chem_comp_atom.alt_atom_id as old_id, use it in new function update_old_atom_names()
* riding hydrogens: added H had wrong occupancy in special, rare cases
* added Vec3f – Vec3 with single-precision numbers
* minor API changes: Binner::setup() doesn't return anything, changed argument types of Scaling::scale_data(), align_sequences()

Program
* new tool gemmi-diff that compares categories and tags in two (mm)CIF files
* gemmi-align prints vertical list with option --verbose
* gemmi-residues has new options: -e, -sss, --chains
* gemmi-rmsz: added option --missing to print missing atoms
* gemmi-validate: more options for validating monomer files
* gemmi-h: more options
* gemmi-mtz: prints info about SYMM records

* new: normalization of amplitudes using so-called "Karle" approach, similar as in the CCP4 program ECALC
* added X-ray scattering coefficients for ions (previously, the charge of atom was ignored)
* pdb: reading CONECT records, and an option to also write them
* when reading pdb, if any chain has 2+ TER records, all TER records are ignored
* more configuration options for writing pdb files
* added functions Mtz::expand_to_p1() and Mtz::read_file_gz()
* cif::Block::find_value(tag) now returns also value from the corresponding loop if that loop has only one row
* changes in gemmi-validate related to validation with DDL2
* gemmi-sfcalc: added option --sigma-cutoff
* gemmi sf2map --mapmask: if the unit cells in coordinate file is different than in SF file, use only the latter
* improved transform_to_assembly(), expand_ncs() and rename_chain()
* cif2mtz: Mtz column for pdbx_DELPHWT has now label PHDELWT (272)
* fixed ensure_asu(): phase-shift (for phases and H-L coefficients) was wrong
* fixed UnitCell::find_nearest_image() for non-crystals with NCS
* fixed DensityCalculator::requested_grid_spacing()
* changes and enhancements in add_chemcomp_to_block(), in solvent masking, in mtz2cif,
and in several other places
* added python bindings to MtzToCif, cif::Ddl, PdbWriteOptions, changed how options for PDB writing are passed, more bindings for Mtz::Batch

* a number of fixes, mostly in topology preparation
* support for extended (longer) CCD and PDB codes that are about to be introduced by the PDB
* gemmi-convert: added option to rename a monomer
* a few changes and additions in cif2mtz, including:
* anomalous data written as separate rows for F+ and F- is now converted as expected
* _refln.F_squared_meas is now a synonym for F_squared_meas
* gemmi-grep: new option --only-tags
* gemmi-validate: a couple of new checks and options
* pdb and mmCIF: convert MODRES <-> _pdbx_struct_mod_residue
* cif.Block: blocks with no name (just ``data_``) used to have the name set to "", now it's " "

* changed how CISPEP is stored: previously, it was assumed that a link between two residues is either TRANS or CIS; if the residues have atoms with alternative conformations, both link types can be present at the same time
* riding hydrogens: previously, hydrogens had the same altlocs as the parent atom; now if the parent atom has a single conformation, but it has neighbors in multiple conformations, the hydrogens will be also added in multiple conformations
* major changes in NearestNeighbor: now it's possible to search atoms not only in the first nearest [cells][1], but in any number of nearest cells; find_nearest_atom() was changed to find, by default, a nearest atom within any distance
* changes in Mtz.reindex(), primarily to fix determination of the new space group
* gemmi-convert: added option --all-auth to write `_atom_site.auth_atom_id` and `auth_comp_id`, which are skipped by default (because they are always the same as label_…_id)
* added more options to gemmi-rmsz, gemmi-xds2mtz, gemmi-cif2mtz
* fix a recent regression in check_polymer_type(): RNA was returned instead of DNA
* improved heuristic of detecting where the polymer ends (if TER record is missing)
* selection syntax: fix parsing a single sequence id such as "A/208" (it was parsed as A/208-/)
* removed SMat33::calculate_eigenvector() – use eigen_decomposition() instead; SoftwareItem::pdbx_ordinal; NeighborSearch::Mark::x,y,z (use ::pos)
* more code was moved from headers to src/*.cpp

[1]: https://en.wikipedia.org/wiki/Cell_lists