Nextstrain-augur

Features

* export v2: we now limit numerical precision on floats in the JSON. This should not change how a dataset is displayed / interpreted in Auspice but allows the gzipped & minimised JSON filesize to be reduced by around 30% (dataset-dependent). [1512][] (jameshadfield)
* traits, export v2: `augur traits` now reports all confidence values above 0.1% rather than limiting them to the top 4 results. There is no change in the eventual Auspice dataset as `augur export v2` will still only consider the top 4. [1512][] (jameshadfield)
* curate: Excel (`.xlsx` and `.xls`) and OpenOffice (`.ods`) spreadsheet files are now also supported as metadata inputs (`--metadata`). The first sheet in the workbook is read as tabular data. [1550][] (tsibley)

Bug Fixes

* titers sub: Fixes a bug where antigenic weights were assigned to branches for substitutions in the incorrect order of `<derived allele><position><ancestral allele>` instead of `<ancestral allele><position><derived allele>`. [1555][] (huddlej)

[1512]: https://github.com/nextstrain/augur/pull/1512
[1550]: https://github.com/nextstrain/augur/pull/1550
[1555]: https://github.com/nextstrain/augur/pull/1555

Bug Fixes

* curate parse-genbank-location: Fix a bug where a mix of empty and populated location-field values would result in inconsistent fields in the output NDJSON [1531][](genehack)

[1531]: https://github.com/nextstrain/augur/pull/1531

Features

* Support xopen major version 2. Deprecate v1. Schedule for removal around November 2024. [1532][] (corneliusroemer)
* Support networkx major version 3. [1534][] (corneliusroemer)

[1532]: https://github.com/nextstrain/augur/pull/1532
[1534]: https://github.com/nextstrain/augur/pull/1534

Major changes

* curate format-dates: Raises an error if provided date field does not exist in records. [1509][] (joverlee521)
* All curate subcommands: Verifies all input records have the same fields and raises an error if a record does not have matching fields. [1518][] (joverlee521)

Features

* Added a new sub-command `augur curate apply-geolocation-rules` to apply user curated geolocation rules to the geolocation fields in a metadata file. Previously, this was available as a script within the nextstrain/ingest repo. [1491][] (victorlin)
* Added a default color for the "Asia" region that will be used in `augur export` is no custom colors are provided. [1490][] (joverlee521)
* Added a new sub-command `augur curate apply-record-annotations` to apply user curated annotations to existing fields in a metadata file. Previously, this was available as a `merge-user-metadata` in the nextstrain/ingest repo. [1495][] (joverlee521)
* Added a new sub-command `augur curate abbreviate-authors` to abbreviate lists of authors to "<first author> et al." Previously, this was avaliable as the `transform-authors` script within the nextstrain/ingest repo. [1483][] (genehack)
* Added a new sub-command `augur curate parse-genbank-location` to parse the `geo_loc_name` field from GenBank reconds. Previously, this was available as the `translate-genbank-location` script within the nextstrain/ingest repo. [1485][] (genehack)
* curate format-dates: Added defaults to `--expected-date-formats` so that ISO 8601 dates (`%Y-%m-%d`) and its various masked forms (e.g. `%Y-XX-XX`) are automatically parsed by the command. [1501][] (joverlee521)
* Added a new sub-command `augur curate transform-strain-name` to filter strain names based on matching a regular expression. Previously, this was available as the `transform-strain-names` script within the nextstrain/ingest repo. [1514][] (genehack)
* Added a new sub-command `augur curate rename` to rename field / column names. Previously, a similar version was available as the `transform-field-names` script within the nextstrain/ingest repo however the behaviour is slightly changed here. [1506][] (jameshadfield)

Bug Fixes

* filter: Improve speed of checking duplicates in metadata, especially for large files. [1466][] (victorlin)
* curate: Stop adding double quotes to the metadata TSV output when field values have internal quotes. [1493][] (joverlee521)
* curate format-dates: Mask empty date values as `XXXX-XX-XX` to represent unknown dates. [1509][] (joverlee521)

[1466]: https://github.com/nextstrain/augur/pull/1466
[1490]: https://github.com/nextstrain/augur/pull/1490
[1491]: https://github.com/nextstrain/augur/pull/1491
[1493]: https://github.com/nextstrain/augur/pull/1493
[1495]: https://github.com/nextstrain/augur/pull/1495
[1501]: https://github.com/nextstrain/augur/pull/1501
[1509]: https://github.com/nextstrain/augur/pull/1509
[1514]: https://github.com/nextstrain/augur/pull/1514
[1518]: https://github.com/nextstrain/augur/pull/1518
[1506]: https://github.com/nextstrain/augur/pull/1506

Features

* All commands: Allow repeating an option that takes multiple values. Previously, if multiple option flags were specified (e.g. `--exclude-where 'region=A' --exclude-where 'region=B'`), only the last one was used. Now, all values are used. [1445][] (victorlin)
* ancestral, translate: output node data files are now validated. The argument `--validation-mode` is added which controls this behaviour (default: error). This argument also controls validation of the input node-data file (ancestral only). [1440][] (jameshadfield)
* export: Updated default latitudes and longitudes for geography traits. This only applies if you are **not** using `--lat-longs` to override the built in mappings. [1449][] (trvrb)

Bug Fixes

* validation: we no longer exit with a non-zero exit code when the requested validation mode is "warn" [1440][] (jameshadfield)
* validation: we no longer perform any validation when the requested validation mode is "skip" [1440][] (jameshadfield)
* filter: Send all log messages to `stderr`. This allows output to be written to `stdout` (e.g. `--output-strains /dev/stdout`). [1459][] (victorlin)

[1440]: https://github.com/nextstrain/augur/pull/1440
[1445]: https://github.com/nextstrain/augur/pull/1445
[1449]: https://github.com/nextstrain/augur/pull/1449
[1459]: https://github.com/nextstrain/augur/pull/1459

Features

* filter: Added a new option `--max-length` to filter out sequences that are longer than a certain amount of base pairs. [1429][] (victorlin)
* parse: Added support for environments that use pandas 2.x. [1436][] (emollier, victorlin)

Bug Fixes

* filter: Updated docs with an example of tiered subsampling. [1425][] (victorlin)
* export: Fixes bug [1433] introduced in v23.1.0, that causes validation to fail when gene names start with `nuc`, e.g. `nucleocapsid`. [1434][] (corneliusroemer)
* import: Fixes bug introduced in v24.2.0 that prevented `import beast` from running. [1439][] (tomkinsc)
* translate, ancestral: Compound CDS are now exported as segmented CDS and are now viewable in Auspice. [1438][] (jameshadfield)

[1425]: https://github.com/nextstrain/augur/pull/1425
[1429]: https://github.com/nextstrain/augur/pull/1429
[1433]: https://github.com/nextstrain/augur/issues/1433
[1434]: https://github.com/nextstrain/augur/pull/1434
[1436]: https://github.com/nextstrain/augur/pull/1436
[1438]: https://github.com/nextstrain/augur/pull/1438
[1439]: https://github.com/nextstrain/augur/pull/1439