Python-aptly

This is regular release of aptly.

Support for "legacy" Contents indexes

Ubuntu up to and including 16.04 is using "old" layout for Contents indexes for package repositories, while Debian has already switched to the new layout. Now aptly generates both "old" and "new" Contents indexes to allow `apt-file` to work with repositories published with Ubuntu <= 16.04 (667, 729)

Fall back to gpg1

aptly has two ways to sign published repositories: via external `gpg` command and using internal `openpgp` provider. Both of them support only GPG v1.x. To make it easier to support modern systems with GPG 2.x being installed by default, aptly tries to find GPG v1.x distribution by attempting to launch `gpg1` binary if `gpg` is version 2.x (734).

Changes

aptly repository has been moved to the org `aptly-dev`, and it's now being maintained by a team (https://github.com/orgs/aptly-dev/teams/maintainers/members)

repo.aptly.info repository key has been rotated (717)

AWS SDK & other libraries were upgraded (728)

Speed up `Contents` indexes generation (707)

aptly binaries are built with Go 1.10, aptly drops support for Go 1.7

Shell completion function for `zsh` was added to the repo (703)

Bugfixes

Fix error message truncating endpoint name in `aptly publish drop` (711, 713)

Fix Acquire-by-Hash index cleanup (705, 706)

Fix setting Acquire-by-Hash in the publishing API (696)

Fix Acquire-by-Hash in S3 publishing mode (692, 697)

Fix data race in API mode (686, 688)

Fix reading of long fields in Package stanzas (738)

Download

Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/

This is scheduled regular release of aptly (a bit late according to planned to schedule).

Acquire-By-Hash

aptly supports by hash layout for published repositories (https://wiki.ubuntu.com/AptByHash). This fixes a race when `apt` is downloading `Release` file while it's being updated to match new package indexes.

Acquire-By-Hash mode is backwards compatible with clients which do not support it. Feature should be enabled when repo or snapshot is first published, subsequent operations (update, switch) would have `acquire-by-hash` mode enabled.

Example:

aptly publish snapshot -acquire-by-hash snap-repo-20171205

Thanks to sliverc and neolynx for the implementation of this new feature (664, 381, 551).

Dependency Resolution

Several significant improvements were made to aptly internal dependency resolution engine. This specifically affects `-dep-follow-all-variants`, but there are some other changes improving compatibility.

With 1.2.0 filters which have dependency resolution enabled might produce a bit different results (which should be more correct anyway).

Refs: 615, 618, 643, 644

Changes

aptly is more flexible in terms of `.deb` format support, including recent `dpkg` changes (658, 665, 635, 632)

aptly supports SHA512 checksums in `Release` files (668, 660)

aptly searches package pool when importing packages in `aptly repo include` (278, 407)

aptly cleans up `*.buildinfo` files after importing packages (682, 679)

Duplicate packages are being reported correctly in `aptly package show`/`search` commands (446, 623)

Allow editing mirror URL (677)

Cleanup might be skipped while publishing (620)

goleveldb vendored library was updated to mitigate some bugs fixed upstream (665)

Go HTTP framework gin-gonic (used in aptly API) was updated to the latest version (646)

AWS SDK was upgraded (648)

aptly binaries are built with Go 1.9, aptly drops support for Go 1.6 (624)

Bugfixes

When mirror update fails or if it's canceled, aptly keeps already downloaded files (bug introduced in 1.1.0) (683, 651, 641)

S3 publishing with prefix was incorrectly cleaning up files (673) and re-uploading all the files on every upload (622, 619)

S3 publishing was dropping SSE parameters when copying files (649, 647)

Package names starting with uppercase latter are now handled correctly in queries (636)

Download

Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/

This is bugfix release for aptly 1.1.0 series.

aptly 1.1.0 contained bug (600) with `aptly mirror update` - if package files are already downloaded, `aptly mirror update` was clearing fields in internal package database, which resulted in failures for subsequent publishing.

If you are running aptly 1.1.0 upgrading to 1.1.1 and running `aptly mirror update` once again should be sufficient to mitigate that issue.

For more information on new features in 1.1.x, please see [1.1.0 release notes](https://github.com/smira/aptly/releases/tag/v1.1.0).

Download

Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/

aptly 1.1.0 release is following (a bit late) scheduled every 3 months release of aptly.

Upgrading

There're no steps required to upgrade from any version to 1.1.0, but *downgrading* from 1.1.0 to 1.0.x (or previous versions) is not supported due to changes in internal package pool layout.

Custom Filesystem Publishing

aptly supported configuring number of [S3](https://www.aptly.info/doc/feature/s3/) and [Swift](https://www.aptly.info/doc/feature/swift/) publishing endpoints for a long time, but local filesystem publishing was limited to default directory `~/.aptly/public` and hardlinking files. There was number of requests to allow use of symlinks or file copy to support publishing to different filesystems and shared locations like NFS.

With PR 521 from seeraven aptly supports [configuration of custom filesystem endpoints](https://www.aptly.info/doc/feature/filesystem/). Each endpoint has its own filesystem root, it might support either hardlinks, symlinks or simple file copy to reference files from aptly internal package pool.

Internal Package Pool

aptly stores package files deduplicated internal package pool (under `~/.aptly/pool` by default). MD5 checksum was used to build file path in the pool, which had two consequences:

1. MD5 checksum was required to pick location, so mirrors without MD5 checksums were not supported (228, 442).
2. In unlikely event when two different package files with same filename had same first two bytes of MD5 checksum, aptly would fail to import file (329).

In order to overcome those problems, package pool and internal package file storage was changed. Now aptly always recalculates checksums on files being downloaded or imported, making sure each file has full set of checksums (including SHA512). Location of package files in the pool now includes more bytes of SHA256 making conflict unlikely.

When upgrading from previous version, aptly would be able to find and use previously downloaded or imported package files, but new files would be stored in new locations. This means that downgrading from aptly 1.1.0 to previous versions is no longer supported (it's not advised to do that for any version of aptly, but with package pool changes this would make aptly loose track of package files). There are no special actions to upgrade aptly to 1.1.0.

Database Locking

There were several issues with `aptly api serve` with `-no-lock` which got fixed (431, 582). With 1.1.0 every aptly command on startup performs 10 attempts to reopen DB if database is locked. This makes easier to use CLI from cron or in CICD environments (401). Number of attempts is configurable with `-db-open-attempts` flag for every command. If you want to go back to pre-1.1.0 behaviour, use `-db-open-attempts=0`.

PGP Providers

GPG implementation is changing a lot in 2.x series, it's becoming harder and harder to support reliable interface to `gpg` command-line tool. aptly now supports [pluggable PGP providers](https://www.aptly.info/doc/feature/pgp-providers/). Two providers are implemented:

1. Previously used `gpg` provider (default for 1.1.0), which is invoking `gpg` command-line tool.
2. Internal PGP implementation based on https://github.com/golang/crypto/tree/master/openpgp.

Internal implementation is not as feature full as `gpg`, but it has better integration when signing repos, it doesn't require `gpg` to be installed. Key management (public/secret keyring) still requires `gpg`.

PGP provider could be changed with `-gpg-provider=[gpg|internal]` command-line flag or via `gpgProvider` configuration value.

Changes

aptly sorts search results and package lists (135, 214) with respect to correct version ordering.

It's allowed to create snapshots of empty repos (288).

aptly now correctly matches checksums for indexes while mirroring (376).

aptly packages from repo.aptly.info now correctly list `xz-utils` in dependencies (395).

aptly doesn't abort execution if `Contents` generation fails (451, 533).

aptly generates detailed log while resolving dependencies with `-dep-verbose-resolve` flag (508).

aptly can now optionally skip checking package files while downloading from the mirror (520).

aptly supports setting and inheriting from the mirror values for `Origin`, `NotAutomatic`, `BadAutomaticUpgrades` while publishing (577, thanks sliverc).

aptly trims slashes on publishing prefixes (613, 607).

Development

Development documentation is [now available](CONTRIBUTING.md).

aptly code is verified with set of linters both for Go source code and Python functional tests.

Files for back completion are now part of aptly source repository for easier packaging and PRs.

Download

Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/

Bugfix release for 1.0.0, contains single fix (543) to make aptly cleanup temporary directories created while publishing.

Download

Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/

Changes
=======

`Contents` index generation was re-implemented to use temporary DB instead of keeping index in memory which should lower memory requirements a lot during publishing.

Debian package reader was enhanced to guess correct archive type even if filename is incorrect (e.g. `data.tar.gz` while it's regular `tar` archive).

`.xz` compression format is supported for mirrors (index files).

`aptly mirror create` supports new flag `-force-architectures` which disables validation of supplied architectures in `Release` file (for mirrors with broken `Release` files).

`aptly mirror update` supports download retries via new flag `-max-tries`.

`aptly search *` family of commands allows query expression to be missing which means "display all the packages".

New command `aptly publish show` to display detailed information about published repository.

New command `aptly repo create ... from snapshot ...` to initialize local repository with snapshot contents.

`aptly api serve` supports systemd activation, listening on UNIX sockets and it bails out quickly if root dir is not writable.

aptly now supports `-dbgsym` packages while processing `aptly repo include`.

`aptly snapshot show` output was enhanced to include information about snapshot sources (and its current names).

`aptly graph` supports rendering to output files instead of launching viewer, viewer is now custom per platform. Also additional `vertical` layout is supported which might be useful to get better-looking diagrams.

Repo edit API doesn't touch repo fields which are not specified in the request.

`aptly version` command now reports correct version for nightly builds.

Extended support for OpenStack environment variables in Swift publisher.

Versioning
========

From now on, aptly release should happen every 3 months. For every release minor version would be bumped (e.g. next release would be `1.1.0`). Bugfix-release will bump patch version (e.g. `1.0.1`).

Nightly builds versioning has changed, now version format is following `git describe --tags` format: `x.y.z+<N>+<hash>`, where `x.y.z` is previous aptly release (important, before it was next version of aptly) followed by monotonically increasing `N` and git commit `hash`.

Development
==========

aptly stopped using [gom](https://github.com/mattn/gom) for vendored dependency management. From now on aptly has all the deps committed to the source tree under `vendor/` directory (standard Go vendoring). Vendored dependency are managed using [dep](https://github.com/golang/dep) tool.

Go source code is checked using [gometalinter](https://github.com/alecthomas/gometalinter), with only some most important checks enabled, more checks to be enabled in the next releases. Use `make check` to check your source code for problems.

aptly packages are built with Go 1.8.

Download
=======

Debian packages and binary distributions for various platforms are available at https://www.aptly.info/download/

Previous versions
=============

Changelog for previous version could be found at [aptly website](https://www.aptly.info/doc/history/).