Pulpcore

REST API

Bugfixes

- Fixed two instances of Pulp not writing to the task worker's temporary directory.
[2061](https://github.com/pulp/pulpcore/issues/2061)
- Taught task-purge to process tasks in batches of 1000. This prevents large purges from using
large amounts of memory as a result of reading all the affected Tasks into memory at once.
[2215](https://github.com/pulp/pulpcore/issues/2215)
- Loosened the version-restrictions on PulpImport to only require X.Y matching.
[2269](https://github.com/pulp/pulpcore/issues/2269)
- Fixed a (rare) deadlock around bulk_update() during syncs with overlapping content.
[2430](https://github.com/pulp/pulpcore/issues/2430)
- Fixed a bug where notifications to workers may go unnoticed. This may lead to idle workers while
there are tasks waiting.
[2506](https://github.com/pulp/pulpcore/issues/2506)

Plugin API

No significant changes.

REST API

Bugfixes

- Reduced memory usage during tasks like sync by holding fewer objects in-memory unnecessarily.
[2069](https://github.com/pulp/pulpcore/issues/2069)
- Fixes duplicate key error `Key (content_artifact_id, remote_id)` when creating `RemoteArtifacts`
during syncs in pulp_container and possibly other plugins.
[2381](https://github.com/pulp/pulpcore/issues/2381)
- Declared proper dependency on user model in migration 0040.
[2403](https://github.com/pulp/pulpcore/issues/2403)
- Fixed a rare deadlock when sync'ing overlapping content in high-concurrency envs.
[2420](https://github.com/pulp/pulpcore/issues/2420)

Plugin API

Bugfixes

- Adjusted the default size of the queues between pipelines to be 1 instead of 1000. The batchers in
the stage will still accumulate up to 500 (by default) items so batching is still in-effect there
where it matters.
[2069](https://github.com/pulp/pulpcore/issues/2069)

REST API

Bugfixes

- Fix delete repository version causing "duplicate key value violates unique constraint" error.
[2047](https://github.com/pulp/pulpcore/issues/2047)

- Fixed migration 0064_add_new_style_task_columns to purge extraneous ReservedResource and
TaskReservedResource entries, which could block sync and publish tasks post-upgrade.

Also taught the migration to bulk-update the Task changes. In large installations, this
should have a positive impact on the time it takes to apply the migration.
[2101](https://github.com/pulp/pulpcore/issues/2101)

- This fix prevents the lost track of a content removed version when deleting a repository version that deletes a content that is added back in the subsequent version, but deleted again in a later version.
[2267](https://github.com/pulp/pulpcore/issues/2267)

- Fix a mistake in a previous migration which may have caused improperly encrypted remote fields.
[2327](https://github.com/pulp/pulpcore/issues/2327)

Plugin API

No significant changes.

REST API

Bugfixes

- Added transactions around repository version operations to prevent data loss.
[2268](https://github.com/pulp/pulpcore/issues/2268)

Plugin API

No significant changes.

REST API

Features

- Added a pulpcore-manager command remove-plugin to remove a Pulp plugin.
[1945](https://github.com/pulp/pulpcore/issues/1945)

- Implemented the Redis caching machinery that can be used within the synchronous context.
[2003](https://github.com/pulp/pulpcore/issues/2003)

- Add a "current_task" field to workers so that you can easily see which workers are currently active.
[2034](https://github.com/pulp/pulpcore/issues/2034)

- Allowed Pulp to install without Redis.
[2057](https://github.com/pulp/pulpcore/issues/2057)

- Added `has_repository_obj_perms` and `has_repository_model_or_obj_perms` as access conditions
that can be used by viewsets nested beneath repository viewsets.
[2076](https://github.com/pulp/pulpcore/issues/2076)

- Queryset scoping can now be disabled by changing the `DEFAULT_PERMISSION_CLASSES`.
[2115](https://github.com/pulp/pulpcore/issues/2115)

- Specifying a different value for `DEFAULT_PERMISSION_CLASSES` will now automatically disable the
permission assignment provided by the `creation_hooks` portion of an `AccessPolicyFromDB`.
[2116](https://github.com/pulp/pulpcore/issues/2116)

- The Pulp API can now be rerooted using the new `API_ROOT` setting. By default it is set to
`/pulp/`. Pulp appends the string `api/v3/` onto the value of `API_ROOT`.
[2148](https://github.com/pulp/pulpcore/issues/2148)

- Added a redirecting content guard that can be employed by plugins to forward from a REST call to
the content app.
[2151](https://github.com/pulp/pulpcore/issues/2151)

- Added a new analyze-publication management command to facilitate debugging.
[2200](https://github.com/pulp/pulpcore/issues/2200)

- Added a read-only task schedule view for configured tasks schedules.

NOTE: This feature is in tech-preview and may change in backwards incompatible ways in the future.
[2204](https://github.com/pulp/pulpcore/issues/2204)

Bugfixes

- Fix import and export OOM error.
[2072](https://github.com/pulp/pulpcore/issues/2072)

- Fixed downloader retry logic with partially written files.
[2078](https://github.com/pulp/pulpcore/issues/2078)

- Fix content summary showing incorrect count after previous version deletion.
[2084](https://github.com/pulp/pulpcore/issues/2084)

- Fixed issue with listing repository versions after deleting previous versions.
[2085](https://github.com/pulp/pulpcore/issues/2085)

- Fixed file descriptior leak during upload.
[2087](https://github.com/pulp/pulpcore/issues/2087)

- Fixed an api schema bug where both `permission_assignment` and `creation_hooks` were required
by the `AccessPolicy` serializer.
[2089](https://github.com/pulp/pulpcore/issues/2089)

- Fixed migration 0081 to be compatible with custom User models.
[2090](https://github.com/pulp/pulpcore/issues/2090)

- Fixed PulpImport to correctly save relative to MEDIA_ROOT.
[2091](https://github.com/pulp/pulpcore/issues/2091)

- Added proper logging around certain ways a task could fail.
[2093](https://github.com/pulp/pulpcore/issues/2093)

- Taught PulpImport to retry in the event of a concurrency-collision on ContentArtifact.
[2102](https://github.com/pulp/pulpcore/issues/2102)

- Fixed an edge case where the first (streamed) response from an repo synced as "on_demand" could be incorrect.
[2119](https://github.com/pulp/pulpcore/issues/2119)

- Fixed bug where retries of partially downloaded files failed digest and size validation.
[2135](https://github.com/pulp/pulpcore/issues/2135)

- Fixed the calculation of response range headers in streaming answers from the content app.
[2147](https://github.com/pulp/pulpcore/issues/2147)

- Fixed potential deadlock-window in touch() path.
[2157](https://github.com/pulp/pulpcore/issues/2157)

- Fixed reporting tasks being canceled before being picked up by a worker as canceled instead of
failed.
[2183](https://github.com/pulp/pulpcore/issues/2183)

- Return a concise message exception on 500 in case file is missing on the FS.
[2187](https://github.com/pulp/pulpcore/issues/2187)

- Fixed import/export of repositories with sub-content.

An example would be the sub-repositories in pulp_rpm
DistributionTrees.
[2192](https://github.com/pulp/pulpcore/issues/2192)

- touch() now uses standard Django instead of raw-sql to update.
[2229](https://github.com/pulp/pulpcore/issues/2229)

Misc

- [2086](https://github.com/pulp/pulpcore/issues/2086), [#2094](https://github.com/pulp/pulpcore/issues/2094), [#2173](https://github.com/pulp/pulpcore/issues/2173), [#2202](https://github.com/pulp/pulpcore/issues/2202)

Plugin API

Features

- The `AutoAddObjPermsMixin` now calls a `handle_creation_hooks` interface on the configured DRF
permission class, e.g. the default `AccessPolicyFromDB`.
[2116](https://github.com/pulp/pulpcore/issues/2116)
- Added a redirecting content guard that can be employed by plugins to generate preauthenticated URLs
that forward from a REST call to the content app. Added the `GetOrCreateSerializerMixin` to
`get_or_create` objects still validating them through the serializer.
[2151](https://github.com/pulp/pulpcore/issues/2151)
- Exposed synchronous and asynchronous caching machinery. The classes `AsyncContentCache` and
`SyncContentCache` can be now managed by plugin writers.
[8806](https://github.com/pulp/pulpcore/issues/8806)

Bugfixes

- Started using a plugin provided serializer in has_remote_param_obj_perms and
has_remote_param_model_or_obj_perms to solve the Unexpected field error during sync by
a non-admin user.
[2088](https://github.com/pulp/pulpcore/issues/2088)
- Exposed adjust_roles in the plugin api.
[2092](https://github.com/pulp/pulpcore/issues/2092)
- Expose Roles model in plugin api.
[2185](https://github.com/pulp/pulpcore/issues/2185)

Removals

- Deprecate the the use of `custom_file_object` parameter in `BaseDownloader`.
[2123](https://github.com/pulp/pulpcore/issues/2123)

Deprecations

- The `API_ROOT` constant has been deprecated and turned into a setting. Its removal is scheduled
for 3.20. Please use the setting with the same name instead.
[2148](https://github.com/pulp/pulpcore/issues/2148)
- The `ACCESS_POLICY_VIEWSET_NAME` attribute is no longer expected to be present on models. The
RBAC machinery no longer uses this, and if present a deprecation warning will be emitted.
[2209](https://github.com/pulp/pulpcore/issues/2209)

REST API

Bugfixes

- Taught PulpImport to stream imports rather than reading files into memory in one chunk.

This largely alleviates the memory-pressure that results from importing multiple
large repositories in parallel.
[2307](https://github.com/pulp/pulpcore/issues/2307)

- Fixed does_batch method in sync pipeline to allow waiting on content that is already resolved.
[2557](https://github.com/pulp/pulpcore/issues/2557)

- Ensure downloader resets file on retry.
[2576](https://github.com/pulp/pulpcore/issues/2576)

- Taught PulpImport to retry more than once in the event of creation-collisions.

This fixes a rare import-failure during high-concurrency, high-content-overlap imports.
[2589](https://github.com/pulp/pulpcore/issues/2589)

- Improved the error message when HTTP proxies reject requests from Pulp.
[2654](https://github.com/pulp/pulpcore/issues/2654)

Plugin API

No significant changes.