Zhmcclient

^^^^^^^^^^^^^^

This version contains all fixes up to version 1.11.4.

Released: 2023-11-16

**Incompatible changes:**

* The pull_full_properties(), pull_properties(), get_property() and props()
methods on resource objects
now raise zhmcclient.CeasedExistence in all cases where the resource no
longer exists on the HMC. This provides a consistent behavior across different
cases the method can encounter. Previously, that exception was raised only for
resources that had auto-update enabled, and resources with auto-update
disabled raised zhmcclient.HTTPError(404,1) instead when the resource no
longer existed on the HMC.
If you use these methods and check for resource existence using
HTTPError(404,1), you need to change this to check for CeasedExistence
instead.

* The pull_properties() methods on resource objects now retrieves all properties
from the HMC when one or more of the specified properties are not supported
by the resource. This provides a consistent behavior across the different
cases the method can encounter. Previously, that method behaved differently
when the property was not supported by the resource: It has retrieved all
properties when the resource type or HMC version does not support property
filtering, but has raised HTTPError(400,14) in case the resource type and
HMC version did support property filtering.
If you use this method and check for HTTPError(400,14), this check can now be
removed.

**Deprecations:**

* Use of the 'status_timeout' and 'allow_status_exceptions' parameters of the
following methods has been deprecated because the underlying HMC operations
do not actually have deferred status behavior. The waiting for an expected
status has been removed from these methods:
- Lpar.stop()
- Lpar.psw_restart()
- Lpar.reset_normal()
- Lpar.reset_clear()

**Bug fixes:**

* Test: Circumvented a pip-check-reqs issue by excluding its version 2.5.0.

* Addressed safety issues up to 2023-11-05.

* Fixed the maximum number of concurrent threads in bulk operations to be
the documented maximum of 10.

* Test: Added unit tests and end2end tests for list permitted partitions operation

* Docs: Corrected and improved the description of the Lpar.activate() method.

* Test: Added end2end tests for LPAR activation in classic mode.

* Fixed the waiting for LPAR status in Lpar.activate(). Previously, the method
was waiting for 'operating' or 'not-operating', so when an auto-load
happened it already returned when status 'not-operating' was reached, but
the load was still going on in parallel. Now, the method finds out whether
the LPAR is expected to auto-load or not and waits for the corresponding
status.

* Added a debug log entry when Lpar.wait_for_status() is called. This happens
for example when Lpar.activate/deactivate/load() are called with
wait_for_completion.

* Fixed that the Lpar.reset_normal() and Lpar.reset_clear() methods were
waiting for a status "operational", which never happens with these operations.
This was fixed by removing the waiting for an expected status, because the
underlying HMC operations do not actually have deferred status behavior.
(issue 1304)

* Fixed the incorrect empty request body in Lpar.psw_restart().

* Shortened the status timeout from 900 sec to 60 sec. This timeout is used
when waiting for an expected Partition or LPAR status after operations
that change the status and that have deferred status behavior (ie. the
status changes only after the asynchronous HMC job is complete).
This change allows to more reasonably surface the situation where an LPAR
load succeeds but the status of the LPAR does not go to 'operating' due to
issues with the operating system.

* Docs: Fixed the description of the 'status_timeout' parameter of the Partition
and Lpar methods that have deferred status behavior.

* The 'wait_for_completion' and 'operation_timeout' parameters of
Cpc.export_profiles() and Cpc.import_profiles() have never worked, because
the underlying HMC operations are not actually asynchronous. This has been
fixed by removing these parameters from these functions. This does not count
as an incompatible change because using these parameters with non-default
values has failed. (part of issue 1299)

**Enhancements:**

* Added support for Python 3.12. Had to increase the minimum versions of
setuptools to 66.1.0 and pip to 23.1.2 in order to address removal of the
long deprecated pkgutils.ImpImporter in Python 3.12, as well as several
packages used only for development. (issue 1300)

* Mock support: Improved mocked Hipersocket adapters; they now have all their
properties and default values for all except adapter-id and channel-path-id.

* Added support for the "List OS Messages" operation on partitions (in DPM mode)
and LPARs (in classic mode). (issue 1278)

* Examples: Added example script increase_crypto_config.py for increasing the
crypto configuration of a partition on a CPC in DPM mode.

* The pull_properties() method on resource objects was extended so that its
'properties' parameter can now also be a single string (in addition to the
already supported list or tuple of strings).

* Added a get_properties_pulled() method for resource objects, which gets the
current value of a set of properties from the HMC. If the resource has
auto-update enabled, it gets the value from the (automatically updated) local
cache. Otherwise, it retrieves the properties from the HMC in the fastest
possible way, considering property filtering if supported.

* Added support for passing an exception message directly to the
zhmcclient.NotFound exception, instead of creating it from the 'manager' and
'filter_args' parameters, which are now optional.

* Added support for asynchronous job cancellation via a new method Job.cancel().
Documented for all asynchronous methods returning Job objects whether or not
they can be cancelled. (issue 1299)

* Added support for low level management of asynchronous jobs via new methods
Job.query_status() and Job.delete(). Note that higher level methods
Job.check_for_completion() and Job.wait_for_completion() already existed.
(issue 1299)

* Added support for creation and deletion of activation profiles on z16.
This requires the SE to have a code level that has the
'create-delete-activation-profiles' API feature enabled.
(issue 1329)

* Added Lpar.start() to perform the "Start Logical Partition" operation in
classic mode. (issue 1308)

^^^^^^^^^^^^^^

This version contains all fixes up to version 1.10.1.

Released: 2023-09-07

**Incompatible changes:**

* Fixed BaseResource.pull_properties() by returning None when no properties
were specified. Before that, it returned the full set of properties when
the Get Properties operation for the resource does not support the 'properties'
query parameter, and produced 'properties=' as a query parameter when
the resource does support the 'properties' query parameter.

This is incompatible when your code uses pull_properties() on resource objects
and relies on the prior behavior.

* Installation of this package using "setup.py" is no longer supported.
Use "pip" instead.

**Bug fixes:**

* Fixed safety issues from 2023-08-27.

* Fixed zhmcclient_mock support for LDAP Server Definitions.

* Fixed end2end testcases for adapters, auto-updating, and groups.

* Fixed that SubscriptionNotFound exception message did not resolve its
format string.

* Fixed the zhmcclient_mock support by adding support for query parameters,
fixing the the Group operations and the "Query API Version" operation,
and fixing the z16 mock environment definitions.
Auto-update tests are now skipped when testing against mocked environments,
because the mock support does not support notifications.

**Enhancements:**

* Docs: Improved documentation for developing tests.

* Implemented mock support for aggregation service operation "Submit requests"
(bulk operations) (issue 1250).

* Added support for requesting additional properties in list() methods for
Adapter, Certificate, Partition, VirtualSwitch, ImageActivationProfile
resources, and for Console.list_permitted_lpars().

* Improved performance of list() method of all resource types when called
with full_properties=True by using a bulk operation (aggregation service).

* Test: Added a new make target "end2end_mocked" to run the end2end tests against
the mocked environments in the "examples" directory. As part of that, combined
the coverage results of unit tests and end2end tests into a single data file
that each test contributes to.

**Cleanup:**

* Consolidated common code of list() methods into the _utils.py module.

^^^^^^^^^^^^^^

This version contains all fixes up to version 1.9.1.

Released: 2023-08-04

**Bug fixes:**

* Fixed issue with PyYAML 5.4 installation on Python>=3.10 that fails since
the recent release of Cython 3.

* Fixed example mocked environments to be useable in end2end test.

**Enhancements:**

* Added support for upgrading the HMC and SE to a new bundle level via new
zhmcclient.Console.single_step_install() and
zhmcclient.Cpc.single_step_install() methods. (issue 1219)

* Added resource class and name to HMC log entries. (issue 1058)

* Test: Added pytest fixtures for mocking at the HTTP level for unit tests
in cases where zhmcclient mock support is not implemented.

* Added support for LPAR Load from FTP via a new Lpar.load_from_ftp()
method. (issue 1048)

* Added support for STP configuration of CPCs via new operations of
zhmcclient.Cpc: swap_current_time_server(), set_stp_config(),
change_stp_id(), join_ctn(), leave_ctn(). (issue 750)

**Cleanup:**

* Fixed new issue reported by flake8 6.1.0.

* Converted all the percent-style string usages to format style except
the logging calls. Logging will continue to use percent-style. (issue 663)

^^^^^^^^^^^^^

This version contains all fixes up to version 1.8.2.

Released: 2023-07-14

**Incompatible changes:**

* Renamed the `Session.resource_updater` property to `auto_updater` and the
`zhmcclient.ResourceUpdater` class to `AutoUpdater` to take into account that
the class and property now represent auto-updated manager objects in addition
to auto-updated resource objects. Note that the property and class are
still experimental in this version.

**Deprecations:**

* Deprecated the 'verify' parameter of 'Session.logoff()'. Its use with
verify=True caused an invalid session to first be renewed and then deleted
again. It is no longer used.

**Bug fixes:**

* Fixed and improved session creation, deletion and automatic renewal.
Fixed the arguments passed to the retried HTTP operations in case the session
gets renewed.
Added the ability to log off sessions properly in case the session ID is
invalid, by adding a 'renew_session' flag to Session.get/post/delete() (this
ability is needed for zhmccli to address its issue 421).
Fixed Session.is_logon(verify=True) which would log on in certain cases.
Optimized Session.logoff(verify=True) which had logged on and then off again
in case the session was already invalid.
Improved and fixed the descriptions of Session.logon(), logoff(), is_logon()
and session_id.

* In addition to 403.5 (session ID invalid), 403.4 (no session ID provided) is
now also automatically handled by the zhmcclient in the same way, i.e. by
performing a logon to the HMC and a retry of the HMC operation.

* Circumvented the removal of Python 2.7 from the Github Actions plugin
setup-python, by using the Docker container python:2.7.18-buster instead.

* Addressed safety issues from 6+7/2023, by increasing 'requests' to 2.31.0
on Python >=3.7, and by increasing other packages only needed for development.

* Fixed the handling of HTTP status 202 with empty response content: The
old code tested the content for '' but the content is always a binary string.
In Python 3.x, that check results in False and subsequently in an attempt
to parse the empty string using JSON, which failed with a ParseError.
Fixed by comparing the empty string against b''.

* Improved the handling of logoff: It now also tolerates a ConnectionError,
which may be raised when the console.restart() method is used and the
HMC quickly enough becomes unavailable.

* Fixed the bug issue template by correcting the command to display debug data.

**Enhancements:**

* Reworked export_dpm_configuration() to avoid using the "cpc" category when
doing the initial GET Inventory call. This reduces the likelihood of running
into problems during export due to problems with any of the CPCs managed by
the HMC.

* Improved performance of metrics retrieval and processing for NIC and partition
related metrics for CPCs in DPM mode.

* Added optimized lookup by name in list() methods of the following resource
classes: `LdapServerDefinition`, `PasswordRule`, `Task`, `User`,
`UserPattern`, `UserRole`,

* Added support for auto-updated resource managers. An auto-updated resource
manager has its list of resources automatically updated as resources are
created and deleted on the HMC, based on HMC notifications. (issue 1055)

Added an example script examples/show_auto_updated_partition_manager.py
to demonstrate an auto-updating enabled partition manager.

Renamed the existing example script show_auto_update.py to
show_auto_updated_partition.py, for clarity.

* Docs: In the description of the list() methods of the resource manager
classes, described the optimized lookup behavior for auto-updated managers
and optimized access via the name-to-URI cache.

* In the NotificationReceiver class, added support for managing subscriptions
for topics dynamically with new methods 'subscribe()', 'unsubscribe()',
'is_subscribed()' and 'get_subscription()'.

^^^^^^^^^^^^^

This version contains all fixes up to version 1.7.3.

Released: 2023-05-16

**Incompatible changes:**

* The default value for the 'full_properties' parameter of the 'list()' method
of some zhmcclient resource types (Console, LDAPServerDefinition,
PasswordRule, User, UserPattern, UserRole, Task) has been changed from 'True'
to 'False' in order to improve performance. This change also affects the
set of properties of resources returned by 'find()' and 'findall()'.

In many cases, this is not an incompatible change since property access by
methods such as 'get_property()' or 'prop()' causes resource property
retrieval under the covers if the full set of properties had not been
retrieved in 'list()'.

However, there are also cases where this change is incompatible, for example
when accessing the resource properties via the 'properties' property. In such
cases, you need to change the call to 'list()' by specifying
'full_properties=True'. In cases where you had used 'find()' or 'findall()',
that parameter cannot be specified, and you need to fall back to using
'list()'.

**Bug fixes:**

* Addressed safety issues by increasing minimum versions of packages, where
possible.

* Changed use of 'method_whitelist' in urllib3.Retry to 'allowed_methods'.
The old method was deprecated in urllib3 1.26.0 and removed in 2.0.0.
Related to that, increased the minimum versions of urllib3 to 1.26.5 and of
requests to 2.25.0. Added urllib3 to the dependencies for installing zhmcclient,
because the indirect depndency of requests is not sufficient. (issue 1145)

* Fixed RTD docs build issue with OpenSSL version by providing a .readthedocs.yaml
file that specifies Ubuntu 22.04 as the build OS.

* Added trouble shooting info for urllib3 2.0 ImportError requiring
OpenSSL 1.1.1+.

* Increased dependent package jsonschema to >=3.0.1 to resolve dependency
issue with jupyter. (issue 1165)

**Enhancements:**

* Disabled the default retrieval of the full set of properties in list()
methods that was enabled by default, for the following resource types:
Console, LDAPServerDefinition, PasswordRule, User, UserPattern, UserRole,
Task. This provides a performance boost in cases where find() or findall()
is used with filters that can be handled by the HMC, because in such cases
the resource properties do not need to be retrieved.

* Added a 'pull_properties()' method to zhmcclient resource classes, that
performs a "Get Properties" HMC operation with the 'properties' query
parameter defined. This can be used to speed up certain property retrieval
operations, for example on the Console or on CPCs. (issue 862)

* Added a 'list_sibling_adapters()' method to the zhmcclient.Adapter class
that lists the other Adapter objects on the same physical adapter card.
Added end2end testcases for the new method.

* Test: Added end2end testcases for property retrieval.

* Added zhmcclient.GroupManager and zhmcclient.Group to support Group resources.
Group resources represent user-defined groups of resources; they can be used
for example in User Role permissions. Added zhmcclient mock support for
Group resources. Added testcases for both of that. (issue 1017)

* Enhanced export_dpm_configuration() to include Certificate objects.

* Introduced Certificate objects as new category of resources and added new
methods to assign/unassign Certificate objects to/from DPM mode partitions and
classic mode LPARs and activation profiles.

* Added two new methods Console.list_api_features() and
Cpc.list_api_features() and accompanying documentation to support the
new "API features" concept.

**Cleanup:**

* So far, the `Partition.hbas` property was set to `None` for CPCs that have the
"dpm-storage-management" feature enabled (i.e. starting with z14), because
HBAs are then represented as Virtual Storage Resource objects. For
consistency, this property was changed to provide an `HbaManager` object.
Since that property uses lazy initialization, there is no change at runtime
unless the property is actually accessed.

^^^^^^^^^^^^^

Released: 2023-03-26

**Incompatible changes:**

* export_dpm_configuration(): the default behavior when exporting the DPM
configuration has been changed to only include those adapters that are
referenced by other elements of the exported configuration data.
Old behavior is available by passing a new parameter to the function. (1115)

**Bug fixes:**

* Added the missing dependent packages for using the 'zhmcclient.testutils'
sub-package by adding a Paython package extra named 'testutils'. This is
only needed when performing end2end tests, or when using the example scripts.
The extra can be installed with 'pip install zhmcclient[testutils]'.

* Fixed incorrect list of managers in 'managers' attribute of zhmcclient
exception 'MetricsResourceNotFound' when a CPC was not found. (issue 1120)

**Enhancements:**

* Added missing test environments (Python >=3.6 on MacOS and Windows) to the
weekly full tests.

* Addressed issues reported by safety by increasing package versions. (1103)

* Test: Added more tools to missing requirements checking.

* export_dpm_configuration(): sorting result lists for more stable output