Descarteslabs

Catalog

- Added support for data storage. The `Blob` class provides mechanism to upload, index, share, and retrieve arbitrary byte sequences (e.g. files). `Blob`s can be searched by namespace and name, geospatial coordinates (points, polygons, etc.), and tags. `Blob`s can be downloaded to a local file, or retrieved directly as a Python `bytes` object. `Blob`s support the same sharing mechanisms as `Product`s, with `owners`, `writers`, and `readers` attributes.
- Added support to `Property` for `prefix` filtering.

Compute

- Added method to update user credentials for a `Function`.
- Added methods to retrieve build and job logs.

General

- Added support for Shapely=2.X.

- This is an internal-only release. There is as of yet no updated documentation. However, the user-facing client APIs remain fully compatible with v1.12.1.

Compute

- Added support for managed batch compute under the `compute` module.

Auth and Configuration

- Removed the check on the Auth for configuration, since it is all AWS all the time.

Raster Client

- Fixed a bug in the handling of small blocks (less than 512 x 512) that caused rasterio to generate bad download files (the desired image block would appear as a smaller sub-block rather than filling the resulting raster).

- This is an internal-only release. There is as of yet no updated documentation. However, the user-facing client APIs remain fully compatible with v1.12.1.

Supported platforms

- Deprecated support for Python 3.7 (will end of life in July).
- Added support for Python 3.10 and Python 3.11
- AWS-only client. For the time being, the AWS client can be used to communicate with the legacy GCP platform (e.g. `DESCARTESLABS_ENV=gcp-production`) but only supports those services that are supported on AWS (`catalog` and `scenes`). This support may break at any point in the future, so it is strictly transitional.

Dependencies

- Removed many dependencies no longer required due to the removal of GCP-only features.

Configuration

- Major overhaul of the internals of the config process. To prepare for supporting other clients using namespaced packages within the `descarteslabs` package, the top level has been cleaned up, and most all the real code is down inside `descarteslabs.core`. However end users should never have to import anything from `descarteslabs.core`. No more magic packages means that `pylint` will work well with code using `descarteslabs`.
- GCP environments only support `catalog` and `scenes`. All other GCP-only features have been removed.

Catalog

- A bug in the code handling the default context for image collections when working with a product with a CRS based on degrees rather than meters has been fixed. Resolutions should always be specified in the units used by the CRS.

Workflows

- Fixed a bug causing `descarteslabs.workflows.map.geocontext()` to fail with an import error. This problem
also affected the autoscaling feature of workflows map layers.

Catalog/Scenes/Raster

- Fixed a bug causing downloads of single-band images to fail when utilizing rasterio.

Catalog

- Catalog V2 is now fully supported on the AWS platform, including user ingest.
- Catalog V2 has been enhanced to provide substantially all the functionality of the Scenes API. The `Image` class now
includes methods such as `ndarray` and `download`. A new `ImageCollection` class has been added, mirroring `SceneCollection`.
The various `Search` objects now support a new `collect` method which will return appropriate `Collection` types
(e.g. `ProductCollection`, `BandCollection`, and of course `ImageCollection`). Please see the updated Catalog V2
guide and API documentation for more details.
- Previously, the internal implementation of the `physical_range` attribute on various band types was inconsistent with
that of `data_range` and `display_range`. It has now been made consistent, which means it will either not be set,
or will contain a 2-tuple of float values. It is no longer possible to explicitly set it to `None`.
- Access permissions for bands and images are now managed directly by the product. The `readers`, `writers`, and
`owners` attributes have been removed from all the `*Band` classes as well as the `Image` class. Also the
`Product.update_related_objects_permissions` and `Product.get_update_permissions_status` methods have been removed
as these are no longer necessary or supported.
- All searches for bands (other than derived bands) and images must specify one or more product ids in the filtering.
This requirement can be met by using the `bands()` and `images()` methods of a product to limit the search to that
product, or through a `filter(properties.product_id==...)` clause on the search.
- Products have a new `product_tier` attribute, which can only be set or modified by privileged users.
- The `Image.upload_ndarray` will now accept either an ndarray or a list of ndarrays, allowing multiple files per image.
The band definitions for the product must correspond to the order and properties of the multiple ndarrays.

Scenes

- With the addition of the Scenes functionality to Catalog V2, you are strongly encouraged to migrate your Scenes-based
code to use Catalog V2 instead. Scenes will be deprecated in a future release. Some examples of migrating from Scenes
to Catalog V2 are included in the Catalog V2 guide. In the meantime the Scenes API has been completely reimplemented
to use Catalog V2 under the hood. From a user perspective, existing code using the Scenes API should continue to
function as normal, with the exception of a few differences around some little-used dark corners of the API.
- The Scenes `search_bands` now enforces the use of a non-empty `products=` parameter value. This was previously
documented but not enforced.

Metadata

- With the addition of the Scenes functionality to Catalog V2, you are strongly encouraged to migrate your Metadata-based
code to use Catalog V2 instead. Metadata will be deprecated in a future release.
- As with Catalog and Scenes, one or more products must now be specified when searching for bands or images.

Raster

- The Raster client API now requires a `bands=` parameter for all rastering operations, such as `raster`, `ndarray`
and `stack`. It no longer defaults to all bands defined on the product.

DLTile

- An off-by-1/2-pixel problem was identified in the coordinate transforms underlying
`DLTile.rowcol_to_latlon` and `DLTile.latlon_to_rowcol`. The problem has been corrected,
and you can expect to see slight differences in the results of these two methods.

REST Clients

- All the REST client types, such as `Metadata` and `Raster`, now support `get_default_client()` and `set_default_client()`
instances. This functionality was previously limited to the Catalog V2 `CatalogClient`. Whenever such a client is required,
the client libraries use `get_default_client()` rather than using the default constructor. This makes it easy to
comprehensively redirect the library to use a specially configured client when necessary.

Geo package

- The `GeoContext` types that originally were part of the Scenes package are now available in the new `descarteslabs.geo` package,
with no dependencies on Scenes. This is the preferred location from which to import these classes.

Utils package

- The `descarteslabs.utils` package, added in the previous release for the AWS client only, now exists in the GCP client
as well, and is the preferred location to pick up the `DotDict` and `DotList` classes, the `display` and `save_image` functions,
and the `Properties` class for property filtering in Catalog V2.
- The `display` method now has added support for multi-image plots, see the API documentation for the `figsize`, `nrows`,
`ncols` and `layout_direction` parameters.

Property filtering

- The `property_filtering.GenericProperties` class has been replaced with `property_filtering.Properties`, but remains
for back compatibility.
- Property filters now support `isnull` and `isnotnull` operations. This can be very useful for properties which may or
may not be present, e.g. `properties.cloud_fraction.isnull | properties.cloud_fraction <= 0.2`.

Configuration and Authentication

- The `Config` exceptions `RuntimeError` and `KeyError` were changed to `ConfigError` exceptions
from `descarteslabs.exceptions`.
- `Auth` now retrieves its URL from the `Config` settings. If no valid configuration can be found,
it reverts to the commercial service (`https://iam.descarteslabs.com`).

General
- Dependencies for the descarteslabs library have been updated, but remain constrained to continue to support Python 3.7.
- Numerous bug fixes.

Installation

- The extra requirement options have changed. There are four extra requirement options now, `visualization`, `tables`,
`complete`, and `tests`. `visualization` pulls in extra requirements to support operating in a Jupyter notebook or
environment, enabling interactive maps and graphical displays. It is not required for operating in a "headless"
manner. `tables` pulls in extra requirements to support the `Tables` client. `complete` is the combination of
`visualization` and `tables`. `tests` pulls in extra requirements for running the tests. As always,
`pip install 'descarteslabs[complete]'` will install a fully enabled client.

Configuration

- The Descartes Labs client now supports configuration to support operating in different environments. By default,
the client will configure itself for standard usage against the GCP platform (`"gcp-production"`), except in the case of AWS Marketplace users, for whom
the client will configure itself against the AWS platform (`"aws-production"`).
Alternate environments can be configured by setting the `DESCARTESLABS_ENV` environment variable before starting python, or by using a prelude like

from descarteslabs.config import Settings
Settings.select_env("environment-name")

before any other imports of any part of the descarteslabs client package.
- The new AWS Enterprise Accelerator release currently includes only Auth, Configuration
and the Scenes client.

Auth and Exceptions

- The `descarteslabs.client.auth` package has moved to `descarteslabs.auth`. It is now imported
into the original location at `descarteslabs.client.auth` to continue to work with existing
code, but new code should use the new location.
- The `descarteslabs.client.exceptions` module has moved to `descarteslabs.exceptions`. It is
now imported into the original location at `descarteslabs.client.exceptions` to continue to
work with existing code, but new code should use the new location.

Scenes

- Fixed an issue in `scenes.DLTile.from_shape` where there would be incomplete coverage of certain geometries. The function may now return more tiles than before.
- Added support for the new `all_touched` parameter to the different `GeoContext` types. Default behavior remains the same
as always, but if you set `all_touched=True` this communicates to the raster service that you want the image(s) rastered
using GDAL's `CUTLINE_ALL_TOUCHED` option which will change how source pixels are mapped to output pixels. This mode is
only recommended when using an AOI which is smaller than the source imagery pixel resolution.
- The DLTile support has been fixed to avoid generating gaps when tiling regions that span
a large distance north-to-south and straddle meridians which are boundaries between
UTM zones. So methods such as `DLTile.from_shape` may return more tiles than previously,
but properly covering the region.
- Added support for retrieving products and bands.
- Methods added: `get_product`, `get_band`, `get_derived_band`, `search_products`,
`search_bands`, `search_derived_bands`.
- Disallows search without `products` parameter.
- Scaling support has been enhanced to understand processing levels for newer products. The
`Scene.scaling_parameters` and `SceneCollection.scaling_parameters` methods now accept
a `processing_level` argument, and this will be factored in to the determination of
the default result data type and scaling for all rastering operations such as `Scene.ndarray`
and `SceneCollection.mosaic`.
- If the user provides the `rasterio` package (which implies providing GDAL), then rasterio
will be used to save any downloaded images as GeoTIFF, allowing for the use of compression.
Otherwise, by default the `tifffile` support will be used to generate the GeoTIFF files
but compression is not supported in this mode.
- As the Places client has been deprecated, so has any use of the `place=` parameter supported
by several of the Scenes functions and methods.

Catalog

- (Core users only) Added support for specifying the image index to use when creating a new `Product`.
- Added support for defining per-processing-level `data_type`, `data_range`, `display_range`
and `physical_range` properties on processing level steps.

Discover

- Added support for filtering `Assets` by type and name fields.
- Supported filter types `blob`, `folder`, `namespace`, `sym_link`, `sts_model`, and `vector`. Specifying multiple types will find assets matching any given type.
- The name field supports the following wildcards:
- `*` matches 0 or more of any character.
- `?` matches 1 of any character.
- Find assets matching type of `blob` and having a display name of `file name.json` or `file2name.txt` but **not** `filename.json`:
- `Discover().list_assets("asset/namespace/org:some_org", filters="type=blob&name=file?name.*")`
- `Discover().list_assets("asset/namespace/org:some_org", filters=AssetListFilter(type=AssetType.BLOB, name="file?name.*"))`
- Find assets of type `blob` or `vector`:
- `Discover().list_assets("asset/namespace/org:some_org", filters="type=blob,vector")`
- `Discover().list_assets("asset/namespace/org:some_org", filters=AssetListFilter(type=[AssetType.BLOB, AssetType.VECTOR], name="file?name.*"))`

Metadata

- `Metadata.products` and `Metadata.available_products` now properly implement paging so that
by default, a DotList containing every matching product accessible to the user is returned.

Raster

- If the user provides the `rasterio` package (which implies providing GDAL), then rasterio
will be used to save any downloaded images as GeoTIFF, allowing for the use of compression.
Otherwise, by default the `tifffile` support will be used to generate the GeoTIFF files
but compression is not supported in this mode.

Tables

- Fixed an issue that caused a user's schema to be overwritten if they didn't provide a primary
key on table creation.
- Now uses Discover backend filtering for `list_tables()` instead of filtering on the client to
improve performance.
- `list_tables()` now supports filtering tables by name
- `Tables.list_tables(name="Test*.json")`

Tasks

- New Tasks images for this release bump the versions of several dependencies, please see
the Tasks guide for detailed lists of dependencies.

Workbench

- The new Workbench release bumps the versions of several dependencies.

Workflows

- Added support for the new `all_touched` parameter to the different `GeoContext` types.
See description above under `Scenes`.

General

- The Places client has been deprecated, and use thereof will generate a deprecation warning.
- The older Catalog V1 client has been deprecated, and use thereof will generate a deprecation
warning. Please use the Catalog V2 client in its place.
- Documentation has been updated to include the `AWS Enterprise Accelerator" release.
- With Python 2 far in the rearview mirror, the depedencies on the `six` python package have
been removed throughout the library, the distribution and all tasks images.