Geofileops

Deprecations and compatibility notes

- worker processes are now being created using "forkserver" on linux. This solves risks
of deadlocks and the corresponding warning for that. Consequence is that also on linux
the `if __name__ == "__main__":` construct needs to be used in scripts. Some more info
can be found in the geofileops FAQ (675).
- `erase` was renamed to `difference`, as most other open source applications/libraries
use this terminology. `erase` just keeps existing for backwards compatibility for now,
but a warning is shown that it might be removed in the (distant) future. (595)
- In `copy_layer` and `append_to` the default `dst_layer` was, contrary to the
documentation, not the stem of the destination filename. This is corrected now. (648)
- In `copy_layer`, the `append` parameter is deprecated and replaced by the `write_mode`
parameter that accepts e.g. "append" as value (663).
- The `append_to` function is deprecated, as it is almost identical to `copy_layer` with
`write_mode="append"` (669).

Improvements

- Add function `apply_vectorized` to apply a vectorized function on a geometry column
and use it internally where possible (588, 594)
- Improve performance of `erase` and `intersection` for very complex input geometries.
This gives similar improvements for such datasets to `identity`,
`symmetric_difference` and `union`. (585, 601, 591, 614)
- Improve performance of `export_by_location` and `join_by_location` for simple queries
(548)
- Add support for `query=""` in `export_by_location` (597)
- Add support for GDAL vsi handlers in paths in most general file/layer operations (669)
- Add basic support for ".gpkg.zip" and ".shp.zip" files in most general file/layer
operations (673)
- Add support to rename columns and layers with only a difference in casing (549, 593)
- Use `ST_Equals` and add priority feature to `delete_duplicate_geometries` (638)
- Avoid integer overflow when gpkg written by geofileops is read from .NET (612)
- Speed up processing many small files, mainly on windows:
- reduce calls to `gdal.OpenEx` (622, 625, 677)
- improvements in sqlite3 code for 2 layer operations: start transactions
explicitly, remove obsolete GPKG triggers, use a :memory: temp file where possible
(626, 628, 630)
- Add support to pass a single string for all `column` type parameters if a single
column should be retained (523)
- Enable "CURVE" geometrytype files to be processed in the general file and
layer operations (558)
- Replace `append` parameter by `write_mode` in `copy_file` (663)
- Don't convert to multi-part geometries by default in `copy_layer`,... (570)
- In `add_column`, don't add column if update expression is invalid (650)
- Add configuration option to only warn on dissolve errors (561)
- For `dissolve`, apply `grid_size` within `union_all` (566)
- Add some pre-flight checks when geofileops is imported (573, 627)
- For `select_two_layers`, add the `gpkg_ogr_contents` table + fill out extents in the
`gpkg_contents` table in the output file (647)
- When using join_nearest with spatialite version >= 5.1,
show ST_distance between the two geometries instead of
the distance between the centroid of the two geometries (634)

Bugs fixed

- Fix `copy_layer` to a gpkg.zip file (604)
- Fix GDAL input open options being ignored in `copy_layer` (632)
- Fix `missing_ok` parameter in `remove` being ~ignored (605)
- Fix `dissolve` with `agg_columns` on sqlite 3.49.1 (636)
- Fix an invalid output .gpkg file being created when e.g. `copy_layer` is ran with an
invalid sql statement (641)
- Fix wrong results for `export_by_location` with queries != "intersects is True" (617)
- Fix listlayers returns layer name for a non-existing .shp (672)

Bugs fixed

- For `dissolve`, `agg_columns` aggregations sometimes give wrong results (541)

Deprecations and compatibility notes

- Set the default value of `keep_empty_geoms` to `False` for all standard operations.
This changes the default for `make_valid` and in some cases for `simplify`. The only
exception is `select`, where the default stays `True`. (472, 499)
- Up to now, geofileops always tried to create spatial indexes on output files. For some
formats this has disadvantages thought. Hence, and for consistency, from now on the
default behaviour regarding spatial index creation of GDAL will be followed. E.g.
default a spatial index for "GPKG", but no index for "ESRI Shapefile". (511)
- The default filter clause for `export_by_location` is now "intersects is True" while
in previous versions it was "intersects is True and touches is False", to be in line with `join_by_location`, other libraries and use for non-polygon data. (508)
- When `join_by_location` was applied, a column "spatial_relation" with the spatial
relation between the geometries was added. This is no longer the case. (475)

Improvements

- Add support for self-overlays in overlay operations (468)
- Add support for a spatial query in `export_by_location` (508)
- Improve `dissolve_within_distance` results (494)
- Improve performance of `join_by_location` for relation "intersects is True" and for
complex polygon features in layer to compare with (502, 519)
- Improve handling of queries evaluating to True for disjoint features in
`join_by_location` (509)
- Make fiona an optional depencency (301)
- Add configuration option GFO_REMOVE_TEMP_FILES that can be used to avoid temp files
being removed for debugging purposes (480)
- Add a context manager, TempEnv, to set temporary env variables (481)
- Don't copy both input files if only one is not spatialite based in two layer
operations (247)
- If an output_dir doesn't exist yet, avoid processing being done before an error is
raised. (518)
- Small changes to support geopandas 1.0 (529)
- Linting improvements: ruff version to 0.4.4 + apply import sorting (531)

Improvements

- Small changes to support gdal 3.9 (528)
- Several improvements to documentation: new FAQ, improved user guide,...
(465, 469, 474)
- Linting improvements: use mypy + use ruff-format instead of black for formatting
(478, 479)

Bugs fixed

- Fix crash when using e.g. `erase` with `gridsize <> 0.0` specified on input file
containing an EMPTY geometry (470)
- Fix `dissolve` possibly having EMPTY geometries as output if `gridsize <> 0.0` (473)
- Fix wrong results from `join_by_location` if ran on result of `join_by_location`
with `column_prefixes=""` (475)
- Fix error in two layer operations if equal column aliases used based on a constant or
a function result (477)
- Fix `erase` (and depending two layer operations like `union`,...) giving wrong results
if `subdivide_coords` < 1 (489)
- Fix two-layer operations with `gridsize` sometimes outputting NULL geometries (495)

Bugs fixed

- Fix error in `erase` if `erase_path` countains multiple layers (451)
- Fix error in `dissolve` on polygon input if a pass that is not the last one has 0
onborder polygons in its result (459, 461)
- Fix `groupby` parameter becoming case sensitive in `dissolve` when `agg_columns` is
also specified (462)

Improvements

- Add support to read/add/remove embedded layer styles in gpkg (263)
- Add `gridsize` parameter to most spatial operations (261, 407, 413)
- Add `keep_empty_geoms` and `where_post` parameters to many single layer spatial operations
(262, 398)
- Add `where_post` parameter to many two layer spatial operations (312)
- Add `columns`, `sql` and `where` parameters to `copy_layer` and `append_to` (311, 432)
- Add `dissolve_within_distance` operation (409)
- Add support for lang+ algorithm in `simplify` (334)
- Add support to use `select` and `select_two_layers` on attribute tables (= tables
without geometry column) and/or have an attribute table as result (322, 379)
- Add support to process all file types supported by gdal in most general file and layer
operations, e.g. `get_layerinfo`, `read_file`,... (402)
- Add support for files with Z/M dimensions in the general file and layer operations (369)
- Add support for spatialite 5.1 in `join_nearest` (412)
- Improve performance of `makevalid` and `isvalid` (258)
- Improve performance of `intersection`, 30% faster for typical data, up to 4x faster
for large input geometries (340, 358)
- Improve performance of `clip`: 3x faster for typical data (358)
- Improve performance of `export_by_location`, especially when `area_inters_column_name`
and `min_area_intersect` are `None`: a lot faster + 10x less memory usage (370)
- Improve performance of `erase`, `identity`, `symmetric difference` and `union` by
applying on-the-fly subdividing of complex geometries to speed up processing. The new
parameter `subdivide_coords` can be used to control the feature. For files with very
large input geometries, up to 100x faster + 10x less memory usage.
(329, 330, 331, 357, 396, 427, 438, 446)
- Improve performance of spatial operations when only one batch is used (271)
- Improve performance of single layer operations (375)
- Improve performance of some geopandas/shapely based operations (342, 408)
- Add checks that `output_path` must not be equal to the/an `input_path` for geo
operations (246)
- Follow geopandas behaviour of using shapely2 and/or pygeos instead of forcing pygeos
(294)
- Improve handling of "SELECT * ..." style queries in `select` and `select_two_layers`
(283)
- Improve handling + tests regarding empty input layers/NULL geometries (320)
- Improve logging: use geo operation being executed as `logger name` (410)
- Many small improvements to logging, documentation, (gdal)error messages,...
(321, 366, 394, 439,...)
- Use smaller footprint conda packages for tests (use `geopandas-base`, `nomkl`) (377)
- Use ruff instead of flake8 for linting (317)

Bugs fixed

- Fix parameter `index` in `to_file` being ~ ignored (285)
- Fix `fid` column in output having only null values in e.g. `union` (362)
- Fix "json" aggregate column handling in dissolve on line and point input files gives
wrong results (257)
- Fix error in `read_file` when `read_geometry=False` and `columns` specified (393)
- Fix error in `copy_layer`/`convert` with `explodecollections` on some input files
(395)
- Fix dissolve forcing output to wrong geometrytype in some cases (424)

Deprecations and compatibility notes

- Drop support for shapely1 (329, 338)
- Parameter `precision` of `makevalid` is renamed to `gridsize` as this is the typical
terminology in other libraries (273)
- When `join_nearest` is used with spatialite, >= 5.1, two additional mandatory
parameters need to be specified: `distance` and `expand` (412)
- Parameter `area_inters_column_name` in `export_by_location` now defaults to `None`
instead of "area_inters" (370)
- Deprecate `convert` and rename to `copy_layer` (310)
- Deprecate `split` and rename to `identity` (397)
- Deprecate `is_geofile` and `is_geofile_ext` (402)
- Make the `GeofileType` enum private, in preparation of probably removing it later on
(402)
- Remove the long-deprecated functions `get_driver_for_ext`, `to_multi_type` and
`to_generaltypeid` (276)
- Remove the long-deprecated `vector_util`, `geofileops.geofile` and
`geofileops.geofileops` namespaces (276)
- Remove `geometry_util`, `geoseries_util` and `grid_util` (339):
- Most functions were moved to `pygeoops` because they are generally reusable.
- Remaining functions are moved either to `_geometry_util` or `_geoseries_util` to
make it clearer they are not public.