Earthmover

Latest version: v0.3.1

Safety actively analyzes 629994 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 3

0.3.1

<details>

<summary>Released 2024-04-26</summary>

* internal: allow any ordering of Transformations during graph-building in compile
* internal: only create a `/packages` dir when `earthmover deps` succeeds

</details>

0.3.0

<details>

<summary>Released 2024-04-17</summary>

* feature: add project composition using `packages` keyword in template file (see README)
* feature: add installation extras for optional libraries, and improve error logging to notify which is missing
* feature: `GroupByWithRankOperation` cumulatively sums record counts by group-by columns
* feature: setting `log_level: DEBUG` in template configs or setting `debug: True` for a node displays the head of the node mid-run
* feature: add `optional_fields` key to all Sources to add optional empty columns when missing from schema
* feature: add optional `ignore_errors` and `exact_match` boolean flags to `DateFormatOperation`
* internal: force-cast a dataframe to string-type before writing as a Destination
* internal: remove attempted directory-hashing when a source is a directory (i.e., Parquet)
* internal: refactor project to standardize import paths for Node and Operation
* internal: add `Node.full_name` attribute and `Node.set_upstream_source()` method
* internal: unify graph-building into compilation
* internal: refactor compilation and execution code for cleanliness
* internal: unify `Node.compile()` into initialization to ease Node development
* internal: Remove unused `group_by_with_count` and `group_by_with_agg` operations

</details>

0.2.1

<details>
<summary>Released 2024-04-08</summary>

* feature: [adding fromjson() function to Jinja](https://github.com/edanalytics/earthmover/pull/75)
* feature: [fix docs typos](https://github.com/edanalytics/earthmover/pull/68)
* feature: [`SortRowsOperation` sorts the dataset by `columns`](https://github.com/edanalytics/earthmover/pull/56)

</details>

0.2.0

<details>
<summary>Released 2023-09-11</summary>

* breaking change: remove `source` as Operation config and move to Transformation; this simplifies templates and reduces memory usage
* breaking change: `version: 2` required in Earthmover YAML files
* feature: `SnakeCaseColumnsOperation` converts all columns to snake_case
* feature: `show_progress` can be turned on globally in `config` or locally in any Source, Transformation, or Destination to display a progress bar
* feature: `repartition` can be turned on in any applicable `Node` to alter Dask partition-sizes post-execute
* feature: improve performance when writing Destination files
* feature: improved Earthmover YAML-parsing and config-retrieval
* internal: rename `YamlEnvironmentJinjaLoader` to `JinjaEnvironmentYamlLoader` for better transparency of use
* internal: simplify Earthmover.build_graph()
* internal: unify Jinja rendering into a single util function, instead of redeclaring across project
* internal: unify `Node.verify()` into `Node.execute()` for improved code legibility
* internal: improve attribute declarations across project
* internal: improve type-hinting and doc-strings across project
* bugfix: refactor SqlSource to be compatible with SQLAlchemy 2.x

</details>

0.1.6

<details>
<summary>Released 2023-07-11</summary>

* bugfix: [fixing a bug to create the results_file directory if needed](https://github.com/edanalytics/earthmover/pull/40)
* bugfix: [process a copy of each nodes data at each step, to avoid modifying original node data which downstreams nodes may rely on](https://github.com/edanalytics/earthmover/pull/41)

</details>

0.1.5

<details>
<summary>Released 2023-06-13</summary>

* bugfix: [fixing a bug to skip hashing missing optional source files](https://github.com/edanalytics/earthmover/pull/34)
* feature: [adding a tmp_dir config so we can tell Dask where to store data it spills to disk](https://github.com/edanalytics/earthmover/pull/37)
* feature: [adding a `--results-file` option to produce structured run metadata](https://github.com/edanalytics/earthmover/pull/35)
* feature: [adding a skip exit code](https://github.com/edanalytics/earthmover/pull/36)

</details>

Page 1 of 3

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.