Gusty

Latest version: v0.22.1

Safety actively analyzes 682361 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 4 of 9

0.12.0

- Moves `create_dags` to a top level import (and out of `gusty.experimental`)
- Adds logo

0.11.3

Bug fix ensures that `dag_constructors` take precedence over ABSQL default functions when there are naming collisions.

Removed empty collections as default arguments where applicable.

0.11.2

Ensures environment variable `AIRFLOW__CORE__DAGBAG_IMPORT_TIMEOUT` is a float if found.

0.11.1

- Allows for the special case of an `EmptyOperator` or `DummyOperator` in the `wait_for_class` parameter of `create_dag`
- Additional tests for `external_dependencies`
- Use `str.split()` for retrieving `airflow_version`

0.11.0

Adds a `create_dags` function, located in `gusty.experimental`, that allows users to create multiple DAGs from a single function.

By default, a simple `for` loop is used to create DAGs, but by setting `concurrent=True`, an asynchronous loop is used to create the DAGs, (hopefully) speeding up creation time.

Assuming multiple gusty DAG directories live behind `/usr/local/airflow/dags/gusty_dags` (e.g. `.../gusty_dags/dag_a`, `.../gusty_dags/dag_b`, so on), the concurrent implementation would look something like this:

python3
import airflow
from gusty.experimental import create_dags

create_dags(
"/usr/local/airflow/dags/gusty_dags",
globals(),
concurrent=True,
schedule_interval="0 12 * * *",
default_args={"email": "defaultgusty.com", "owner": "default"},
wait_for_defaults={"timeout": 679},
)


Additional parameters available when `concurrent=True`:

- `concurrent_timeout` - The maximum length (in seconds) that `create_dags` can run before a timeout error occurs. Defaults to 90% of [Airflow's `dagbag_import_timeout`](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#dagbag-import-timeout).

- `max_concurrency` - The maximum number of concurrent `create_dag` operations. Defaults to `8`.

0.10.1

Allows users to pass a custom `wait_for_class` to the `create_dag` function. The `wait_for_class` is expected to be a child class of Airflow's `ExternalTaskSensor`. Usage looks like:

python3
my_dag = create_dag(..., wait_for_class=MyCustomExternalTaskSensor)

Page 4 of 9

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.