Gusty

`extra_callables`

`extra_callables` now available for `.py` files, allowing multiple callables to be passed to an underlying operator, for instance:

python3
---
operator: my.custom.operator
python_callable: main
extra_callables:
my_extra_operator_callable: my_other_func
---

def main():
return 'hello'

def my_other_func():
return 'world'


Where `my_extra_operator_callable` is the name of the argument in the operator and `my_other_func` is the name of the function in the task definition file.

Consider parent class `__init__` arguments

`apache-airflow-providers-common-sql` 1.3.0 allows for SQL-focused operators to no longer have a direct `sql` argument, which can break existing task definition files. The `__init__` method of parent class of SQL operators (e.g. `SqliteOperator`), now contains the `sql` argument.

gusty now considers parent class's `__init__` arguments.

Allows for the ["magic loop"](https://medium.com/apache-airflow/airflows-magic-loop-ec424b05b629) to be enabled in `create_dags`, where users can optionally pass through a `current_dag_id` string.

**Breaking Change** - The `globals` argument of `create_dags` has been renamed to `caller_env`.

Uses `external_dag_id` in wait_for external task name.

Bug fixes:

- `nested_update` ensures copies of dictionaries

- `METADATA.yml` is parsed used the default YAML parser

Adds a `leaf_tasks_from_dict` option to `create_dag` and `create_dags`. Useful if users want to have a uniform leaf task at the end of all DAGs.

Example:

python3
create_dags(
...,
leaf_tasks_from_dict={
"my_dag_is_done": {"operator": "airflow.operators.empty.EmptyOperator"}
}
)

- `multi_task_spec` can now update its base spec collections. This means top-level dictionaries in a spec can be partially or completely updated with their equivalent dictionaries in `multi_task_spec`.

- Updated README to include instructions on how to use `create_dags`

- Updated `dev-requirements.txt`