Nornir-salt

CHANGES

1. SaltEventProcessor updated to use `datetime` module to add milliseconds to timestamps
2. Code refactoring, added `utils` folder to contain various functions for re-use across the code base
3. Updated `netmiko_send_commands`, `scrapli_send_commands`, `pyats_send_commands` and `napalm_send_commands` to use `cli_form_commands` utils function
4. Updated `netmiko_send_commands`, `scrapli_send_commands` to use `cli_send_command`s utils function
5. Code refactoring for `netmikko_send_config`, `scrapli_send_config`, `pyats_send_config`, `napalm_configure` using `cfg_form_commands` utils function
6. `http_call` task add `raise_fo_status` to requests response object to raise if experienced any errors
7. TabulateFormatter changed to honor `headers` argument for `terse and brief` `tabulate` argument values
8. For connections task changed the way how task name formed to make it more human readable and informative
9. `connections` task `conn_list` changed `connection_type` to `connection_plugin`, also plugin name extracted instead of <class xyz>
10. SaltEventProcessor no merges identity dictionary as is with events data, meaning can add any information into idenity dictionary and that will be emited on saltevent bus
11. Updated RetryRunner to handle multiple connections establishment for custom tasks `CONNECTION_NAME` variable or `connection_name` task parameter - can specify comma separated string to list connections to establish

BUGS

1. Fixed issue 2 bug by making sure to pop `connection_name` from task arguments within QueueRunner
2. Fixed docs spelling using pylint spelling check plugin (pyenchant) and sphinxcontrib-spelling RST plugin
3. Discovered issue in `scrapli-netconf` with determining if request failed, adjusted logic in `scrapli_netconf_call` task to account for it, raised [scrapli-netconf github issue]( https://github.com/scrapli/nornir_scrapli/issues/108)

FEATURES

1. Added utils `MakePlugin` function to emit boilerplate code for task and custom test function plugins.
2. Added `repeat, stop_patern, return_last` and `repeat_interval` support to `netmiko_send_commands` and `scrapli_send_commands plugins` to run commands certain amount of times
3. Added `conn_open` function to `connections` task plugin
4. Added to `ncclient_call` and `scrapli_netconf_call` tasks support for glob pattern `capab_filter` for `server_apabilities` function
5. Added RetryRunner task parameters - `run_connect_retry`, `run_task_retry`, `run_creds_retry`, `run_num_workers`, `run_num_connectors` to influence task execution logic on a per-task basis
6. Added support for `creds_retry` option to RetryRunner to specify a list of credentials to try connecting to devices.
7. Added FFun `FH` filter to filter by hostname values using glob patterns, similar to `FB` but for host's hostname parameter

FEATURES

1. `ToFileProcessor added `skip_failed` argument to control if want to save failed tasks
2. Added `add_host` argument to `TestsProcessor` custom test function to address github ticket 1 and allow passing host object to custom function for processing
3. FFun added `FM` - Filter platforM - filter hosts by platform names using glob patterns str or list of patterns
4. FFun added a list `FFun_functions` with names of all Fx function i.e. `FFun_functions = ["FB", "FP" ...etc]` - this is to facilitate kwargs filtering
5. Task `sleep`, added `sleep_random` timer support, e.g. `sleep_random=(1,5)` or `sleep_random=5` to sleep random time within given interval on a per-host basis
6. Added `TestsProcessor` `ContainsTest` support for `count_le` and `count_ge` arguments.

CHANGES

1. Updated `TestsProcessor` to not process/test results if has failed tasks
2. Updated `TestsProcessor` contains function to convert pattern to string
3. Updated `napalm_configure, scrapli_send_config, netmiko_send_config and pyats_send_config` to try and source config from `host.data.__task__["config"]`
4. PyATSUnicon - when making up connections options if "telnet" in platform name use protocol "telnet" instead of "shh"

BUGS

1. Fixed `RetryRunner` jumphost re-connection handling after closing connection to it
2. Fixed `RetryRunner` to not try connecting to device directly if connection via jumphost failed
3. Fixed `RetryRunner` jumphost intermittent connectivity handling bt no longer using host["exception"] to record errors
4. Improved `TestsProcessor` failed tasks handling making sure skip them and log an error if all tasks are failed for that particular test

BUGS

1. ResultSerialiser improved handling of non supported types for results by converting them to string as a lastage effort
2. Fixed `ncclient_call` `transaction` (previously named `locked`) function commit-confirmed handling

FEATURES

1. Added `PyATSUnicon` connection plugin to interact with devices using Genie library
2. Added `pyats_send_commands` task plugin to send commands over CLI to devices using Genie->PyATS->Unicon libraries, supporting onnections pools and output parsing
3. Added `pyats_send_config` task plugin to send config over CLI to devices using Genie->PyATS->Unicon libraries
4. Updated `HostsKeepalive` function to support pyats connection
5. Significantly improved `transaction` handling for `ncclient_call` and `scrapli_netconf_call` task plugins by checking device capabilities and using them to decide on additional steps such as validate or commit-confirmed
6. Added task plugin `pyats_genie_api` to call any Genie device api
7. Added `InventoryFun` function to interact with nornir inventory, these are the `InventoryFun` functions available:

- create_host/create
- update_host/update
- delete_host/delete
- read_host/read
- read_inventory
- load
- list_hosts

CHANGES

1. `ncclient_call` task renamed `locked` function to `transaction`
2. `scrapli_netconf_call` task renamed `locked` function to `transaction`

FEATURES

1. Added `commit_final_dealy` argument for `netmiko_send_config` task plugin to use with commit confirm feature.
2. Added `napalm_send_commands` task plugin to send commands to devices using NAPALM `cli` method
3. Added `ntfsm` parser function to `DataProcessor` to parse CLi output using TextFSM ntc-templates repository
4. Extended `parse_ttp` function to source templates from `ttp_templates` based on host platform and task command
5. Added new task `sleep` to command Nornir to sleep/pause for certain time
6. Added `terse` tabulate argument to TabulateFormatterto emit same columns as `brief` but in a more dense format to occupy less lines
7. Added `batch` keyword to `netmiko_send_config` task to send config commands in batches, useful if need to push big configurations
8. Added `SaltEventProcessor` to emit SaltStack events on task execution progress.

BUGS

1. When was using err_msg with eval test function and assert in expression, if assert fails, err_msg not used as exception, updated EvalTest logic to prefer using err_msg
2. `FFun` when supplying integer as a pattern was failing, made sure to convert pattern to string for all FFun functions

CHANGES

1. TestsProcessor EvalTest - improved err_msg handling by making it always preferred even when assert used.
2. `netmiko_send_config` no longer uses nornir-netmiko netmiko_send_config task, instead interacts with Netmiko connection directly.
3. Change logging level to be debug for `DataProcessor` if it fails to import third party libraries