Nornir-salt

BUG

1. Fixed checks for lod_filter for DataProcessor for integer compare operations - le, ge, lt etc, as it was returning full results if value was string

BUGS

1. Fixed netmiko_send_commands and scrapli_send_commands task to ignore empty commands
2. Fixed `DataProcessor` lod_filter/key_filter glob check integers handling

FEATURES

1. Added `jmespath` function to `DataProcessor`
2. `DataProcessor` `find` function extended to support `xpath` and `jmespath` functions
3. Added connection plugin `PyGNMIPlugin` and `pygnmi_call` task plugin
4. `DataProcessor` added new filtering checks `eq, ge, gt, le ,lt, contains` for `find`, `lof_filter` and `key_filter` functions
5. Added `iplkp` function to `DataProcessor` to replace IP addresses with CSV or DNS lookup values

CHANGES

1. `http_call` updated to use more advanced logic to form target url and changed to assigns call method name to task name

BUGS

1. Fixed requirements

1. Added support for ``FB`` to accept list of patterns or comma separated string with patterns
2. Added ``FC`` - filter contains - new filter to filter based on containment
3. Added ``FR`` - filter regex - new filter to filter host names based on regex match
4. Added ``FN`` - filter negate - to introduce match negate, e.g. if ``FB="ABC*"`` and ``FN=True`` - will match all hosts that do NOT have "ABC*" pattern in their names
5. Added ``DataProcessor`` processor plugin with below functions to simplify Nornir results post-processing:

formatters - structured data to text:
- to_str
- to_json
- to_pprint
- to_yaml

loaders - text to structured data:
- load_xml
- load_json

filters:
- flake
- key_filter - filter dictionary keys
- xpath - XML xpath
- match - similar to include
- xml_flake - XML flatten key filter
- lod_filter - list of dictionaries filter
- find

transformers:
- xml_to_json
- xml_flatten
- xml_rm_ns
- path
- flatten
- unflatten

parsers:
- parse_ttp
- run_ttp

6. Added ``DumpResults`` function plugin to save full results of task execution to local file system
7. Added new task plugins to work with files saved by ``ToFileProcessor``: ``file_read, file_list, file_rm, file_diff, files``
8. Added ``HTTPPlugin`` connection plugin to query API over HTTP using requests
9. Added ``http_call`` task plugin to run any ``requests`` methods against HTTP API endpoints
10. Ncclient ``ncclient_call`` task plugin can now interact with devices behind SSH jumphosts if using ``RetryRunner``
11. Added ``sortby`` and ``reverse`` keywords to ``TabulateFormatter`` to sort table results
12. ``TabulateFormatter`` added new ``extend`` directive to form table out of list of dictionaries results
13. Added new connections tasks to handle connections: ``conn_list, conn_close, connections``

BUGS

1. Improved ``TestsProcesor`` to better handle errors in ``task_instance_completed`` method, without it, if exception happens during that method call worker thread halts and never responds
2. Fixed ``send_command_ps`` to raise ``TimeoutError`` instead of ``NetmikoTimeoutError``
3. Fixed ``DiffProcessor`` logic not to run if any of tasks failed

CHANGES

1. Removed formatting ``fmt`` option from ``ncclient_call`` and ``scrapli_netconf_call`` tasks - instead need to use ``DataProcessor`` to format/transform results
2. Removed ``FindString, ParseTTP, ToFile, TestFuncs`` functions from codebase as their functionality replaced with processors plugins

CHANGES

1. Test functions use ResultSerialiser output instead of Nornir results object, that is to be able to pass Nornir results between processes using queues
2. Renamed ``ContainsLinesTest`` ``lines`` argument to ``pattern`` to keep arguments uniform
3. Changes if empty FL list supplied, none of the hosts matched, i.e. supplying FL=[] will not match any host effectively stopping task execution.
4. ResultSerialized will return exception parameter set to None value instead of 'None' string if no exception was raised within task execution
5. Removed netmiko_kwargs argument from netmiko_send_commands task plugin, instead whatever additional supplied arguments used - **kwargs

FEATURES

1. Added support for new return structure to ResultSerialiser
2. RunTestSuite has support for brief output
3. Test functions' tabulate formatter has support for brief output
4. Added support for ``serialize`` argument to ``CustomFunctionTest`` function to opt for results type passed to it
5. ``ContainsTest`` has support for ``count`` argument to count the number of pattern ``occurrences``
6. ``RunTestSuite`` has support for ``print_results`` argument to colour print the results to terminal
7. Added new task netmiko_send_command_ps and extended netmiko_send_commands task to support use_ps
8. Added ToFile processor plugin, ToFile function will be deprecated later on
9. Added TestsProcessor plugin and support for ContainsTest, ContainsLinesTest, EqualTest, CustomFunctionTest, CerberusTest, EvalTest tests
10. Updated promptless mode call - use_ps can be a boolean or dictionary, if dictionary used as **kwargs for ps call
11. Added ncclient connection plugin
12. Added scrapli_netconf_call task plugin
13. Added ncclient_call task plugin
14. Added TabulateFormatter function to represent results in a text-table formatter
15. Made RetryRunner to add stats to result for connection and task execution run attempts
16. FFun added check_if_has_filter argument to check if has Fx filter supplied in kwargs - used by salt nornir proxy minion
17. Added ``scrapli_send_commands`` task plugin
18. Added netmiko_send_config task plugin
19. Modified netmiko_send_command task plugin to support split_lines keyword for use_ps mode to allow sending multi line strings to devices


BUGS

1. ResultSerialized was not returning results if underscored grouped task failed, changed conditions logic to account for such a case
2. ResultSerialized was returning failed task attempt instead of successful one, updated RetryRunner to set "skip_results" to True for previously failed attempts

FEATURES

1. Added ToFile function