Mostly housekeeping release.
We've introduced a set of friendly exceptions for common configuration-related problems that users might encounter while using msticpy in notebooks. In most cases, the cause is lack of a config item such as an API key or setting. Hitting one of these exceptions gives you a few lines of explanatory text plus links to documentation on readthedocs to help you fix it. In most cases the traceback is irrelevant so this is not shown for these exception types (although this can be re-enabled).
Adding some new capabilities like being able to use our time series analysis module on any time-stamped log data. We've also done a lot of documentation additions/improvements - for Anomalous Sequence, TimeSeries and msticpy configuration plus miscellaneous freshening and improving.
A lot of the other work is related to compatibility with nteract environment in Azure Machine Learning - this will soon be the default notebook environment for Azure Sentinel.
New Features
- Friendly exceptions and exception framework for notebooks 73
- Replaced the use of the ipywidgets Output widget with IPython updatable display() objects. nteract/AML compat. 71
- Added timeseries decomposition (using Statsmodel STL) so that you can do time series analysis on any data 69
(previously we relied on Kql/Azure Sentinel to do the decomposition/analysis part). Documentation and notebook added
- Update to Anomalous Sequence modules
- Added modellable_params argument in model classes - this can be used to override the default of using rough heuristics to
decide which params have modellable values, and instead, manually specify the parameters. 65
- Added sample notebook and RST/ReadtheDocs documentation for Anomalous Sequence.
- Added check_version() function that will check the current version against latest on PyPI
import msticpy
msticpy.check_version()
- Relaxing some version requirements in setup.py/requirements.txt to match AzureNotebooks resulting in quicker install 68
- Updated display of logon details in nbdisplay
- Updated README to add missing details, correct some things and add a few more images.
- Added documentation for Azure Sentinel configuration for notebooks - how to use the various config files.
- Updated a lot of the introductory sections to readthedocs to bring up-to-date and align with README
- Replace some badly-formatted tables in readthedocs docs
- Added analysis subpackage to docs so that Anomaly Sequence and TimeSeries module/API docs appear in the API doc tree
(readthedocs)
- Adding new queries for Notebooklets project 67
Fixes
- Some fixes to comp_reqs.py test tool to show missing packages and handle version comparison operators.
- New release of pandas/numpy surfaced a bug where we were doing datetime comparisons between timezone
naive and timezone-aware datetimes - fixed in timeline, processtree and eventcluster.
- New behavior in Kqlmagic (> 0.1.111) causes queries in the middle of a cell to output by default. Added workaround
to suppress this.
- Rogue __init__.py in root of repo (had been there forever) was preventing test cases using absolute imports of tested
modules. removed the offending file and updated all tests to use absolute imports
- Seems that we'd been inadvertently (at some point) including test files in our setuptools/PyPI package - these should
now be gone.
- Fixed an issue with GeoIPLite (maxmind) database download and cleaned up logic.
- Added better exception handling in nbinit
- Making pkginstall and notebook init errors more friendly
- Removing some deprecated terms.
- Fixing timestamp Timezone issue in process_tree_utils, eventcluster, process_tree and syslog_utils.cluster_syslog_logons_df
- Fixing/constraining some azure dependencies for Sphinx 68
- Fix for schema property in kql_driver 70