Releases: microsoft/msticpy
Multi-timeline Interactive Visualization
New Features
- Major update to display_timeline control:
- allows arbitrary number of event series
- input as dict of data series or grouped DataFrame
- added interactive visual time range control
- added display_timeline_values to display timelines with a scalar value (line, circle, vbar)
- added sample notebook
- added ReadtheDocs page describing usage
- Moved to timeline.py module
Other Improvements/Additions
- Updated Base64Unpack, EventClustering, NotebookWidgets and TIProviders notebooks.
- Added unit test capability for UI-dependent packages by running notebooks within the unit test
- nbwidgets: added filtering text box to all select widgets
- nbwidgets: added SelectSubset widget allowing you to pick from one list and add to selected subset
- Updates to documentation/README.md
- Added checks for no TI Providers or missing keys and updated TIProvider docs for this.
- Added network data query yaml - kql_sent_winevent.yaml
- Added WinSecurityEvent.json events file
- Added pre-commit hooks including local hook script download_tlds.py
Fixes
- Fixes from testing notebook development:
- Minor change to base64unpack.py to prevent pandas warning
- entityschema: fixing repr to always return a string
- security_base: removing broken and deprecated properties adding repr
- ti_lookup - remove unneeded import
- nbwidgets - bug in restoring current index in selected items list
- eventcluster - first/last time range for clustered events was not properly calculated.
- Fixed foliummap error to display in notebook (implemented repr_html so that instances display directly in notebook.)
Azure Sentinel TI Provider
New Features
- Azure Sentinel Threat Intel provider in TILookup
- kql_base.py provider for TILookup to support other LogAnalytics TI sources
- Refactored unit tests for TIProviders with mocking of data sources.
- TIProviders notebook and ReadTheDocs TI Providers doc page.
- Added package config and ability for WSConfig to get workspace and tenant config from msticpyconfig.yaml
Fixes
- Fixes for mypy warnings - now mypy clean
- Addressed most other linting warnings
- Fixed broken multiple TI lookups for http providers
- Black formatting
- Bug in geoip that would throw exception for private IP addresses and issue warnings rather than exceptions if something goes wrong with GeoLite DB download
- Fixed errors in several network query definitions
- IoCExtract bug when trying to download TLD file offline
Enabling data library support
Some of the data library support was not included in the previous release. This is just catching up.
Data Query Library
New Features
- Data query library supporting multiple data providers (kqlmagic, Odata...).
- Query definitions are stored in YAML files in data/queries. Allowing default parameters and replaceable parameters at runtime.
Fixes
- Updates to IocExtract and base64unpack.
Minor README/Setup Updates
New Features
- Updated README with links to documentation and sample notebooks.
- Added links to code and documentation in setup.py allowing links to appear in the PyPI repo side menu.
Fixes
- Fixed a broken project url in setup.py.
Sphinx Documentation
New Features
Adding Sphinx documentation for Read the docs
Numpy docstrings should now be used (almost) everywhere
Added function to kql.py execute simple kql string query.
Added function to auditdextract.py to read audit logs from file.
Added these HowTo docs to repo:
- Jupyter And Security
- Azure Sentinel and Jupyter
- Enabling Auditd on Linux in Azure Sentinel
Fixes
Fixing some errors in iocextract
Pylint and Flake8 warnings (mostly reducing line length to < 90)
Adding test cases and correcting linting warnings
New Features
Added observationlist module
Fixes
Fixing linting warnings.
Adding unittests test_event_cluster, test_observationlist, test_security_event + test data
Miscellaneous small fixes and improvements.
Adding documentation notebooks
New Features
Added several doc notebooks for components
Updated Readme.md
Initial refactoring of docstrings to use numpy standard format.
Additional unit tests.
Fixes
Fixing some bugs found while doing the documentation.