MSTICPy

No entries match these filters. Clear filters

1. v3.1.0 Sep 21, 2026 · issue 082

   MSTICPy 3.1.0 adds a new ScanMalware TI provider for DNS, IP, and URL observables with no API key required.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v3.1.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v3.1.0

   • Adds the ScanMalware threat-intelligence provider supporting verdict lookups for hosts and URLs, passive-DNS context for IP addresses, and quick query modes for bulk lookups — no API credentials required.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Adds Python 3.14 to the supported package classifiers and CI test matrix.
   • Adds the ScanMalware threat-intelligence provider supporting verdict lookups for hosts and URLs, passive-DNS context for IP addresses, and quick query modes for bulk lookups — no API credentials required.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   1 more in Everything

   └──▷ BREAKING ON UPGRADE

   • !NumPy 1.x is no longer supported by the declared runtime requirements — review environment constraints for NumPy 2 compatibility before upgrading.
   • !Code that directly indexes the dictionary returned by mp_ua_header() must replace ["UserAgent"] with ["User-Agent"]; code that passes the complete dictionary as HTTP headers does not need to change, and code needing the raw identifier can use MSTICPY_USER_AGENT.
2. v3.0.1 May 11, 2026 · release notes

   MSTICPy v3.0.1 adds path traversal protection for archive extraction across GeoIPGeoIPA database that maps IP addresses to geographic locations; tools use it to attribute traffic or alerts to countries, cities, or ASNs without querying external services., Mordor, and Sentinel query pack downloads.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v3.0.1 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v3.0.1

   • Adds centralized msticpy.common.archive_utils module that validates archive member paths before extraction, preventing tar-slip/zip-slip attacks when extracting GeoIP databases, Mordor datasets, and Sentinel query packs.

   1 detail in Everything

3. v3.0.0 Apr 28, 2026 · release notes

   MSTICPy v3.0.0 adds OpenObserve data provider, Sentinel certificate auth, and Python 3.13 support.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v3.0.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v3.0.0

   └──▷ USE IT
   Query an OpenObserve instance for error logs in the last 24 hours — useful for integrating OpenObserve into existing MSTICPy investigation notebooks.
   python copy

   import msticpy as mp
   mp.init_notebook()
   
   qry_prov = mp.QueryProvider("OpenObserve")
   qry_prov.connect(
       connection_str="https://localhost:5080",
       user="admin",
       password="[PLACEHOLDER]",
       verify=True,
   )
   
   df = qry_prov.exec_query(
       'search "error"',
       days=1,
       limit=100,
       timeout=120,
   )

   Use Cybereason with rate-limit retry and pagination control to reliably pull large result sets from high-volume environments.
   python copy

   qry_prov = mp.QueryProvider("Cybereason")
   qry_prov.connect(
       instance="cr1",
       tenant_id="...",
       client_id="...",
       client_secret="[PLACEHOLDER]",
   )
   
   df = qry_prov.exec_query(
       "{}",
       page_size=100,
       timeout=120,
       retry_on_error=True,
   )

   Configure OpenObserve credentials in msticpyconfig.yaml, storing the password in Key Vault, before connecting from any notebook.
   yaml copy

   DataProviders:
     OpenObserve:
       Args:
         connection_str: "https://your-openobserve-host:5080"
         user: "your-username"
         password:
           KeyVault:
         verify: true
         timeout: 300

   • New OpenObserve data provider enables querying OpenObserve instances directly via mp.QueryProvider('OpenObserve'), with support for time-range parameters, result limits, timezone config, and custom timeouts.
   • Microsoft Sentinel connections now support certificate-based authentication as an alternative to client secrets, improving security for automated pipelines and service accounts.
   • Cybereason driver gains automatic HTTP 429 rate-limit retry with backoff, additional query parameters (page_size, timeout, retry_on_error), improved timeout handling, and better error reporting via MsticpyDataQueryError.
   • Cybereason driver's data flattening (_flatten_simple_values, _flatten_element_values) reworked for more reliable output, with extra fields now mapped to the result DataFrame.
   • The ThreatIntelIndicators BYOTI provider now supports both old and new table schemas, enabling seamless operation across Sentinel workspaces with either schema version.

   +3 more show less

   • Configurable TI confidence threshold — now a class constant that can be overridden rather than a hardcoded value.
   • New KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. timespan conversion utilities for converting KQL timespan strings to Python timedelta objects using azure-kusto-data's parse_timedelta.
   • Python 3.13 is now fully supported.

   8 details in Everything

   └──▷ BREAKING ON UPGRADE

   • !Python 3.8 and 3.9 are no longer supported; supported versions are 3.10, 3.11, 3.12, and 3.13.
   • !The msrestazure package is removed; code catching msrestazure.azure_exceptions.CloudError must be updated to catch azure.core.exceptions.HttpResponseError instead.
   • !The kql install extra (pip install msticpy[kql]) is removed; KqlmagicCustom is no longer a supported dependency.
   • !The aiagents install extra (pip install msticpy[aiagents]) is removed; the autogen-agentchat integration is no longer included.
   • !The sql2kql extra now requires mo-sql-parsing>=11,<12 (previously >=8,<9); environments using sql_to_kql must be updated.
4. v2.17.0 Sep 25, 2025 · release notes

   MSTICPy v2.17.0 adds RRCF outlier detection, expanded Prisma Cloud AWS coverage, and OAuth2OAuth2An open authorization framework (RFC 6749) that lets a service grant limited access to a user's resources on another service without exposing credentials, giving cyber tools a standard way to authenticate against third-party APIs. for Defender providers.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v2.17.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v2.17.0

   • Adds RRCF (Random Cut Forest) outlier detection for ensemble-based anomaly scoring on telemetry data.
   • Expands Prisma Cloud AWS detection coverage for broader cloud security analysis.

   2 details in Everything

   └──▷ BREAKING ON UPGRADE

   • !Defender provider auth will fail until msticpyconfig.yaml scopes are updated to OAuth2 .default values — legacy non-OAuth2 or incorrect scope names are no longer accepted.
5. v2.16.0 Feb 21, 2025 · release notes

   MSTICPy v2.16.0 adds Prisma Cloud data driver, Cyberint TI provider, and certificate auth for OData drivers

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v2.16.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v2.16.0

   • Adds PrismaCloudDriver for querying Prisma Cloud assets, configurations, and events via its CSPMCSPMCloud security posture management, checking cloud resources for risky settings and policy violations. It helps teams find exposed services, overly broad permissions and other configuration gaps. API within MSTICPy workflows.
   • Adds Cyberint as a new Threat Intelligence provider for IoCIoCAn indicator of compromise, an observable clue that may point to malicious activity, such as a suspicious file hash or domain. It is evidence to investigate, rather than proof by itself. lookup via the Cyberint API.
   • Adds certificate authentication support for OData-based drivers (Microsoft Defender and Microsoft GraphMicrosoft GraphMicrosoft's unified REST API for accessing data and services across Microsoft 365, Azure AD, and related cloud products, giving cyber tools a single authenticated endpoint to query users, emails, files, and sign-in logs.).
   • Adds experimental Azure Monitor search driver with support for custom tables (tracked as issue #831).

   4 details in Everything

6. v2.15.0 Dec 6, 2024 · release notes

   MSTICPy v2.15.0 adds multi-dimensional outlier visualizations and federated auth improvements for M365D/Graph providers.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v2.15.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v2.15.0

   └──▷ USE IT
   Detect and visualize outliers across multiple numeric features — useful when hunting anomalies in multi-variate log telemetry without collapsing to a single score.
   python copy

   from msticpy.analysis.outliers import identify_outliers, plot_outlier_results
   import numpy as np
   
   data = np.random.rand(100, 7)
   feature_columns = [f'feature{i}' for i in range(1, 8)]
   
   clf, X_outliers, y_pred_outliers = identify_outliers(data, data, contamination=0.1, max_features=0.4)
   
   plot_outlier_results(
       clf,
       data,
       data,
       X_outliers,
       feature_columns=feature_columns,
       plt_title="Isolation Forest Anomaly Detection for Multi Dimension Features"
   )

   • Adds multi-dimensional plot support to analysis.outliers via plot_outlier_results, enabling visual interpretation of Isolation Forest anomaly detection across N-dimensional feature sets.
   • Improves federated authentication (delegated auth) support in the MDATP/Defender data provider, with cleaned-up code alongside the existing documentation.

   2 details in Everything

7. v2.14.0 Oct 21, 2024 · release notes

   MSTICPy v2.14.0 adds declarative user session config, nested-dict DataFrame expansion, and HTTP 429 retry control.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v2.14.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v2.14.0

   └──▷ USE IT
   Bootstrap a notebook's full provider stack from a YAML file instead of writing repetitive connect calls — useful in automation or shared notebook templates.
   python copy

   import msticpy as mp
   mp.init_notebook()
   mp.load_user_session("my_config.yaml", verbose=True)
   # qry_prov_sent, qry_prov_md, mssentinel now available as variables

   Flatten a column of nested JSON/dicts (e.g. raw API responses) into analysis-ready tabular columns and rows in one call.
   python copy

   expanded_df = raw_df.mp_pivot.dict_to_dataframe(col="my_nested_column")

   • Adds mp.load_user_session() to declaratively initialize and connect multiple data providers and components from a YAML file, publishing them directly into the notebook namespace.
   • Adds mp_pivot.dict_to_dataframe(col=...) pandas extension method to recursively expand nested dictionaries and lists in a DataFrame column into multiple columns and rows.
   • Adds a max-retry parameter to prevent HTTP 429 (rate-limit) errors during query execution.
   • Supports Maxmind GeoIPLite's new AccountID-based download authentication mechanism via msticpyconfig.yaml.

   4 details in Everything

8. v2.13.0 Sep 13, 2024 · release notes

   MSTICPy v2.13.0 adds an LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-powered document Q&A agent via Autogen/RAGRAGRetrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. and a new BinaryEdge threat-intel provider.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v2.13.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v2.13.0

   └──▷ USE IT
   Ask natural-language questions about MSTICPy from a notebook — useful when onboarding or exploring unfamiliar query provider configuration.
   python copy

   %load_ext msticpy.aiagents.mp_docs_rag_magic
   
   %%ask
   What are the three things that I need to connect to Azure Query Provider?

   • Adds an LLM/RAG-based document Q&A agent (msticpy.aiagents.mp_docs_rag_magic) powered by Autogen, enabling natural-language queries against MSTICPy documentation inside notebooks.
   • Adds a new Threat Intelligence provider for BinaryEdge, expanding TI lookup coverage.
   • Adds upload data styles to the Splunk uploader.

   3 details in Everything

9. v2.12.0 May 10, 2024 · release notes

   MSTICPy v2.12.0 adds MSSentinel key support, Args subkey for auth params, and JWTJWTAn open standard (RFC 7519) for encoding claims as a signed, optionally encrypted JSON object passed between parties. Cyber tools use it to verify identity and permissions without querying a central session store. expiry checks for Splunk.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v2.12.0 https://github.com/microsoft/msticpy.git
   # already have the repo? check out this version:
   $ git checkout v2.12.0

   └──▷ USE IT
   Pass explicit authentication tokens directly to the MSSentinel query provider connect method.
   code copy

   query_provider.connect(client_id='<client_id>', client_secret='<secret>')

   • msticpyconfig.yaml now supports the MSSentinel key as an alias for AzureSentinel
   • WorkspaceConfig entries in msticpyconfig.yaml support an Args subkey to pass authentication parameters like client_id to query_provider.connect()
   • The MSSentinel QueryProvider now accepts explicit connection parameters (e.g. client_secret) via the connect() method instead of relying solely on config keys

   3 details in Everything

10. v2.11.0 Mar 25, 2024 · release notes

    Adds fail_if_partial parameter to Sentinel exec_query to enforce failure on partial results

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.11.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.11.0

    └──▷ USE IT
    Ensure your analysis completes fully by failing the query immediately if it hits a pagination limit.
    python copy

    qry_prov.exec_query(query_string, fail_if_partial=True)

    • Adds fail_if_partial parameter to the Sentinel query provider's exec_query method to raise an exception when queries return partial results
    • Introduces a warning for Kusto/Sentinel queries that return partial results in split queries to prevent silent data loss

    2 details in Everything

11. v2.10.0 Feb 12, 2024 · release notes

    Adds VirusTotal search queries and SplunkUploader token auth support in v2.10.0

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.10.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.10.0

    • VirusTotalQueries class provides Virus Total Search Capabilities
    • Config key authentication.token in msticpyconfig.yaml enables security token auth for SplunkUploader

    2 details in Everything

12. v2.9.0 Nov 15, 2023 · release notes

    Adds IPQualityScore threat intel provider and Graph-based Defender Advanced Hunting via M365DGraph environment.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.9.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.9.0

    • Adds IPQualityScore as a new TI provider to query IP reputation statistics
    • Adds support for Microsoft Defender Graph APIGraph APIMicrosoft's unified REST API for accessing data across Microsoft 365 services — mail, calendar, users, files — letting cyber tools query or monitor cloud environments through a single authenticated endpoint. by using the M365DGraph DataEnvironment name in the data provider

    2 details in Everything

13. v2.8.0 Oct 4, 2023 · release notes

    MSTICPy v2.8.0 adds Splunk bearer tokenbearer tokenA credential that grants access to whoever presents it, without a separate proof of possession. Protecting the token matters because a stolen copy can be used by someone else. auth, Cybereason paging, and YAML validation.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.8.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.8.0

    • Add bearer token auth to splunk driver - Supports authentication via security token instead of username/password
    • Cybereason pagination support + multi-threading - Handles large CyberReason queries with paging
    • Validation of the YAML structure of query files - Validates query yaml file structure

    3 details in Everything

14. v2.7.0 Aug 15, 2023 · release notes

    MSTICPy v2.7 adds CrowdSec and AbuseIPDB TI providers, a new Query Editor UI, and native Azure data connectors.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.7.0

    └──▷ USE IT
    Initialize the new query editor to manage KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. templates.
    python copy

    from msticpy.nbtools import QueryEditor
    QueryEditor()

    • Adds CrowdSec Threat Intelligence provider for malicious IP lookups
    • Adds AbuseIPDB Threat Intelligence provider for malicious IP address abuse lists
    • Introduces an ipywidgets-based QueryEditor to edit and manage KQL query templates with parameter definitions and metadata
    • Enables parallel query execution across multiple workspaces by splitting large queries into time chunks via the split_query_by parameter in new MSSentinel/Kusto drivers
    • Allows passing an AzureCredential object directly to the credential parameter of AzureData.connect() and MicrosoftSentinel.connect() methods for authentication

    +2 more show less

    • Supports specifying sovereign clouds via the cloud parameter in AzureData.connect() and MicrosoftSentinel.connect() methods
    • Adds delete_watchlist_item method to the MS Sentinel API integration

    7 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Kqlmagic dependencies are no longer installed by default; install with pip install msticpy[kql] if needed
    • !Default data environment identifiers changed to MSSentinel and Kusto (replacing MSSentinel_New and Kusto_New); use MSSentinel_Legacy or Kusto_Legacy for previous Kqlmagic-based drivers
15. v2.6.0 Jul 25, 2023 · release notes

    MSTICPy v2.6.0 adds parallel multi-instance querying and a Velociraptor local data provider

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.6.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.6.0

    └──▷ USE IT
    Run queries across multiple Sentinel workspaces in parallel using the new add_connection and max_threads capabilities.
    python copy

    qry_prov = QueryProvider("MSSentinel", max_threads=10)
    qry_prov.connect(workspace="WS1")
    qry_prov.add_connection(workspace="WS2", alias="WS2")
    results = qry_prov.exec_query(query="SecurityAlert | take 5")

    Load and query local Velociraptor forensic logs using the new VelociraptorLogs provider.
    python copy

    qry_prov = QueryProvider("VelociraptorLogs", data_paths=["~/my_logs"])
    qry_prov.connect()
    tables = qry_prov.list_queries()
    process_info = qry_prov.vc_prov.velociraptor.Windows_Forensics_ProcessInfo()

    • QueryProvider.add_connection() enables running queries across multiple Sentinel/Defender/Splunk instances in parallel.
    • max_threads parameter on QueryProvider controls concurrent thread count (default 4, configurable via init).
    • VelociraptorLogs data environment added to read local exported Velociraptor logs into pandas DataFrames.

    3 details in Everything

16. v2.5.0 May 30, 2023 · release notes

    MSTICPy v2.5.0 adds OSQuery and new MSSentinel/Kusto drivers with plugin extensibility.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.5.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.5.0

    └──▷ USE IT
    Side-load custom threat intelligence providers from specific directories.
    code copy

    import msticpy as mp
    mp.load_plugins(plugin_paths=["./my_custom_providers", "./shared_plugins"])

    Import OSQuery forensic logs from local directories into a QueryProvider.
    code copy

    qry_prov = mp.QueryProvider("OSQueryLogs", data_paths=["/var/osquery/logs"])
    qry_prov.connect()
    df = qry_prov.osquery.processes()

    Download hunting queries from the MS Sentinel repository to local YAML files.
    code copy

    from msticpy.data.drivers.sentinel_query_reader import download_and_write_sentinel_queries
    download_and_write_sentinel_queries(query_type="Hunting", yaml_output_folder="./sentinel_hunt")

    • mp.QueryProvider("OSQueryLogs") enables importing OSQuery JSON logs into dataframes for forensic analysis
    • mp.load_plugins(plugin_paths=...) allows side-loading private data, TI, and context providers alongside built-in ones
    • New MSSentinel_New provider uses azure-kusto-data SDK and supports multi-threaded parallel queries across multiple workspaces
    • download_and_write_sentinel_queries(query_type="Hunting") downloads detection/hunting queries from the Sentinel repo for use with MSTICPy
    • Panel Tabulator is now available as a data viewer control, configurable via initialization parameters

    5 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !The mp_az_auth parameter is deprecated in favor of auth_types (will be removed in a future release)
    • !The mp_az_auth_tenant_id parameter is deprecated and unsupported in new providers, replaced by tenant_id
17. v2.4.0 Apr 7, 2023 · release notes

    MSTICPy v2.4.0 adds Pulsedive Threat Intel provider, Bokeh 3.x compatibility, and Python logging controls.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.4.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.4.0

    └──▷ USE IT
    Query Pulsedive using custom threat intelligence filters via the new provider.
    code copy

    from msticpy.sectools import ThreatIntLookup
    pdlookup = ThreatIntLookup()
    pdlookup.init(provider='Pulsedive')
    pddetail = pdlookup.explore(query="ioc=pulsedive.com or threat=AgentTesla")

    Enable detailed logging for debugging MSTICPy functions like init_notebook.
    code copy

    import msticpy as mp
    mp.set_logging_level("INFO")

    • Adds PulsediveProvider with standard TI methods and custom explore(query=...) and scan(observable=...) functions for Pulsedive data queries
    • Enables compatibility with Bokeh 3.x libraries despite default requirements still listing Bokeh 2.4.3
    • Adds %save_to_cell IPython magic to export Python objects (e.g., DataFrames) as base64-encoded blobs in new notebook cells
    • Adds set_logging_level() function and MSTICPYLOGLEVEL / MSTICPYLOGFILE environment variables for centralized Python logging control

    4 details in Everything

18. v2.3.1 Feb 24, 2023 · release notes

    Adds progress parameter to Threat Intel lookup functions and removes auto-displayed license info in geoip.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.3.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.3.1

    └──▷ USE IT
    Suppress noisy progress bars when running Threat Intel lookups in batch or looped workflows.
    code copy

    tilookup.lookup_iocs(data, progress=False)

    • tilookup.lookup_iocs() now accepts a progress=False argument to suppress the progress bar output.
    • The geoip module no longer automatically displays license information during execution.

    2 details in Everything

19. v2.3.0 Jan 25, 2023 · release notes

    MSTICPy v2.3.0 adds Sentinel Dynamic Summaries, dynamic parameterized queries via QueryProvider.add_custom_query, and query search

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.3.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.3.0

    └──▷ USE IT
    Create and persist a Sentinel summary to archive automated notebook results for later retrieval.
    code copy

    sentinel.connect()
    sentinel.create_dynamic_summary(
        name='My_XYZ_Summary',
        description='Summarizing the running of the XYZ job.',
        data=summary_df,
        tactics=['discovery', 'exploitation'],
        techniques=['T1064', 'T1286'],
        search_key='host.domain.dom',
    )

    Define and register a reusable parameterized KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. query in a notebook without editing static query files.
    code copy

    qry_prov = mp.QueryProvider('MSSentinel')
    query = """
    SecurityEvent
    | where EventID == {event_id}
    | where TimeGenerated between (datetime({start}) .. datetime({end}))
    | where Computer has "{host_name}"
    """
    qp_host = qry_prov.Param('host_name', 'str', 'Name of Host')
    qp_start = qry_prov.Param('start', 'datetime')
    qp_end = qry_prov.Param('end', 'datetime')
    qp_evt = qry_prov.Param('event_id', 'int', None, 4688)
    qry_prov.add_custom_query(
        name='get_host_events',
        query=query,
        family='Custom',
        parameters=[qp_host, qp_start, qp_end, qp_evt]
    )

    Find all queries in a provider that match specific table names or parameter keywords.
    code copy

    matches = qry_prov.search(
        search='syslog',
        table='Syslog',
        param='hostname'
    )

    • Adds sentinel.list_dynamic_summaries(), create_dynamic_summary(), get_dynamic_summary(), and new_dynamic_summary() to manage Microsoft Sentinel dynamic summaries in preview
    • Adds DynamicSummary class (importable from msticpy.context.azure.sentinel_dynamic) for manipulating summary objects with methods like to_df()
    • Adds QueryProvider.add_custom_query() API to register ad-hoc parameterized queries at runtime with defined parameters
    • Adds QueryProvider.Param constructor for defining query parameter metadata (name, type, description, default)
    • Adds QueryProvider.search() method for finding queries by name, properties, table, or parameter using regex-like terms

    5 details in Everything

20. v2.2.0 Nov 29, 2022 · release notes

    MSTICPy v2.2.0 adds ServiceNow context provider, IoCIoCAn indicator of compromise, an observable clue that may point to malicious activity, such as a suspicious file hash or domain. It is evidence to investigate, rather than proof by itself. de-fanging, and Python 3.11 support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.2.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.2.0

    └──▷ USE IT
    Query ServiceNow for threat intelligence context by IP or email using the new generic provider.
    python copy

    context_lookup = ContextLookup()
    result = context_lookup.lookup_observable('10.0.0.1', providers=['ServiceNow'])

    Configure the ServiceNow context provider with tenant and auth details in the MSTICPy config file.
    yaml copy

    ContextProviders:
      ServiceNow:
        Primary: True
        Args:
          TenantId: <tenant_id>
          AuthKey: 'auth_key'
          AuthId: 'auth_id'
        Provider: "ServiceNow"

    Defang or refang IOCsIOCsAn indicator of compromise, an observable clue that may point to malicious activity, such as a suspicious file hash or domain. It is evidence to investigate, rather than proof by itself. using the new utility functions.
    python copy

    from msticpy.common.utility import defang_ioc, refang_ioc
    
    defanged = defang_ioc('http://evil.com')
    refanged = refang_ioc('hXXp://evil[.]com')

    • Adds ContextProviders config section in msticpyconfig.yaml with a ServiceNow provider for querying ServiceNow via the new generic context lookup API.
    • Adds defang_ioc and refang_ioc utility functions in msticpy.common.utility to handle common de-fanging markup ([.], AT, hXXp) in IoCs.
    • Adds support for Python 3.11 to the MSTICPy library.
    • Adds GCC and Regional Cloud support to MDE/M365 data providers for government cloud customers.

    4 details in Everything

21. v2.1.5 Nov 1, 2022 · release notes

    MSTICPy v2.1.5 allows Process Tree viewer to accept process GUIDs as ProcessID

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.5 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.1.5

    └──▷ USE IT
    Enable process tree visualization using standard Windows process GUIDs instead of parsing hex or decimal integers.
    code copy

    process_tree(ProcessID='{123e4567-e89b-12d3-a456-426614174000}')

    • process_tree now accepts GUID format for ProcessID and ParentProcessID

    1 detail in Everything

22. v2.1.4 Oct 25, 2022 · release notes

    Adds create_* API ID returns and init_notebook config param for custom settings files in MSTICPy v2.1.4

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.4 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.1.4

    └──▷ USE IT
    Initialize the notebook session using a specific custom configuration file instead of defaults.
    python copy

    mp.init_notebook(config="~/configs/all_ti_provs.yaml")

    • MicrosoftSentinel connect methods now default to Default workspace or the name supplied as workspace parameter without requiring explicit configuration for those cases.
    • Sentinel create_* APIs (for incident, bookmark, analytic, and watchlist creation) now return the ID of the newly created item instead of None.

    2 details in Everything

23. v2.1.3 Oct 6, 2022 · release notes

    MSTICPy v2.1.3 adds mp_plot_process schema argument and MS Sentinel query retrieval support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.3 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.1.3

    └──▷ USE IT
    Map custom log fields to the process tree schema using a dictionary
    code copy

    cust_schema = { 'process_name': 'ImageFileName', 'process_id': 'PID', 'parent_id': 'PPID', 'time_stamp': 'CreateTime' }
    df.mp_plot_process(schema=cust_schema)

    • Adds schema parameter to mp_plot_process and related process tree functions to accept dictionary mappings instead of only ProcSchema objects
    • Adds capability to retrieve stored queries from an MS Sentinel workspace via a new API integration

    2 details in Everything

24. v2.1.0 Sep 13, 2022 · release notes

    MSTICPy v2.1.0 adds Malware Bazaar provider, df.mp.whois() accessor, Synapse notebook support, and extended Azure auth types

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.1.0

    • df.mp.whois() pandas accessor for bulk whois queries from a dataframe

    1 detail in Everything

25. v2.0.0 Jul 7, 2022 · release notes

    MSTICPy v2.0 introduces new mp.search, plot_map, async TI lookups, and msticpyconfig.yaml support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0

    └──▷ USE IT
    Find the correct import path for a TI provider after the v2 reorganization
    code copy

    import msticpy as mp
    mp.search("riskiq")

    Perform fast asynchronous lookups across multiple configured TI providers for a list of indicators
    code copy

    ti_lookup.lookup_iocs(["192.168.1.1", "evil.com"])

    Initialize MSTICPy using a custom configuration file located in a project-specific directory
    code copy

    import msticpy as mp
    mp.init_notebook(config="/path/to/project/msticpyconfig.yaml")

    • Adds mp.search() function to search for modules across the reorganized package structure
    • Adds plot_map function in msticpy.vis.foliummap for direct DataFrame-to-map plotting with layer and icon control
    • Adds .mp_plot.folium_map pandas accessor for plotting maps directly from DataFrames
    • Enables asynchronous execution for TILookup.lookup_iocs() when multiple Threat Intelligence providers are configured
    • Adds ti_lookup.enable_provider, ti_lookup.disable_provider, and ti_lookup.set_provider_state methods to control TI providers at runtime

    +1 more show less

    • Adds config parameter to init_notebook() to allow specifying a custom msticpyconfig.yaml path

    6 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Drops official support for Python 3.6; minimum supported version is now Python 3.8
    • !Many modules have moved to new package locations (e.g., sectools migrated to context, transform, or analysis); old import paths remain but issue Deprecation warnings
26. v1.8.2 May 23, 2022 · release notes

    MSTICPy v1.8.2 adds user-delegated auth to OData drivers and switches token cache to memory

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.8.2 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.8.2

    • Adds Delegated Auth support to all OData Drivers, allowing MDE and Graph users to use User-delegated authentication instead of app ID/secrets.
    • Replaces MSAL auth plaintext file cache with a memory cache, removing the ability to use plaintext token cache due to security concerns.

    2 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !The MSAL auth plaintext file cache is replaced with a memory cache; the previous mechanism for persisting tokens to disk is removed.
27. v1.8.0 May 4, 2022 · release notes

    Adds Sentinel Search API support, MSAL token caching, configurable HTTP timeouts via timeout parameter, and database in Kusto query files.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.8.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.8.0

    └──▷ USE IT
    Query an MDE provider with a custom timeout to avoid default hanging.
    code copy

    from msticpy.providers.mde_provider import MDEProvider
    mde = MDEProvider()
    results = mde.my_query(params={'limit': 100}, timeout=30)

    Configure a Kusto query file to specify the database explicitly.
    code copy

    # kql_query.kusto
    #database = 'SecurityDatabase'
    SecurityEvent | where TimeGenerated > ago(1d)

    • SentinelSearchProvider enables creation, status checking, and deletion of automated search jobs via the Sentinel Search API
    • Adds MSALTokenCache support for authentication to leverage MSAL token caching
    • Configurable HTTP timeouts via configuration and timeout=<seconds> parameter in network providers (e.g., mde_prov.my_query(..., timeout=30))
    • Kusto query files now support a database element to control the target database without encoding it in the datafamily

    4 details in Everything

28. v1.7.5 Mar 23, 2022 · release notes

    MSTICPy v1.7.5 adds Splunk async queries, PowerShell de-obfuscation, and keyring management functions

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.7.5 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.7.5

    • Introduces msticpy.analysis.code_cleanup.format_powershell to de-obfuscate and format PowerShell strings into readable code
    • Adds refresh and delete functions in the keyring module for managing cached secrets

    2 details in Everything

29. v1.7.0 Feb 17, 2022 · release notes

    Adds IntSights and Cybereason data providers, SentinelAlert entity, and async HTTP support via httpx

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.7.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.7.0

    └──▷ USE IT
    Fetch threat intel data using the new IntSights provider
    code copy

    import msticpy
    from msticpy.provider.IntSightsProvider import IntSightsProvider
    
    provider = IntSightsProvider(api_key='your-key')
    print(provider.indicators.get(indicator='example.com'))

    Acquire and enrich a Sentinel alert with related entities automatically
    code copy

    from msticpy.sectools.sentinel_alert import SentinelAlert
    
    alert = SentinelAlert(alert_data=sentinel_json_payload)
    print(alert.related_entities)

    Query Cybereason for host data using the new provider
    code copy

    import msticpy
    from msticpy.providers.cybereason.CybereasonProvider import CybereasonProvider
    
    provider = CybereasonProvider(host='https://cybereason-api')
    hosts = provider.hosts.get_all()
    print(hosts)

    • IntSights provider for threat intelligence lookups
    • Cybereason data provider for endpoint detection and responseendpoint detection and responseEndpoint detection and response, monitoring computers for suspicious behavior and providing investigation and response controls, such as isolating a device. data
    • SentinelAlert entity class that auto-extracts related entities from MS Sentinel alerts
    • Underlying HTTP client switched from requests to httpx for async support

    4 details in Everything

30. v1.6.1 Jan 19, 2022 · release notes

    MSTICPy v1.6.0 adds Sentinel Analytics/Watchlists APIs, VirusTotal V3 pivoting, multiple Dataprovider instances, and Mordor caching

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.6.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.6.1

    • Adds FoliumMap layering and subgrouping support
    • Adds Analytics and Watchlists modification operations to the Sentinel API module (class names now drop the Azure prefix)
    • Adds VirusTotal V3 FileBehavior, ObjectBrowser, and Pivot functions via new VT Relationship APIs
    • Enables multiple configuration entries in msticpyconfig.yaml for Splunk, Sumologic, and Microsoft Sentinel workspaces using an instance suffix (e.g., Spunk-Env1)
    • Adds default caching for Mordor Security Datasets driver, storing cache files in $HOME/.msticpy/mordor or the path specified in Dataproviders/Mordor in msticpyconfig.yaml

    +1 more show less

    • Adds a ProcSchema mapping for Sysmon process data in the process_tree visualization

    6 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !The Sentinel API module and class names no longer include the Azure prefix (renamed from Azure Sentinel to Microsoft Sentinel)
31. v1.5.1 Dec 22, 2021 · release notes

    MSTICPy v1.5.1 adds MSI SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. for Azure Sentinel and fixes MDE process tree datetime handling

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.5.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.5.1

    • QueryProvider.connect() accepts mp_az_auth parameter to control Azure authentication methods for Microsoft Sentinel connections.
    • Supports Managed Identity (MSI) credentials automatically for authenticating to Microsoft Sentinel when running in Azure ML notebooks.

    2 details in Everything

32. v1.5.0 Nov 29, 2021 · release notes

    v1.5.0 adds EntityGraph visualization, RiskIQ threat intel provider, M365D/Kusto query providers, and sovereign cloud support for MS Graph.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.5.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.5.0

    └──▷ USE IT
    Visualize an investigation timeline directly from a Pandas DataFrame
    code copy

    df.mp_plot.incident_graph(timeline=True)

    Query risk intelligence for an IP and run pivot functions on DNS resolutions
    code copy

    (
        Dns.RiskIQ.resolutions("jquery.su")
        .query("recordtype=='A'")
        .mp_pivot.run(Dns.RiskIQ.reputation, column='resolve', join='left')
        .mp_pivot.run(IpAddress.RiskIQ.resolutions, column='resolve', join='left')
    )

    Configure credentials for the new M365D provider in the config file
    yaml copy

    DataProviders:
      MicrosoftDefender:
        Args:
          ClientId: <client_id>
          ClientSecret:
            KeyVault: <keyvault_name>
          TenantId: <tenant_id>

    • New EntityGraph class and .mp_plot.incident_graph() Pandas accessor for visualizing investigation graphs and timelines
    • New RiskIQ threat intel provider with registered pivot functions for hostnames, FQDNs, and IP addresses
    • New M365D QueryProvider name enabling consolidated access to Microsoft Defender 365 APIs
    • Preview support for the Kusto query provider targeting Azure Data Explorer clusters
    • Sovereign cloud support added to the Microsoft GraphMicrosoft GraphMicrosoft's unified REST API for accessing data and services across Microsoft 365, Azure AD, and related cloud products, giving cyber tools a single authenticated endpoint to query users, emails, files, and sign-in logs. provider

    5 details in Everything

33. v1.4.5 Oct 15, 2021 · release notes

    Adds JSON serialization support for MSTICPy entities and lazy-loading config editor for Azure ML notebooks

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.4.5 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.4.5

    └──▷ USE IT
    Serialize a list of MSTICPy entities to JSON for external storage or transmission without manual mapping.
    python copy

    import json
    from msticpy.sectools.entities import Entity
    
    # Assuming 'my_entities' is a list of entity instances
    json_output = json.dumps(my_entities, cls=Entity.JSONEncoder)
    print(json_output)

    • Improves config_editor performance in Azure ML notebooks by lazy-loading tab controls instead of rendering all widgets at once

    1 detail in Everything

34. v1.4.0 Sep 10, 2021 · release notes

    MSTICPy v1.4.0 adds Azure sovereign cloud support, MatrixPlot, generic ProcessTree via mp_plot accessor

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.4.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.4.0

    └──▷ USE IT
    Configure MSTICPy for Azure China or Government clouds via the settings editor.
    steps copy

    # Open MpConfigEdit and navigate to Azure tab
    # Select 'Azure China', 'Azure Germany', or 'US Government'
    # from the cloud provider dropdown menu

    • Adds comprehensive support for Azure sovereign clouds (China, Germany and US Government) via MpConfigEdit UI and AzureCloudConfig class.
    • Adds MatrixPlot visualization for entity interactions with dedicated vis/matrix_plot.py module.
    • Updates ProcessTree to accept generic datasets and Microsoft Defender for Endpoint logs with automatic format detection.
    • Introduces the mp_plot pandas accessor (e.g., df.mp_plot.timeline()) as a central interface for all visualization methods.

    4 details in Everything

35. v1.3.1 Aug 10, 2021 · release notes

    Kql/Azure Sentinel provider now imports Kqlmagic directly for use in standard Python scripts

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.3.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.3.1

    • kql_driver now uses Kqlmagic.kql() as a standard import instead of requiring IPython magics

    1 detail in Everything

    └──▷ BREAKING ON UPGRADE

    • !MSTICPY now requires pandas 1.1.5 as a minimum
36. v1.3.0 Jul 19, 2021 · release notes

    Adds timeline duration visualization for grouping activity by columns

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.3.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.3.0

    └──▷ USE IT
    Visualize communication duration grouped by source, destination, and protocol.
    python copy

    az_net_flows_df.mp_timeline.plot_duration(
        group_by=["SrcIP", "DestIP", "L7Protocol"]
    )

    Display a timeline with reference events for host logons.
    python copy

    from msticpy.nbtools.timeline_duration import display_timeline_duration
    
    display_timeline_duration(
        host_logons,
        group_by="Account",
        ref_events=host_logons.sample(3),
        ref_col="TargetUserName",
    )

    • Adds mp_timeline.plot_duration pandas extension for visualizing grouped timeline durations
    • Adds display_timeline_duration function in msticpy.nbtools.timeline_duration

    2 details in Everything

37. v1.2.1 Jun 17, 2021 · release notes

    MSTICPy v1.2.1 adds Azure Resource Graph and Sumologic data providers plus a Bokeh-based DataViewer control.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.2.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.2.1

    └──▷ USE IT
    Query Azure infrastructure directly using KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. within a Jupyter environment.
    code copy

    from msticpy.data import DataProvider
    provider = DataProvider(query_provider='AzureResourceGraph')
    df = provider.query('Resources | where type =~ "microsoft.compute/virtualmachines"')

    Visualize and filter a pandas DataFrame interactively using the new Bokeh widget.
    code copy

    from msticpy.vis import DataViewer
    viewer = DataViewer(df)
    viewer.show()

    Connect to Sumo Logic for security data analysis.
    code copy

    from msticpy.data import DataProvider
    provider = DataProvider(query_provider='Sumologic')
    results = provider.search('error', count=10)

    • Adds AzureResourceGraph data provider for querying Azure resources via KQL
    • Adds Sumologic data provider connecting to the Sumo Logic Search API
    • Adds DataViewer control for sorting, filtering, and browsing pandas DataFrames in notebooks

    3 details in Everything

38. v1.1.0 May 13, 2021 · release notes

    MSTICPy v1.1.0 adds Greynoise TI provider, pd accessors (filter_cols, sort, etc.), and instance shortcut methods for entities.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.1.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.1.0

    └──▷ USE IT
    Query threat intel using the new Greynoise provider within a ThreatIntel lookup context.
    python copy

    from msticpy.sectools.threat_intelligence import ThreatIntel
    with ThreatIntel() as ti:
        results = ti.query_provider('Greynoise', '1.2.3.4')

    Use the new sort and filter_cols pd accessors directly on a pivot result DataFrame to prepare data for analysis.
    python copy

    # Assuming 'pivot_df' is the result of a pivot operation
    pivot_df = pivot_df.sort('TimeGenerated', ascending=False).filter_cols(['Host', 'Account', 'TimeGenerated'])

    Use the new instance shortcut method on an entity object to retrieve WHOIS information more concisely.
    python copy

    from msticpy.sectools.ipaddr import IpAddress
    my_ip = IpAddress(Address='8.8.8.8')
    whois_info = my_ip.whois()

    • Adds support for the Greynoise Community API as a Threat Intel provider
    • Adds Pandas accessors filter_cols, filter, sort, list_to_rows, and parse_json to pivot data frames
    • Adds instance shortcut methods to entities (e.g., IpAddress.whois() instead of IpAddress.util.whois()) for easier use in containers
    • Adds join_ignore_case parameter to standard pivot function joins
    • Adds tilookup functions to allow enabling/disabling individual Threat Intel providers

    +1 more show less

    • Adds find_anomaly_periods function and a shorter alias for timeseries_anomalies_stl in timeseries.py

    6 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Renamed pivot query functions to shorter aliases, which may break existing code relying on the previous longer names
    • !Some Azure Sentinel query function names have been shortened, potentially breaking scripts that call them by their old names
39. v1.0.0 Apr 15, 2021 · release notes

    MSTICPy v1.0 introduces pip extras, settings management with auto-loading, SQL to KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. translation, and pivot browser

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.0.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v1.0.0

    └──▷ USE IT
    Automatically load specific providers like Azure Sentinel and Threat Intelligence at notebook start.
    config copy

    # In msticpyconfig.yaml
    providers:
      DataProviders:
        - AzureSentinel
      TIProviders:
        - MISP

    • Adds pip extras for selective installation of MSTICPy dependencies via [extras] syntax.
    • Adds automatic loading of DataProviders, TI Providers, Notebooklets, and Pivot functions in msticpyconfig.yaml during init_notebook().
    • Adds msticpy.data.sql_to_kql module to translate ANSI SQL queries to KQL.
    • Adds pivot_browser for browsing and searching pivot functions.
    • Adds ability to save and run pipelined pivot operations from YAML files via pivot_pipeline.py.

    +2 more show less

    • Adds check_version parameter to init_notebook() to detect new MSTICPy releases.
    • Adds Entity.create static method to instantiate entities from dict or pd.Series.

    7 details in Everything

40. v0.9.0 Feb 11, 2021 · release notes

    MSTICPy v0.9.0 introduces pivot functions for unified entity-based queries and lookups

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.9.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.9.0

    • pivot functions add a consistent API where entities like IpAddress expose lookup methods (e.g., whois) directly as attributes, returning DataFrames for all inputs (string, list, DataFrame).
    • AzureResource entity gains a new function to parse and expose individual elements from resource strings.
    • entities refactored into separate modules with graphing extensions included.

    3 details in Everything

41. v0.8.9 Jan 22, 2021 · release notes

    MSTICPy v0.8.9 adds Azure Sentinel API support for incidents/queries and updates data obfuscation

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.8.9 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.8.9

    • Adds AzureSentinelAPIClient to retrieve Security Incidents and Queries with chained Kqlmagic authentication

    1 detail in Everything

42. v0.8.5 Oct 16, 2020 · release notes

    MSTICPy v0.8.5 adds VirusTotal V3 API, Azure Sentinel API, Mordor data provider, and chained Azure authentication

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.8.5 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.8.5

    └──▷ USE IT
    Install the specific extras package required to access the new VirusTotal V3 API module.
    shell copy

    pip install msticpy[vt3]

    Use the new DataProvider option to split a massive query into daily chunks for manageable processing.
    code copy

    df = data_providers.get_data(query, split_query_by='1D')

    • Adds VirusTotal V3 API support with relationship graphing via the vt3 pip extra
    • Enables chained authentication for Azure modules to reuse existing Azure CLI or MSI credentials
    • Adds initial Azure Sentinel API support for retrieving Workspaces, Alert Rules, Hunting Queries, and bookmarks
    • Introduces a Mordor data provider and notebook browser for searching and downloading adversary simulation datasets
    • Adds split_query_by option to DataProvider queries to split large results by time intervals

    5 details in Everything

43. v0.8.0 Sep 21, 2020 · release notes

    MSTICPy v0.8.0 adds query_browser module, PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. obfuscation functions, and persistent widget memory in Jupyter notebooks

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.8.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.8.0

    └──▷ USE IT
    Explore available data providers interactively before writing queries
    python copy

    from msticpy.sectools.query_browser import query_browser
    query_browser()

    Obfuscate sensitive data fields before logging or export
    python copy

    from msticpy.data.query_browser import obfuscate_data
    obfuscated = obfuscate_data(input_data)

    • Adds query_browser module for interactive data exploration
    • Adds methods to tilookup and data_providers modules to access respective browser widgets
    • Adds functions to obfuscate/hash data to protect PII
    • Implements memory in GetText and QueryTimes widgets to revert to last settings on cell re-run

    4 details in Everything

44. v0.7.0 Aug 6, 2020 · release notes

    New data provider for Splunk and uploaders for Azure Sentinel/Splunk in MSTICPy v0.7.0

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.7.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.7.0

45. v0.6.2 Jul 23, 2020 · release notes

    Adds Threat Intelligence results browser widget

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.6.2 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.6.2

46. v0.6.0 Jun 30, 2020 · release notes

    MSTICPy v0.6.0 introduces time series decomposition on any log data and check_version() utility

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.6.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.6.0

    └──▷ USE IT
    Check if the current msticpy installation is up to date with PyPI.
    python copy

    import msticpy
    msticpy.check_version()

    • check_version() function added to msticpy module for version checking against PyPI
    • timeseries_decomposition capability using Statsmodel STL in the time series module for analyzing any timestamped log data
    • Added modellable_params argument to Anomalous Sequence model classes to override default heuristic parameter selection

    3 details in Everything

47. v0.5.1 May 22, 2020 · release notes

    v0.5.1 adds LocalDataDriver for offline CSV/pickled DF queries and AzureSentinel alias for LogAnalytics

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.5.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.5.1

    └──▷ USE IT
    Use local CSV/pickled files as a query source in notebooks when online data access is unavailable.
    python copy

    from msticpy.data.local_data_driver import LocalDataDriver
    provider = LocalDataDriver()
    query_store = provider.query_provider

    • Adds LocalDataDriver class to enable using local CSV and pickled DataFrames as a drop-in QueryProvider replacement
    • Adds AzureSentinel as an alias for the LogAnalytics DataEnvironment

    2 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Removed deprecated kql.py, query_builtin_queries, query_mgr.py, and query_schema.py modules
    • !Changed location of query_defns.py; existing imports will break until updated
48. v0.5.0 May 12, 2020 · release notes

    v0.5.0 adds Markov chain anomaly analysis, Morph Chart visualization, and nbinit startup function to MSTICPy.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.5.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.5.0

    • msticpy.anomaly.markov provides anomalous sequence detection using Markov Chain analysis for session commands/patterns
    • msticpy.visualization.morph_charts enables 3D Morph Chart visualization of event data (experimental)
    • nbinit() provides a streamlined Jupyter notebook startup function handling package installs, imports, and option setting
    • Added msticpy.common.pkg_config.validate_config() to validate current config or external config file

    4 details in Everything

49. v0.4.1 May 8, 2020 · release notes

    MSTICPy v0.4.1 adds Markov chain anomaly analysis, Morph Chart visualization, config validation API, and Azure Sentinel Linux logon queries.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.4.1 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.4.1

    • msticpy.anomalymap.AnomalyDetection enables Markov Chain-based sequence analysis to detect anomalous command patterns in sessions
    • msticpy.visualization.MorphChart provides experimental 3D visualization of event data via the plot_morph_chart function
    • Adds msticpy.common.pkg_config.validate_config() function for validating current or external configuration files
    • Adds two new Azure Sentinel query templates: linux_logon_failures and host_linux_logons under azure_sentinel.queries.logon_queries

    4 details in Everything

50. v0.4.0 Mar 28, 2020 · release notes

    MSTICPy v0.4.0 adds KeyVault secrets management, Bokeh time series visualization, and pandas DataFrame accessors for whois data.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.4.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.4.0

    • Adds config2kv.py script to push configuration secrets into Azure KeyVault with support for multiple Azure clouds
    • Adds time series anomaly detection for Kusto data sets and Bokeh chart visualization in Jupyter Notebooks via generic decomposition queries
    • Adds pandas DataFrame accessor function for get_whois_info to retrieve WHOIS information as a DataFrame

    3 details in Everything

51. v0.3.2 Feb 14, 2020 · release notes

    MSTICPy v0.3.2 adds pandas DataFrame extensions and IPython magics for IoCExtract, Base64Unpack, Timeline, and ProcessTree

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.3.2 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.3.2

    └──▷ USE IT
    Render security event logs as an interactive timeline directly from a pandas DataFrame without explicit function calls.
    code copy

    my_events_df.mp_timeline.plot()

    Execute IoCIoCAn indicator of compromise, an observable clue that may point to malicious activity, such as a suspicious file hash or domain. It is evidence to investigate, rather than proof by itself. extraction on text pasted directly into an IPython notebook cell.
    code copy

    %%ioc
    192.168.1.1
    evil.com

    Unpack base64 encoded strings in a notebook cell using the new magic command.
    code copy

    %%b64
    c2VjcmV0IGRhdGE=

    • timeline gets a new pandas extension allowing df.mp_timeline.plot() to render timelines directly from a DataFrame
    • process_tree gains a pandas extension with df.mp_process_tree.plot() for rendering process trees from DataFrames, returning the plot figure and layout object
    • base64unpack adds a pandas extension for unpacking Base64 data in DataFrames plus an IPython %%b64 magic for notebook cells
    • iocextract adds a pandas extension for extracting Indicators of Compromise from DataFrames plus an IPython %%ioc magic for inline notebook use
    • wsconfig adds a method to display available workspaces

    5 details in Everything

52. v0.3.0 Jan 7, 2020 · release notes

    Adds Microsoft Defender ATP query support, interactive Bokeh process tree viewer, and Azure resource querying capabilities.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.3.0 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.3.0

    └──▷ USE IT
    Query Microsoft Defender alerts using the new query provider.
    code copy

    from msticpy.sector.mdatp import MDATPQueryProvider
    
    mdatp_prov = MDATPQueryProvider()
    alerts = mdatp_prov.get_alerts()

    Visualize a process tree interactively in a Jupyter notebook using the new Bokeh viewer.
    code copy

    from msticpy.view.process_tree import ProcessTreeViewer
    
    viewer = ProcessTreeViewer(processes)
    viewer.show()

    • ti_provider_settings refactored to generic provider_settings module
    • Added a query provider/driver to query Microsoft Defender alerts, machines, processes and arbitrary KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. queries of the Hunting data
    • Template queries for MDATP for hunting and standard entities
    • Added Process Tree Viewer - Bokeh interactive graphical view for one or more Process Trees in a data set
    • Process tree utilities - data library to create and query process trees

    +4 more show less

    • Azure properties of subscriptions and resources such as VMs can be queried from Notebooks.
    • Query providers now accept ISOISOA widely adopted international standards body that publishes technical specifications; cyber tools cite ISO standards to signal compliance with recognized benchmarks for quality, security, or interoperability.-string format for datetime fields for queries (in addition to datetime and timedelta)
    • Added Progress widget to nbwidgets.
    • Added config support for GeoIPGeoIPA database that maps IP addresses to geographic locations; tools use it to attribute traffic or alerts to countries, cities, or ASNs without querying external services. providers from msticpyconfig.yaml

    9 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !GeoIP classes are no longer loaded by default.
53. v0.2.8 Dec 20, 2019 · release notes

    MSTICPy v0.2.8 adds Microsoft Defender ATP query provider, interactive Process Tree browser, and Azure subscription querying.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.2.8 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.2.8

    └──▷ USE IT
    Hunt for alerts in Microsoft Defender ATP using KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. queries via the new provider.
    code copy

    from msticpy.sectools import QueryProvider
    
    query_prov = QueryProvider('DefenderATP')
    alerts = query_prov.alerts.search(query='AlertSeverity == "High"')

    Visualize process hierarchy for incident analysis using the new interactive Bokeh viewer.
    code copy

    from msticpy.sectools import ProcessTreeViewer
    import pandas as pd
    
    tree_df = pd.DataFrame([...]) # Your process tree data
    ProcessTreeViewer(tree_df).show()

    Query Azure subscription details and resource properties directly from your analysis notebook.
    code copy

    from msticpy.cloud import AzureMgmtSubscription, AzureMgmtResource
    
    sub = AzureMgmtSubscription()
    subs = sub.get_subscriptions()
    resources = AzureMgmtResource(subscription_id=subs[0].subscription_id)
    vms = resources.get_virtual_machines()

    • QueryProvider now supports Microsoft Defender ATP for hunting alerts, machines, processes, and arbitrary KQL queries
    • New ProcessTreeViewer (Bokeh) interactive graph for visualizing Windows and Linux process trees
    • Process tree utilities (ProcessTree) data library for creating and querying tree structures in Python
    • AzureMgmtSubscription and AzureMgmtResource providers added to query subscription and resource properties from notebooks
    • QueryProvider queries now accept ISOISOA widely adopted international standards body that publishes technical specifications; cyber tools cite ISO standards to signal compliance with recognized benchmarks for quality, security, or interoperability.-string format for datetime fields alongside existing datetime/timedelta types

    +1 more show less

    • New progress_bar widget added to the nbwidgets module for Jupyter notebook progress visualization

    6 details in Everything

54. 0.2.7 Oct 31, 2019 · release notes

    MSTICPy 0.2.7 adds syslog/IP/domain utilities, new TI providers, date parameters in query templates, and msticpy_settings attribute

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch 0.2.7 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout 0.2.7

    └──▷ USE IT
    Use the new syslog command line analysis module to process logs.
    code copy

    import msticpy.cmd_line as cmd
    result = cmd.analyze_cmd('cmd.exe /c dir')

    Access configuration via the new top-level settings attribute.
    code copy

    import msticpy as m
    config = m.settings.get('TIProvider', {})

    • Added syslog_utils module for common syslog data manipulation
    • Added cmd_line module for syslog CMD analysis
    • Added ip_utils module for common IP Address operations
    • Added domain_utils module for common domain/DNS/URL operations
    • Added new TI providers - open_page_rank and tor_exit_nodes

    +4 more show less

    • Added package auto-install function to utility.py
    • In query templates you can now express date parameters (and query defaults) as a pos/neg number relative to current date or a KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly.-like range (-30d, 15h, etc.)
    • wsconfig.py - can create a Connection string directly from wsconfig.
    • Added settings as a top-level msticpy attribute.

    9 details in Everything

55. v0.2.6 Sep 17, 2019 · release notes

    MSTICPy v0.2.6 adds timeline visualization controls and new TI provider validation checks

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.2.6 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.2.6

    • Adds display_timeline_values to display timelines with a scalar value (line, circle, vbar)
    • Adds filtering text box to all nbwidgets select widgets via the nbwidgets module
    • Adds SelectSubset widget in nbwidgets allowing users to pick from one list and add to selected subset
    • Adds validation checks for missing TI Providers or keys with updated documentation in TIProvider

    4 details in Everything

56. v0.2.5 Aug 23, 2019 · release notes

    MSTICPy v0.2.5 adds Azure Sentinel and KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. TI providers, WSConfig workspace/tenant resolution from msticpyconfig.yaml

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.2.5 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.2.5

    └──▷ USE IT
    Query threat intelligence data using the new Azure Sentinel provider.
    python copy

    from msticpy.sectools import TILookup
    ti = TILookup()
    ti.init(provider='AzureSentinel')

    Use the new KQL base provider for LogAnalytics TI sources.
    python copy

    from msticpy.sectools import TILookup
    ti = TILookup()
    ti.init(provider='kql_base')

    • Adds AzureSentinel Threat Intel provider to TILookup
    • Adds kql_base.py provider in TILookup for LogAnalytics TI sources
    • Enables WSConfig to read workspace and tenant configuration from msticpyconfig.yaml

    3 details in Everything

57. v0.1.8 May 31, 2019 · release notes

    Adds YAML-based query definitions in data/queries with default and runtime replaceable parameters

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.1.8 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.1.8

    • data/queries directory for storing query definitions in YAML files
    • Support for default parameters in YAML query files
    • Runtime parameter replacement for query definitions

    3 details in Everything

58. v0.1.6 Apr 12, 2019 · release notes

    Adds execute_simple_query to msticpy.query.kql for direct KQLKQLKusto Query Language, Microsoft's read-only query language for Azure Data Explorer and Microsoft Sentinel, used by cyber tools to filter, aggregate, and correlate large volumes of log and telemetry data quickly. execution and file-reading to auditdextract

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v0.1.6 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout v0.1.6

    └──▷ USE IT
    Run a simple KQL query string directly without setting up full authentication context
    python copy

    from msticpy.query.kql import execute_simple_query
    result = execute_simple_query('SecurityEvent | take 5')
    print(result)

    • Adds execute_simple_query function to msticpy.query.kql module
    • Adds file-reading capability to auditdextract.py

    2 details in Everything

59. 0.1.5 Apr 5, 2019 · release notes

    Adds observationlist module to MSTICPy

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch 0.1.5 https://github.com/microsoft/msticpy.git
    # already have the repo? check out this version:
    $ git checkout 0.1.5

    └──▷ USE IT
    Import the new observation list utility for data handling in your scripts.
    python copy

    import msticpy.observationlist

    • observationlist module added

    1 detail in Everything