Guides

Azure Log Analytics

Integrate with Azure Log Analytics Workspace tables to search them on demand without writing KQL.

Suggest Edits

📘

TL;DR

To integrate Microsoft Sentinel with Query:

  • Register an App Registration with a Client Secret.
  • Grant the App Registration Service Principal the Log Analytics Reader IAM Role in your Azure Subscription.
  • Onboard one or more tables from your Log Analytics Workspace.
  • Use Query Search to provide decision support for Incident Response, Investigations, Threat Hunting, Red Team targeting operations, and/or Continuous Compliance Monitoring use cases.

Overview

Azure Log Analytics is a time-series wide-column NoSQL-like logging service within the Azure Monitor ecosystem. Each workspace contains multiple tables organized into separated columns with multiple rows, defined by a set schema of columns that contains structured and semi-structured logging and event data. Logs and other data can be read from these tables using the Kusto Query Language (KQL) to perform detailed analyses or simple searches, and anything in between.

Azure Log Analytics Workspace tables can be populated by various means, such as via Microsoft Sentinel Connectors, Azure Monitor Agents (AMAs), and other mechanisms to store important security, networking, and/or application performance monitoring logs.

Query Federated Search integrates directly with the Azure Log Analytic Workspace (LAW) by providing a 1:1 translation interface with KQL. Using Query's no-code data mapping utility, Configure Schema, you can onboard specific LAW tables and map the data into the Query Data Model, a derivative of the OCSF. Query handles query construction, fine tuning, time-windowing, and schema introspection on your behalf. The data is transformed to OCSF/QDM format at search time and none of the data is retained by Query. This allows you to perform ad-hoc searches to support Incident Response, Investigations, Threat Hunting, Red Teaming, and Compliance use cases outside of what Analytics Rules can provide.

Prerequisites

You must have Global Administrator permissions for your Entra ID tenant and Owner permissions for your Azure Subscription to be able to create an App Registration, generate Client Credentials, and grant the App Registration Service Principal proper roles in your Subscription to integrate with LAW tables.

Create an Application Registration & Grant an Azure Role

  1. In the Microsoft Azure portal navigate to Microsoft Entra ID -> Add -> App Registration as shown below.

  2. Populate a Name for the application, leave the Supported account types option as the default (Accounts in this organizational directory only (Default Directory only - Single tenant)) then select Register as shown below.

  3. By default you will be taken to the Overview screen of your new application. Navigate to Certificates & Secrets -> Client Secrets -> New client secret. Enter a Description, choose an Expires date from the selector, and finally choose Add as shown below.

  4. WARNING! Before navigating away copy the Value entry and save it somewhere secure, this is the Client Secret Value required to perform authentication into your Azure Subscriptions. If you did not copy it, no big deal, just delete the Client Secret and create another - but remember to copy the Value that time.

  5. Navigate to the Overview screen and copy the values for the Application (client) ID and Directory (tenant ID) as shown below. The App ID and Tenant ID are also required to perform authentication into your Azure subscription.

  6. In the Microsoft Azure portal navigate to Subscriptions -> choose a Subscription where Microsoft Sentinel is deployed -> Access control (IAM) -> Add role assignment as shown below.

  7. Add the Log Analytics Reader permission from Job function roles, you can only choose one at a time so you will need to repeat this step. Once you have selected the Role, select Next as shown below.

  8. In the following screen, ensure User, group, or service principal is highlighted for Assign access to and select Select members. In the pop-up window type the name of the application from Step 2, toggle the checkbox, choose Select as shown below. Then choose Review + assign (twice!) to proceed.

Before moving to the next section, copy the Tenant ID, App (Client) ID, and the App Secret Value as these are required for the Query Connectors to configure access.

Retrieve Log Analytics Workspace and Resource Group information

In addition to the App Registration information, you will need to retrieve your Log Analytics Workspace ID and the name of the table(s) you wish to connect.

  1. Navigate to the Microsoft Azure portal -> Log Analytics workspaces -> your Microsoft Sentinel's Workspace name -> Overview

  2. Copy the value of Workspace ID as shown below.

  3. To retrieve table names, navigate to Logs and expand the various headers as shown below. These should correspond to active Microsoft Sentinel Connectors as well as any individual tables you have created within Azure Log Analytics.

In this section you learned how to create a new App Registration, grant it Azure Subscription IAM roles, and retrieve Azure LAW attributes. In the next section you will learn how to configure the Azure Log Analytics Connector.

Configuring Microsoft Sentinel Connectors

There are two types of Connectors available for Microsoft Sentinel, one solely for querying Incidents from Sentinel and another for querying LAW tables. The information you copied in the previous section will be use for both Connectors, no other permissions or external configuration is required.

Configure the Azure Log Analytics Connector

The Azure Log Analytics Connector uses the Microsoft Azure Azure Monitor Logs SDK to describe and introspect schema from Log Analytics Workspace tables, submit KQL queries, and retrieve results from the queries.

The Azure Log Analytics Connector is a dynamic schema platform configurations. Static schemas are platforms in which the Query team pre-configures the type of data normalization that happens and a dynamic schema platform gives the user control for mapping and normalizing data into the Query Data Model. '

For dynamic schemas, Query provides a no-code data mapping workflow to allow you to map your source data into the Query Data Model. For more information, see the Configure Schema and the Normalization and the Query Data Model sections, respectively.

🚧

Configure Schema Limitations

Due to limitations in the introspection and mapping process, you may only map one Log Analytics table per Connector. There are not any limits to the amount of Connectors you can create.

Use the following steps to create a new Query Federated Search Connector for Azure Log Analytics.

  1. Navigate to the Connections page, select Add Connections, and selectAzure Log Analytics from the SIEM and Log Management category as shown below. Alternatively you can use the search bar to find the Connector.

  2. In the Connection Info section of the platform connector, provide the following details, select Save, and then Test Connection as shown below (FIG. 11).

    1. Connector Alias Name: The human-readable name you want to give to this connector, you can name it whatever you want, but you can use this to differentiate multiple Sentinel deployments across various Azure Subscriptions.
    2. Platform Login Method: Leave the default value: Default Login.
    3. Microsoft Entra ID Tenant ID : The Tenant ID (also known as the Directory ID) for the Microsoft Entra ID tenant associated with your Azure Subscription.
    4. App Registration Client ID : The Client ID (also know as the Application ID) for the Entra ID App Registration you created in the Prerequisites section.
    5. Azure Log Analytics Workspace ID : The ID of your Azure Log Analytics Workspace associated with your Microsoft Sentinel deployment.
    6. Azure Log Analytics Table Name : Name of the table in Azure Log Analytics - these tables do not need to be associated with Microsoft Sentinel to work.
    7. App Registration Client Secret Value : The Client Secret Value for the credentials you created for your App Registration.

  3. Execute the Configure Schema workflow to map your target table data into the QDM.

After completing the Configure Schema workflow, you are ready to perform Federated Searches against your Log Analytics tables.

Querying the Microsoft Sentinel Connectors

After you have onboarded at least one Azure Log Analytics Connector, you are ready to begin searching. Within the Query Federated Search and Query Summary Insights UI, all Platforms are enabled by default.

To either ensure your Microsoft Sentinel Connectors are enabled, or to only query them specifically, navigate to the SIEM and Log Management section of the Selected Platforms dropdown menu and ensure the checkbox is (de)selected to match your desired search criteria, as shown below.

The Azure Log Analytics Connector allows you to search whatever you configured during the Configure Schema workflow.

For more information about this terminology, refer to the Normalization and the Query Data Model (QDM) section of the docs or check out our QDM Schema website.

Resources

For more information to help with troubleshooting or general information about Microsoft Sentinel and its related services, refer to the links below.

Updated 6 months ago