Azure Sentinel analytics queries (custom detection rules) to create incidents - Part 1

October 05, 2020 ,

Azure Sentinel analytics queries  (custom detection rules) to create incidents - Part 1

If you are new to the Azure Sentinel or have configured all the Data connectors but not sure what to do next while learning it then this post might give you direction to move forward. Why not start with something simple that we work on in day to day as a cloud engineer like Sentinel should generate an alert when we create VM or when we change rules in NSG or change storage account access policy to understand how Sentinel works on such events. Once, we will be comfortable with all this then we move to more security things. This Azure Sentinel has built-in analytics or out-of-box threat detection capability that covers most of the things but sometimes you need to monitor something that is not out-of-the-box like Sentinel should generate an alert when a new VM has created in your environment.  In this post, we will take an example of VM creation and deletion for creating analytics rule. There are two types of rules for analytics in Azure sentinel to detect threats as blow.

1. Scheduled query rule: In this rule type, we put our own custom KQL query and map it with the entities to trigger incidents. 

2. Microsoft incident creation rule: This rule uses Microsoft predefined 'Microsoft security service' like Azure Security Center, Microsoft Cloud App Security, AAD Identity Protection, Office 365 ATP, Azure ATP, Microsoft ATP, and Security center for IoT to generate incidents.

In this post, we will use the Scheduled query rule. Prerequisite:  Azure Activity data connectors should have configured in this Sentinel workspace. The most important part of the custom analytics rule is to write KQL queries.

Step 1. Navigate to Azure Sentinel Workspace ->Analytics and click on Create then select Scheduled query rule.


Step 2. Supply the details like the name of the rule, tactics, and severity as below and click on Next.


Step 3. On the Set rule logic page, we need to provide the value like KQL query, rule schedule, map entities, etc. Here, we are not going to use Map entities for now.



Run query every: 5 minutes

Lookup data from the last: 5 minutes

In the below example, we have used 5 minutes for both 'Run query every' and 'Lookup data from the last'. It means the custom rule runs every five minutes on data collected from the last five minutes. If there is no VM created in the last five minutes then the scheduled query will not detect any incident. Next time the rule will look for the last five minutes data so there won't be any duplicate incident created. Suppose, you set the query run time every five min and lookup time one hour then the query runs in every five min on data collected from the last one hour, its means the query runs 12 times for the same data, and 12 duplicate incidents will be created so be careful with query scheduling parameters.  We will talk about 'Map entities' later in this post.


Step 4. Keep the other settings default for now and click on the Create button. After that, you can see the new custom rule has created below.  Now, if any new VM creates in your environment then Sentinel will trigger alerts and incidents.


Step 5. Create a VM in your environment and after 5 minutes you will see the Sentinel has triggered an incident as below.


Map Entities: In the above example, we didn't configure map entities so you get the incident but there are no entities associated and the investigation button has gray out as shown below image.  You cannot do further investigation and analysis, including the grouping of alerts into incidents in the Incident settings tab as shown in the below image. So, you need to configure map entities to investigate the incidents.


Now let's configure the 'Map Entities' for the same query to enable investigation for the detected threats. Actually, it will add a line of code in the existing query once you click on the Add button. We want to see the name of VM created VM so we have chosen Resource as a Host. Here, you play with it and determine what you want to associate with as an entity for investigation.  Select the newly created rules and then click on the Edit button and navigate to the Map entities section.


Once you click on Add button then the code will be added into the existing query like below. Here, two last lines of code have added after clicking on the add button. 


Create again a VM and wait for the Sentinel to generate the incidents. You will see the VM name has associated as an entity and the investigation button has enabled. You will be able to do further analysis. Still, it is not fully configured, you need to add more entities to get a better view.


Now create a custom rule when VM deleted in your environment.

The query for the scheduled query to trigger an incident when VM deletes in your environment.

The query for schedule query to trigger incident when NSG has modified in your environment.


Now, you have a basic understanding of how to create custom analytics rules for schedule query run to trigger incidents in Azure Sentinel, and in the next post, I will come with more security-related and more depth examples. I hope this is helpful and if there’s something specific you want to hear about, let us know in the comments, or tweet us at @sakaldeep