Azure Sentinel analytics queries (custom detection rules) to create incidents - Part 1

If you are new to the Azure Sentinel or have configured all the Data connectors but not sure what to do next while learning then this post might give you direction to move forward. Why not start with something simple that we work on day to day as a cloud engineer like Sentinel should generate an alert when we create VM or when we change rules in NSG or change storage account access policy to understand how Sentinel works on such events. Once, we will be comfortable with all this then we move to more security things. This Azure Sentinel has built-in analytics or out-of-box threat detection capability that covers most of the things but sometimes you need to monitor something that is not out-of-the-box like Sentinel should generate an alert when a new VM has been created in your environment.  In this post, we will take an example of VM creation and deletion for creating an analytics rule. There are two types of rules for analytics in Azure sentinel to detect threats as blow.

1. Scheduled query rule: In this rule type, we put our own custom KQL query and map it with the entities to trigger incidents.

2. Microsoft incident creation rule: This rule uses Microsoft predefined 'Microsoft security service' like Azure Security Center, Microsoft Cloud App Security, AAD Identity Protection, Office 365 ATP, Azure ATP, Microsoft ATP, and Security center for IoT to generate incidents.

In this post, we will use the Scheduled query rule. Prerequisite:  Azure Activity data connectors should have been configured in this Sentinel workspace. The most important part of the custom analytics rule is to write KQL queries.

Step 1. Navigate to Azure Sentinel Workspace ->Analytics and click on Create then select Scheduled query rule.

Step 2. Supply the details like the name of the rule, tactics, and severity as below and click on Next.

Step 3. On the Set rule logic page, we need to provide the value like KQL query, rule schedule, map entities, etc. Here, we are not going to use Map entities for now.

Query:

AzureActivity
| where OperationName == "Create or Update Virtual Machine" or OperationName =="Create Deployment"
| where ActivityStatus == "Succeeded"

Run query every: 5 minutes

Lookup data from the last: 5 minutes

In the below example, we have used 5 minutes for both 'Run query every' and 'Lookup data from the last'. It means the custom rule runs every five minutes on data collected from the last five minutes. If there is no VM created in the last five minutes then the scheduled query will not detect any incident. Next time the rule will look for the last five minutes' data so there won't be any duplicate incidents created. Suppose, you set the query run time every five min and lookup time one hour then the query runs every five min on data collected from the last one hour, this means the query runs 12 times for the same data, and 12 duplicate incidents will be created so be careful with query scheduling parameters.  We will talk about 'Map entities' later in this post.

Step 4. Keep the other settings default for now and click on the Create button. After that, you can see the new custom rule created below.  Now, if any new VM creates in your environment then Sentinel will trigger alerts and incidents.

Step 5. Create a VM in your environment and after 5 minutes you will see the Sentinel has triggered an incident as below.

Map Entities: In the above example, we didn't configure map entities so you get the incident but there are no entities associated and the investigation button has grayed out as shown below image.  You cannot do further investigation and analysis, including the grouping of alerts into incidents in the Incident settings tab as shown in the below image. So, you need to configure map entities to investigate the incidents.

Now let's configure the 'Map Entities' for the same query to enable investigation of the detected threats. Actually, it will add a line of code to the existing query once you click on the Add button. We want to see the name of VM created VM so we have chosen Resource as a Host. Here, you play with it and determine what you want to associate with as an entity for investigation.  Select the newly created rules and then click on the Edit button and navigate to the Map entities section.

Once you click on Add button then the code will be added to the existing query like below. Here, two last lines of code have been added after clicking on the add button.

AzureActivity
| where OperationName == "Create or Update Virtual Machine" or OperationName =="Create Deployment"
| where ActivityStatus == "Succeeded"
| extend HostCustomEntity = Resource
| extend IPCustomEntity = CallerIpAddress

Create again a VM and wait for the Sentinel to generate the incidents. You will see the VM name has been associated as an entity and the investigation button has been enabled. You will be able to do further analysis. Still, it is not fully configured, you need to add more entities to get a better view.

Now create a custom rule when VM is deleted in your environment.

The query for the scheduled query triggers an incident when VM deletes in your environment.

AzureActivity
| where OperationName == "Delete Virtual Machine"
| where ActivityStatus == "Succeeded"
| extend HostCustomEntity = Resource
| extend IPCustomEntity = CallerIpAddress

The query for schedule query to trigger incidents when NSG has been modified in your environment.

AzureActivity
| where OperationName == "Create or Update Security Rule"
| where ActivityStatus == "Succeeded"
| extend HostCustomEntity = Resource

Now, you have a basic understanding of how to create custom analytics rules for schedule queries run to trigger incidents in Azure Sentinel, and in the next post, I will come up with more security-related and more depth examples. I hope this is helpful and if there’s something specific you want to hear about, let us know in the comments, or tweet us at @sakaldeep