Revolutionizing log collection with Azure Monitor Agent

With the Log Analytics agent (also known as MMA or OMS) being deprecated, it’s a great opportunity to discuss its successor, the Azure Monitor agent, or AMA for short, and learn why it’s so much better and keeps getting better.

AMA is a lightweight log collection agent designed to consume as few resources as possible when collecting metrics and logs from servers. It can be installed on various flavors. OS versions for both Linux and Windows machines Hosted in Azure, on-premises, or other cloud environments. If installed on a non-Azure machine, AMA requires the installation of the Azure Arc agent to provide mirroring and centralized cloud management capabilities to the machine.

All logs collected from computers that have AMA installed and associated with a Microsoft Sentinel workspace are sent to various Microsoft Sentinel tables, depending on the type of source collected (Windows DNS, Windows Security Events, Firewall, IIS, Syslog, CEF, etc.).

AMA can be controlled using: Data Collection Rules (DCR) You can define where to collect logs and what data manipulation to perform. KQL Transformation (makes you possible FilteringSpecify where to send the logs (workspace, Eventhubs (Azure VMS only), secondary tier, etc.) and where to parse, enrich, etc. You can group machines using different DCRs.

AMA’s DCR can be generated in several ways.

• Through the portal (Azure or Unified Security Operations Platform): This method provides a user-friendly interface for defining data collection scenarios.

• composition AMA based connectionR From Microsoft sentry > DCR is created via Configuration > Data Collection (Azure or Security Portal). This option sends data to Microsoft Sentinel tables, some of which are not accessible when defining a DCR in the Azure Monitor portal (e.g. CommonSecurityLog, SecurityEvent, WindowsEvent, and ASIM tables).

• Creating and Editing DCRss via Azure Monitor Go to Azure Portal > Azure Monitor > Settings > Data Collection Rules. Note: Creating a DCR using this UI does not make it accessible. Microsoft The sentinel table is not accessible and you need to edit the outputStream of the DCR to convert the data. Microsoft Sentinel’s Table).

• Azure Resource Manager (ARM) Template: ARM templates allow you to: Define DCR With code, you can deploy and manage it programmatically, which is useful for automation and consistency across environments.

• Azure CLI and PowerShell: These command-line tools provide another way to generate: DCR Management Programmatically, especially useful for scripting and automation.

• REST API: The Azure Monitor REST API allows you to: make, Update and manage DCR programmatically. This method provides the greatest flexibility and integration with other systems.

So why do we like AMAs better?

Ah! Easy:

• AMA performs better It outperforms existing agents by up to 25% compared to Linux OMS and 500% EPS improvement over MMA for Windows!

• DCR allows you to group machines, as they are centrally configured in the cloud. Azure policies allow you to scale deployments, upgrades, and configuration changes over time. Any changes to the DCR configuration are automatically rolled out to all agents, without requiring any additional action on the client side.

• AMA supports multi-homing, sending events to multiple workspaces, multiple regions, and multiple tenants (including Azure Lighthouse).

• Most importantly, AMA is more secure when connecting to the cloud using Managed Identity and Microsoft Entra tokens. When using machines that are not controlled by Azure, Arc enforces and processes authentication and ID tokens to secure the connection.

The best part is that AMA is constantly evolving with many improvements and enhancements we are constantly working on!

Next, let’s take a look at some notable changes to the connector.

Windows Security Events:
We have improved the schema of the SecurityEvent table that hosts Windows Security Events, and added new columns that will be populated in AMA versions 1.28.2 and later. These improvements are intended to provide better coverage and visibility into the collected events.

The newly added columns are:

• Keyword: A 64-bit mask of keywords defined for the event. The keywords classify the event type (e.g., events related to reading data), and the leftmost two bits indicate audit success or failure.

• Command: An action code and task that identifies the location in the application where the event was recorded.

• Event Record ID: Record number assigned when an event is recorded

• System thread ID: The thread that generated the event

• System Process ID: The process that generated the event

• Correlation: An activity identifier that consumers can use to group related events.

• Version: Version number of the event definition.

• System User ID: ID of the person in charge of the event

Common Event Format (CEF) and Syslog:
We all know how important it is to collect and analyze data from various sources such as firewalls, routers, switches, servers, DNS, and applications. Two of the most common protocols used by many devices to emit logs are: CEF and Syslog.

With legacy agents, you had to configure connectors separately for each source, which could be tedious and time-consuming. So Syslog and Safe Data connectors via AMA improve the overall experience of using Microsoft Sentinel data connectors. All devices will now depend on either the generic CEF or generic Syslog connector, depending on the protocol used by the log source. The relevant generic connectors are distributed as part of the device solution (don’t forget to check the box to install after clicking the ‘Install with dependencies’ button!).

We added a dedicated workbook installed with the solution to monitor log collection from disjoint device types using graphs, where device types are aggregated into a single location. You can further filter the view based on device type or connection status.

To simplify the logging, we’ve included several general instructions or references to help you set up your source device. CEF Appliances or System Log In our documentation.

Windows Events:
How do I collect other Windows audit events? These events cannot be sent to the SecurityEvents table because they do not come from a secure channel and do not match the table schema. Instead, non-security events can be sent to the WindowsEvents table using: Windows forwarding events To use the data connector that can be used to stream both forwarded events collected from WEC/WEF servers and Windows servers, you must set the DCR wizard to the Custom option and specify an XPath expression to point to the desired events.

Windows Firewall Log:
This connector allows you to collect firewall logs from your machine. Added granularity selection for the profile to collect and stream logs to the ASimNetworkSessionLogs table.

Custom logs:
Some devices packaged with the Content Hub solution stream data to the _CL table. To help you quickly set up file ingestion for those 15 specific devices, we’ve added: Custom log connector.

We hope you found this post informative and if you have already upgraded your agent to AMA or are planning to do so soon, please see: For more information on other connector agent-based or other Data Connector Documentation Or browse our content hub to find sources of interest. If you’d like more content on using AMAs, let us know in the comments below!

Finally, to stay up to date with the latest updates and announcements, check out: This is the new news page.