SIEM Migration Update: Now Migrate with contextual depth in translations with Microsoft Sentinel!

What’s new in SIEM migration?

The process of moving from Splunk to Microsoft Sentinel through the SIEM migration experience has been enhanced with three key additions that help customers get more context-aware translations of what they are detecting in Splunk to Sentinel. These features allow customers to provide contextual details about their Splunk environment and usage to the Microsoft Sentinel SIEM migration translation engine so that it can take these into account when translating detections from SPL to KQL. These features are:

1. Schema Mapping
2. Splunk macro support in translation
3. Support for Splunk lookups in translation

Let’s walk through our SIEM migration experience to see how these features can make your life easier when migrating to Microsoft Sentinel.

Schema Mapping

How does it help?

Most existing translation tools only consider grammar translation when translating from one query language to another. More precisely, they deal with the “how” of queries. How are these queries structured? How are the operational and computational logic defined? Above all.

What often gets lost is the translation: “What data sources are being queried?” “What do these data sources actually map to in the target SIEM?”

“What” is often the environmental customer context that must be taken into account in translation to ensure that the grammar translation is applied to the correct source.

Approach

Schema Mapping in the SIEM Migration Environment allows you to define exactly how your Splunk sources (indexes, data models, etc.) map to Microsoft Sentinel tables in a new “Schema Mapping” section in the UI environment. This provides flexibility and customization to ensure your data aligns to your migration requirements. When you upload a Splunk export, the system extracts all sources from the SPL query. Known sources, such as Splunk CIM schemas and data models, are automatically mapped to ASIM schemas where applicable.. Other custom sources queried in the detection are listed unmapped and require manual mapping to existing Microsoft Sentinel/Azure Log Analytics tables. You can then review and modify all mappings or add new sources. The mapping schema is hierarchical. That is, Splunk sources are mapped 1-1 to Sentinel tables in addition to fields within those sources.

The best part? Manual changes to schema mappings are saved per workspace, so you never have to repeat them again.

Step-by-step instructions

To utilize schema mapping:

1. Move from Microsoft Sentinel Content Hub to a SIEM migration experience.
2. Review the prerequisites and “Next: Upload File“.
3. Follow the on-screen instructions to export your Splunk detection inventory and then upload it to Sentinel. “Next: Schema Mapping (Preview)“
4. Review the Splunk data sources identified in the export process. To review field mappings within a data source, select the Splunk source and a side panel with field mappings will open on the right.

5. Review, modify, and add schema mappings

• Data Source Mapping: To edit a Sentinel table that has a Splunk source mapped to it: to Select a Sentinel Table from the Sentinel Table dropdown.
• Field Mapping: To edit a field mapping, locate the Splunk field on the left whose mapping you want to change, then select the corresponding Sentinel field from the dropdown for that Splunk field.

• Add a new schema mapping: In scenarios where you cannot find a Splunk source identified and listed in the data sources list, “+ Add source”. Now, in the right panel, continue adding the name of your Splunk data source and select the Sentinel table from the drop-down menu. “+Add Mapping” Continue adding field mappings by directly entering the Splunk field name on the left and selecting the corresponding Sentinel field name on the right.

Once the change is complete, “Save Changes”. The mapping status now changes to “.Manually mapped“.

Once the schema mapping is complete, the changes will be taken into account when searches stored in SPL are converted to KQL queries.

Translation support for Splunk Lookups

Splunk Lookups, like Sentinel Watchlists, are lists of field-value pairs that can be used to query/correlate collected data. The SIEM Migration environment handles the conversion of Splunk Lookups from SPL queries (Splunk detections) to the use of Sentinel Watchlists generated from KQL queries.

Note: You must first create a Sentinel Watchlist to be able to map it to Splunk Lookups when you begin the migration.

Approach

A Splunk query is a complete data set, defined and available outside the scope of an SPL query, and an SPL query only references queries that are invoked with the “lookup”, “inputlookup”, and/or “outputlookup” keywords. Translation support is only available for the “lookup” and “inputlookup” keywords, which allow querying/correlating lookup data. The “outputlookup” operation, where data is written to a lookup, is not supported for translation, but can be implemented by defining automation rules in Microsoft Sentinel.

To translate the query call, SIEM Migration’s translation engine can use the “_GetWatchlist()” KQL function to map it to the correct Sentinel watchlist, and complement the operation with other KQL functions to translate the entire logic.

Step-by-step instructions

It is important to have this mapping context in your SIEM Migration experience to ensure proper Splunk Lookup to Sentinel Watchlist mapping. This experience now allows customers to map Splunk Lookups (automatically identified from uploaded Splunk queries) to Sentinel Watchlists (created outside of the experience as a prerequisite).

Follow the instructions here Create a Sentinel watchlist.

Once your watch lists are created, follow the instructions below to map these Sentinel watch lists to Splunk queries.

1. Move from Microsoft Sentinel Content Hub to a SIEM migration experience.
2. Review the prerequisites and “Next: Upload File“.
3. Follow the on-screen instructions to export your Splunk detection inventory and then upload it to Sentinel. “Next: Schema Mapping (Preview)“.
4. ” Clickcheck” tab and start reviewing/mapping your queries.
5. To add a field mapping, click on the Splunk Lookup you want to map and select the corresponding Sentinel Watchlist on the right in the right panel.

When you select a Sentinel Watchlist, you can complete the field mapping by selecting the Watchlist field from the field dropdown corresponding to the Lookup field on the left.

6. When you complete the review, “Save Changes”. The mapping status now changes to “.Manually mapped“.

Once all Splunk queries are reviewed, “Next: Configuring Rules“Start translating with KQL.

memo: If a Splunk query does not have a corresponding Sentinel Watchlist mapped to it, the translation engine will keep the names of the Sentinel Watchlist and its fields the same as the Splunk query and fields.

Translation support for Splunk macros

How does this help?

The core principles of developers are automation and feature reuse. Macros are essential for rapid development, but every architect quietly curses these “shortcuts” when it comes to migrating to a different technology stack.
When upgrading the SIEM migration experience, the team thought: What if someone said to the architect, “Hey, we’ve done this all by ourselves?” All (SPL) detection queries are seamlessly extended by replacing macro references inline with their corresponding macro definitions, and are passed to the translation engine so that the core detection logic is preserved when language translation occurs.

Approach

To use this macro expansion, the experience needs more context and data. This could be a mapping of data fields or context to the Splunk code associated with the macro. This enrichment is done through the initial file query and uploader, and now there is a richer query that retrieves the information needed (the metadata of the detection and all macro definitions). This additional information helps identify and ensure that all the pieces of the puzzle are in place before translation.

Step-by-step instructions

Don’t worry, there are no extra steps here. 🙂

Experience: The process of getting the import file needed for migration by “copying the query and running it in Splunk” remains the same. As mentioned earlier, the query has been improved to get a broader context in an updated format.

There are no additional touchpoints, the migration experience handles the rest and shows you the experience.anded source query with macro references replaced inline for each definition In the “Configure Rules” tab