Conversion from Sigma Community to KQL That Works

The Sigma Translation Problem

The Sigma detection format has become the de facto standard for vendor-agnostic detection rules. With over 3,000 rules in the SigmaHQ community repository, it represents an enormous body of collective detection knowledge. However, translating these rules into production-ready KQL for Microsoft Sentinel has historically been unreliable.

Existing translation tools - including the official sigma-cli with the microsoft365defender and azure-monitor backends - produce output that frequently fails to execute. Field mappings are incomplete, log source translations are inconsistent, and KQL-specific operators are often incorrectly applied.

Where Existing Tools Fall Short

We analyzed 500 randomly selected Sigma rules from the community repository and ran them through available conversion tools. The results were sobering:

The gap between "syntactically valid" and "production-ready" is significant. A query that parses correctly but uses wrong field names or missing table references is worse than no query at all - it creates a false sense of coverage.

Our Approach

We built sigma2kql, an open-source conversion tool that prioritizes correctness over completeness. The design principles:

1. Fail loudly - If a field mapping is uncertain, emit a warning rather than guessing
2. Table-aware - Map Sigma log sources to specific Sentinel tables with validated schemas
3. Operator fidelity - Translate Sigma modifiers (contains, startswith, endswith, re) to their exact KQL equivalents
4. Post-processing - Apply KQL best practices like has vs contains optimization automatically

Field Mapping Architecture

Rather than maintaining a single flat mapping file, sigma2kql uses a layered approach:

Sigma logsource → Sentinel table → Field mapping per table

For example, a Sigma rule targeting category: process_creation maps to:

process_creation:
  tables:
    - SecurityEvent        # Legacy agent
    - DeviceProcessEvents  # MDE
    - SysmonEvent          # Sysmon via AMA
  field_maps:
    SecurityEvent:
      Image: NewProcessName
      ParentImage: ParentProcessName
      CommandLine: CommandLine
      User: SubjectUserName
    DeviceProcessEvents:
      Image: FolderPath
      ParentImage: InitiatingProcessFolderPath
      CommandLine: ProcessCommandLine
      User: AccountName

This ensures that field names are always correct for the target table, and supports generating multi-table queries when appropriate.

Handling Complex Modifiers

Sigma's modifier system is powerful but creates translation challenges. Consider this rule fragment:

detection:
  selection:
    CommandLine|contains|all:
      - 'bypass'
      - '-enc'
    CommandLine|re: '(?i)(invoke|iex)\s*\('

Our tool translates this to optimized KQL:

| where ProcessCommandLine has "bypass"
    and ProcessCommandLine has "-enc"
    and ProcessCommandLine matches regex @"(?i)(invoke|iex)\s*\("

Note the use of has instead of contains for the simple string matches - has uses the inverted index and is significantly faster in Sentinel.

Results

After building and validating sigma2kql against the same 500-rule sample:

The remaining 6% of syntactically invalid output comes from Sigma rules that use features not yet supported (primarily near temporal proximity and certain aggregation patterns). These generate clear error messages rather than incorrect output.

Getting Started

The tool is available on our GitHub and can be installed via pip:

pip install sigma2kql

# Convert a single rule
sigma2kql convert rule.yml --target sentinel

# Batch convert the entire SigmaHQ repository
sigma2kql convert ./sigma/rules/ --target sentinel --output ./kql-rules/

# Validate existing KQL rules against Sigma source
sigma2kql validate --sigma ./sigma/rules/ --kql ./kql-rules/

We welcome contributions, especially for field mapping coverage in less common log sources. The project includes a comprehensive test suite that validates conversions against known-good KQL output for every supported Sigma rule category.