How to Validate Your Sigma Rules Using Sigmalint

Sigma is a useful format for writing generic detection rules in YAML that can then be translated for use on multiple platforms and shared efficiently. However, when creating new Sigma rules, it can be tough to nail down exactly how to craft the YAML files so that they translate correctly. It’s also challenging to know the quality of a Sigma rule that you acquire from somewhere else. One way to gain some insight into these problems is to validate your Sigma rules. Read on to learn how sigmalint makes this easy. But first, let’s look at the current landscape.

The Current Landscape

There are a few ways of validating Sigma rules. The most tedious of these methods is to read the specification docs on the Sigma project wiki and manually review your files. Here, the maintainers have outlined the official specification for crafting sigma rules, which is also the base for the conversion tools included in the repository. This page displays the Sigma schema in multiple formats, including thorough descriptions of each field and its contents. This page is overall a useful tool for understanding the Sigma schema, but when you have 100+ rules to validate, it becomes too time intensive.

Another way to validate your sigma rules is to use the test suite from the Sigma project repository. The test suite is another valuable tool, but there are a couple of issues relying on it to validate your Sigma rules. First, it’s incomplete. While there are many tests in the suite, it’s not a comprehensive check of your compliance with the Sigma schema. The second issue is that it can be inconvenient to run the test suite on your rules. The quickest way to run them on your rules would be to pull down the Sigma repository and place your rules into the “rules” folder. Additionally, you could pick out the python tests you want to use out of the repository and reconfigure them to work with your rule repository.

What we really want is an automated way to verify that all of the Sigma rules we’re using are compliant with the Sigma schema.

Introducing Sigmalint

At Stage 2 Security, we’ve developed sigmalint to achieve this. Sigmalint is a simple command-line utility that can run your Sigma rules through various validation methods to check schema compliance.

The first validation method is Rx. Rx is a data validation framework written for JSON and YAML and available for many programming languages. In sigmalint, we use the Rx schema file from the Sigma repo to validate our rule structure. One issue with Rx is that it doesn’t give great feedback on why a rule fails validation.

To solve this problem, we introduced a second validation method, JSON Schema. JSON Schema is another data validation framework. As its name suggests, the purpose of JSON Schema is to validate JSON objects. However, with the JSON Schema python module, all we’re doing is passing in a python dictionary, so we can also use it to validate YAML loaded with PyYAML. We also created an approximation of the Rx schema in the JSON Schema format.

We then ran into a final issue: with both the Rx and JSON schemas, many of the rules in the public Sigma repo we’re failing validation. To fix this, we created a second version of the JSON Schema schema with some fixes. The main issue appeared to be around describing how the detection field in a Sigma rule can be formatted as it can get relatively complex. With that discrepancy fixed and a few other issues smoothed out, we had a version of the JSON Schema schema that marked almost all of the rules in the Sigma public repository as valid.

How Sigmalint Works

Using sigmalint is easy. You can pass two parameters: inputdir and method. inputdir is the directory location of your rules, and method is the validation system you want to use (rx, jsonschema, s2).

The script then outputs a report of how many rules are valid and invalid and how many it couldn’t parse.

See below for a demo using the Sigma repository rules.

With this report, you can now go into your invalid Sigma rules and easily fix the issues present.

For more info and to contribute to sigmalint, head over to our github page.