Using Pipelines for Windows Log Processing
Windows logs can be very informative, providing a perfect picture of the activities happening on an endpoint. Unfortunately, the logs can also be hard to decipher when you first start examining them. Graylog uses Pipelines and Data Adapters to enrich logs to make them more functional and easier for you to read.
For example, when you fail to log onto a Windows domain or a Windows computer, you get an error message like the following:
You can easily see the account name was bob and the computer trying to log in was Nicks-MacBook-Pro, but what about that error code? What does 0xC000006A mean? After some Googling, you find out that it means “The user’s password is wrong”, but wouldn’t it be nice to have that clearly written in the logs from the start?
We can set that up from start to finish, using a lookup table and a pipeline rule to modify the data collected by a Winlogbeat collector. Complete the following steps:
1. Start by putting in the lookup table on the server. Copy the file to a directory on the Graylog server (in this example /etc/graylog/lookup_error.csv).
2. Create a Lookup Table (System -> Lookup Tables) and in the “Data Adapters” Section select “Create data adapter”, and fill in one like below:
3. Next, create a Cache with the “Create cache” button like below:
4. In the last step in the Lookup Table, create the table itself, using the created data adapter and cache from the last two steps:
5. Now that you’ve created the lookup table and it’s ready for use, create a pipeline rule to use the table and change the unreadable code to something a bit more useful. Go to (System -> Pipelines) and under the “Manage rules”, create a new rule. Give it a description so you can remember it, and in the Rule Source field, type:
rule "WinlogBeat Cleanup"
//Change Logon Code
let update_source = lookup_value("lookup_error", $message.winlogbeat_event_data_SubStatus);
You can see the first when condition is allowing the rule to process only rules with EventID 4625 and events containing the field winlogbeat_event_data_SubStatus. Once it finds those logs, it runs the lookup against our Lookup Table “lookup_error” with the data in the SubStatus field, then replaces the code. You change your field from:
Now it’s easier to understand why the logon failure happened, and you can add data like this on other Dashboards and Views for your SOC to monitor.