Gathering logs that contain IP addresses are quite common across your infrastructure. Your firewalls, web servers, wireless infrastructure and endpoints can contain IP addresses outside your organization. Having additional data on those logs that gives you the Geolocation of the IP address helps in your investigations and understanding of your traffic patterns.
For Example, if you can see logs on a World Map, you know if you are communicating to a country you don’t normally talk to.
Now you can easily see traffic to Brazil, something your organization might not normally do.
Let’s go through all the steps you will need to do on your Graylog box to get your system enriching your log data.
The first step is to download the database of Geolocation information. In this guide we will be using the MaxMind GeoLite2 Database in the binary format (.mmdb). These files can be downloaded here.
Once these files are downloaded, the next step is to store the geolocation database on all servers running Graylog, and ensure the file permissions are set to allow Graylog the ability to read the file. In this example, the file is located at /etc/graylog/server/GeoLite2-City.mmdb
Now we need to configure the Lookup Table to read the database when an input is given to it. We will first create the data adapter under System -> Lookup Tables and Select “Data Adapters”. On this page, select “Create data adapter”, and fill in one like below:
Next, we need to create a Cache with the “Create cache” button like below:
In the last step in the Lookup Table, we need to create the table itself, using the created data adapter and cache from the last two steps:
Now that the lookup table has been created and is ready for use, we need to create a pipeline rule to utilize it, and add in the metadata to each message with an IP address.
Go to (System -> Pipelines) and under the “Manage rules” we will need to create a new rule. Give it a description so you can remember it, and in the Rule Source put:
rule "GeoIP lookup: src_ip"
let geo = lookup("geoip", to_string($message.src_ip)));
The when condition is only allowing the rule to process when the log has the field src_ip. NOTE: Please change this to the field containing IP addresses in Your logs. Once it finds those logs, it runs the lookup against our Lookup Table “GeoIP” with the data in the src_ip field, then adds in the Location, Country and City.
Notice, this rule only applies to the source IP address. If destination address should be looked up as well, add additional lines to this rule, or create a second rule for logs with destination IP addresses.
Once the rules are created, add them to a stage in the pipeline processing the logs with an IP address in them, and the enrichment will start working.
And once new logs come through the pipeline, you will see the enriched log entries.
Finally, if you run a search aggregate on “src_ip_geo_location” and change the table type to be a “World Map” you will get:
In the screenshot below, you can see there are many fields in the lookup table. You could pull additional data out and put it into the logs if you would like to help enrich your logs further.