So you’ve installed Graylog, now what? Let’s have a quick look to set you on your path.
INPUTS
From here, under system inputs, you’ll see a variety of different types of inputs we have in our log server here. We’ve got some AWS flow logs and CloudTrail different CEF types. We’ve got GELF different formats for HTTP, CAFCA or TCP UDP, but you can see we’ve got JSON path, NetFlow. We’ve got spotlights for Office 365 Okta and Palo Alto and a variety of raw plain text type formats and SIS log formats. What I’m going to do is I’m going to show you this input here, I just created earlier this input here. If we edit it, you’ll see it’s just basic GELF input. We set it up with its specific port number. What I’m going to do is, once you’ve created your input, you can show, receive messages.
In this case, I’ve got some messages from previous, but I’m going to look in the last five minutes for which we don’t have any. What I’m going to do is in our documentation, it shows how you can send a dummy message or just a test message to your Graylog server. From here, I’m going to bring up Linux and I am going to send, as you’ll see, this message three times, let’s go four for good luck. You’ll see here, the message that’s just shown up that I’ve just sent was it, hello there’s my first message. There’s my first message, test message.
INDICES
Inputs and index sets are very important. They’re created to contain unnecessary settings to create, manage, and fill elastic search indices and handle rotation and data retention. What we’ll do here is under system indices. You’ll see a variety of them that I’ve got already in the system. I’m going to create a new one and I’m going to call it GELF HTTP description is going to be GELF HTTP info, and I’m going to create an index that is semi-standard to our approach. I’m going to keep everything else pretty much default for this input. I’m going to save it. Now we have this index and it’s important for later when we want to attach streams to an index, and that will cover it.
STREAMS
Looking at streams, they’re connected to indices or index sets. They are used to route your logs and messages into groups or categorize them in real-time. Looking here, you’ll see under streams. If we utilize the Illuminate package or Illuminate spotlight, it’s part of our enterprise here for Illuminate over 60, 65 logs. We edit the stream. You’ll see that it’s been assigned to the index set of O365 logs. If you click on this actual stream, this is where it will take you in real-time to data that’s flowing in on that stream currently. Here, you’ll see with my information, I have a bunch of logs that have been parsed and coming in on that stream.
PIPELINES&RULES
Let’s take a look at pipelines and rules. Pipelines are attached to streams. Pipelines and rules are where the magic happens to your log data. Let’s have a quick look here. What I’m highlighting is our search window for our Office 365 spotlight. In here, you’ll see all these logs have been enriched so that they’ve been separated into parsable fields and made much easier to understand. If we take this as an example and we go to a system and then we go to pipelines, here you’ll see a whole bunch of pre-built ones, which we use as part of our Illuminate enterprise package. In each one individually, if we started this first one you’ll see multiple stages. Multiple stages are set up to have different modifications to the logs as they come in. Any one of these groups, as you’ll see here, is a rule to modify or manipulate the logs and pass it on to the next stage.
LOOKUP TABLES
Look-up tables are used to look up, map, or translate field values into new values and write them into fields. As an example of this, using a CSV flat text file for mapping IP addresses to hostnames. If we bring it up here under system and lookup tables, here you’ll see the very first one is an Illuminate look-up table for our spotlight for Illuminate the enterprise feature that comes with the package. This one here contains a cache and a data adapter. If we go to a separate tab here and look at the cache, this is the parameters of the cache. When it expires, go back to here under the data adapter, and we look at the actual data adapter, this data adapter is creating a lookup in a database file. In this case, it’s a max mind database file, and it allows you to lookup an IP address in this field and test it. Where this is really useful is if we go back here under dashboards and say, we want to pick the Office 365 dashboard, and we want to change our time here and have a look at our data.
This is what makes this useful here so that you can map your actual IP addresses, where they’re coming from for different types of information. This can be used in a wide range of things within Graylog.
