Today we are releasing the first Release Candidate of Graylog v3.1.
This release brings a whole new alerting and event system that provides more flexible alert conditions and event correlation based on the new search APIs that also power the views. In addition, some extended search capabilities introduced in Graylog Enterprise v3.0 are now available in the open source edition in preparation for unifying the various search features.
Support for building search workflows with parameters remains a Graylog Enterprise function and will be enhanced in future releases once the search unification work is completed.
Please read on for detailed descriptions of each feature.
DOWNLOAD LINKS
- DEB or RPM packages are available in our repositories. Check our documentation for details.
- Docker image
- OVA / Appliance
- Tarball (manual installation)
Please report bugs and any other issues in our GitHub issue tracker. Thank you!
New: Alerts and Events
Turn your log messages into normalized events by defining filters and aggregations and alert on them directly or correlate multiple events over time to express more complex conditions.
Log messages can take many different forms and usually contain a lot of noise. At the same time, not everything noteworthy in logs is necessarily an alert. In order to overcome this gap, Graylog now allows you to define events, through conditions such as filters and aggregations, which form the elements of alerts.
Because those events are also stored in Elasticsearch all Graylog functionality can be used to query, organize and archive them, long beyond the lifetime of the raw, noisy logs that created them.
In Graylog Enterprise you can also correlate events over time to trigger notifications when a combination of events appear or even a certain combination of events does not happen in a certain amount of time.
For example if you are collecting logs about software deployments, you can define events that describe when service was stopped or started from its application server logs. By themselves these are not alerts, but describe something noteworthy about the service that is extracted from the noise of all the other informational messages in those logs.
By correlating stop and start events Graylog Enterprise lets you express a new alert event when, for example, a service that has stopped does not start again within 30 seconds. This new alert can be passed to Graylog’s notification system to trigger your incident management system, send emails or trigger the appropriate Slack channels.
Like other events, correlation events are also stored in Elasticsearch, and are available for further filtering, aggregation or correlation rules, allowing you to build very powerful alert expressions over your data.
In an information security context, events could describe successful and failed logins across a variety of platforms and correlation rules can alert you when a large number of failed logins are followed by a successful login for the same user, indicating a possible brute force attempt.
All events (and thus alerts) in the system have what is called a key: The key tells Graylog what the topic of the event is. In the case of logins it can be the user name and target IP address, in the case of software services it can be the host name and the name of the deployed application.
Different events that have the same key can be correlated, in other words they can be treated as a group.
Prior versions of Graylog could not produce keyed events which required overly specific alert conditions, with the addition of keys the same type of event can now describe different topics. In the context of login events this allows us to write a single condition describe what makes up a login event, and then scoping that to the subject of the event: the user that logged in.
Events and alerts are all run as searches and Graylog now keeps track of which underlying data it has seen, which time frames it produced events and alerts for already, and it will ensure that there are no gaps in coverage – even if the search cluster is temporarily slow or disconnected.
Graylog Enterprise will also run event and alert searches across all Graylog nodes in the cluster in parallel, thereby greatly increasing the throughput when a large number of alerts are defined in the system.
Let us know what you’d like to have included in our GitHub issue tracker.
