Suricata + ELK in Docker

While getting familiar the very popular Docker Linux container tool, I went against best practice and put Suricata, Logstash, Elastic Search and Kibana into a container that is looking promising for demonstration purposes. If you already run this stack on one machine, it might be suitable for real use as well. What you get is a very simple to run application container that abstracts all the tools above into a single application. Assuming you have Docker already installed, you can get a feel for Suricata + ELK with a couple commands:

git pull https://github.com/jasonish/docker-suricata-elk.git
cd docker-suricata-elk
./launcher start -i eth0

The first time ./launcher start is run, Docker will pull down the container file system layers so it may take a while. Subsequent starts will be much quicker. Once it looks like it is up and running, point your browser at http://localhost:7777. A few notes:

• Docker containers are more or less stateless. Changes to the filesystem inside the container are not persisted over a restart. Instead any data that needs to be persisted will end up in the ./data directory where you started the launcher.
• This container uses host networking instead of the usual isolated network you find with Docker containers. This is to give the container access to your physical interfaces. This alone has me questioning Docker for network monitoring deployments.
• As host networking is used, the container will probably fail if you have existing applications bound to port 7777 or 9200. Making these ports configurable is on the todo.
• The containers log directory is available from the host system. Take a look in ./data/log.
• Suricata is built from git master.
• ./launcher enter will give you a shell inside the running container. This is useful to take a look around the runtime environment. Just remember that any changes you make will not be persistent.
• ./launcher bash will start a new container with the bash shell and nothing running. This is mostly useul for development.
• If running a VM, allocate 2GB of memory and/or create a swap file. These are not lightweight applications.

Project links:

• https://github.com/jasonish/docker-suricata-elk
• https://registry.hub.docker.com/u/jasonish/suricata-elk/

Snort, Logstash, Elastic Search and Kibana...

After having fun with Suricata's new eve/json logging format and the Logstash/Elastic Search/Kibana combination (see this and this), I wanted to get my Snort events into Elastic Search as well.  Using my idstools python library I wrote u2json, a tool that will process a unified2 spool directory (much like barnyard) and convert the events to Suricata-style JSON.

Usage is relatively simple, assuming Snort is logging to /var/log/snort, the following command line should do:

  idstools-u2json -c /etc/snort/snort.conf \
    --directory /var/log/snort
    --prefix unified2.log
    --follow --bookmark
    --output /var/log/snort/alerts.json

As the output is in the same format as Suricata's you can refer to this guide for the Logstash setup.

One extra step I did was use Logstash to add an "engine" field to each entry.  This can be accomplished by adapting the following Logstash configuration:

input {
  file {
    path => ["/var/log/suricata/eve.json"]
    codec => json
    type => "suricata-json"
  }
  file {
    path => ["/var/log/snort/alerts.json"]
    codec => json
    type => "snort-json"
  }
}

filter {
  if [type] == "suricata-json" {
    mutate {
      add_field => {
        "engine" => "suricata"
      }
    }
  }

  if [type] == "snort-json" {
    mutate {
      add_field => {
        "engine" => "snort"
      }
    }
  }
}

Checkout out the documentation for information.