Setting Up Global Configuration

Global configurations are applied to all data sources as opposed to other configurations that are applied to a specific sensor. In other words, every message from every sensor is validated against global configuration rules. The format of the global enrichment is a JSON string-to-object map that is stored in ZooKeeper. For example:

{
  "es.clustername": "metron",
  "es.ip": "node1",
  "es.port": "9300",
  "es.date.format": "yyyy.MM.dd.HH",
  "fieldValidations" : [
              {
                "input" : [ "ip_src_addr", "ip_dst_addr" ],
                "validation" : "IP",
                "config" : {
                    "type" : "IPV4"
                           }
              } 
                       ]
}

where

es.ip

A single or collection of elastic search master nodes. They may be specified via the widely accepted hostname:port syntax. If a port is not specified, then a separate global property es.port is required:

Example: es.ip : [ “10.0.0.1:1234”, “10.0.0.2:1234”]

Example: es.ip : “10.0.0.1” (thus requiring es.port to be specified as well)

Example: es.ip : “10.0.0.1:1234” (thus not requiring es.port to be specified)

es.port

The port of the elastic search master node. This is not strictly required if the port is specified in the es.ip global property as described above. It is expected that this be an integer or a string representation of an integer.

Example: es.port : “1234"

Example: es.port : 1234

es.clustername

The elastic search cluster name to which you want to write.

Example: es.clustername : “metron” (providing your ES cluster is configured to have metron be a valid cluster name)

es.date.format

We shard the indices first by sensor and then by date. This provides the granularity time-wise that we shard.

Example: es.date.format : “yyyy.MM.dd.HH” (this would shard by hour creating, for example, a Bro shard of bro_2016.01.01.01, bro_2016.01.01.02, etc.)

Example: es.date.format : “yyyy.MM.dd” (this would shard by day, creating, for example, a Bro shard of bro_2016.01.01, bro_2016.01.02, etc.)

​Setting Up Global Configuration

Setting Up Global Configuration