Run Book - Technical Preview
Also available as:
PDF

Run the Threat Intelligence Loader

Now that you have the threat intel source, threat intel exractor, and threat intel mapping config defined, you can run the loader to move the data from the threat intel source to the Metron threat intel Store and store the enrichment config in ZooKeeper.

  1. Log into the $HOST_WITH_ENRICHMENT_TAG as root.

  2. Run the loader:

    /usr/metron/$METRON_RELEASE/bin/flatfile_loader.sh -n enrichment_config.json -i domainblocklist.csv -t threatintel -c t -e extractor_config_temp.json
    

    The previous command adds the threat intel data into HBase and establishes a ZooKeeper mapping. The data is populated into an HBase table called threatintel.

  3. To verify that the logs were properly ingested into HBase, run the following command:

    hbase shell
    scan 'threatintel'
  4. Now check if the ZooKeeper enrichment tag was properly populated:

    /usr/metron/$METRON_RELEASE/bin/zk_load_configs.sh -m DUMP -z $ZOOKEEPER_HOST:2181

    You should see a configuration for the Squid sensor something like the following

    {
      "index" : "squid",
      "batchSize" : 1,
      "enrichment" : {
        "fieldMap" : {
          "hbaseEnrichment" : [ "ip_src_addr" ]
        },
        "fieldToTypeMap" : {
          "ip_src_addr" : [ "user" ]
        },
        "config" : { }
      },
      "threatIntel" : {
        "fieldMap" : { },
        "fieldToTypeMap" : { },
        "config" : { },
        "triageConfig" : {
          "riskLevelRules" : { },
          "aggregator" : "MAX",
          "aggregationConfig" : { }
        }
      },
      "configuration" : { }
    }
  5. Generate some data by using the Squid client to execute http requests.

    squidclient http://www.actdhaka.com