Now that you have the threat intel source, threat intel extractor, and threat intel
mapping config defined, you can run the loader to move the data from the threat intel source to
the Metron threat intel Store and store the enrichment config in
ZooKeeper.
- Log into the $HOST_WITH_ENRICHMENT_TAG as root.
- Run the loader:
/usr/metron/$METRON_RELEASE/bin/flatfile_loader.sh -n enrichment_config.json -i domainblocklist.csv -t threatintel -c t -e extractor_config.json
The previous command adds the threat intel data into HBase and establishes a ZooKeeper
mapping. The data is populated into an HBase table called threatintel.
- To verify that the logs were properly ingested into HBase, run the following command:
hbase shell
scan 'threatintel'
- Now check if the ZooKeeper enrichment tag was properly populated:
/usr/metron/$METRON_RELEASE/bin/zk_load_configs.sh -m DUMP -z $ZOOKEEPER_HOST:2181
You should see a configuration for the Squid sensor something like the
following:
{
"index" : "squid",
"batchSize" : 1,
"enrichment" : {
"fieldMap" : {
"hbaseEnrichment" : [ "ip_src_addr" ]
},
"fieldToTypeMap" : {
"ip_src_addr" : [ "user" ]
},
"config" : { }
},
"threatIntel" : {
"fieldMap" : { },
"fieldToTypeMap" : { },
"config" : { },
"triageConfig" : {
"riskLevelRules" : { },
"aggregator" : "MAX",
"aggregationConfig" : { }
}
},
"configuration" : { }
}
- Generate some data by using the Squid client to execute http requests.
squidclient http://www.actdhaka.com