After you configure the extractor configuration file and the element-enrichment
mapping, you must run the loader to move the data from the enrichment source to the CCP
enrichment store and store the enrichment configuration in Apache ZooKeeper.
-
If you will be bulk loading enrichments using the MR mode, add the metron user to the
access control list of users and groups in the Yarn Queue Manager.
You do not need to perform this step if you use the LOCAL mode.
-
In Ambari, click (Ambari views) and choose Yarn Queue
Manager.
-
In the upper left corner of the window, under Add Queue,
select default.
Ambari displays the default information on the right side of the window.
-
By Submit Applications, click
Custom.
-
In the Users field, add metron.
For example:
- Use the loader to move the enrichment source to the enrichment store in ZooKeeper:
$METRON_HOME/bin/flatfile_loader.sh -n enrichment_config.json -i whois_ref.csv -t enrichment -c t -e extractor_config.json
CCP loads the enrichment data into Apache HBase and establishes a ZooKeeper mapping.
The data is extracted using the extractor and the configuration is defined in the
extractor_config.json
file and populated into an HBase table called
enrichment
.
- Verify that the logs were properly ingested to HBase:
hbase shell
scan 'enrichment'
- Verify that the ZooKeeper enrichment tag was properly populated:
$METRON_HOME/bin/zk_load_configs.sh -m DUMP -z $ZOOKEEPER_HOST:2181
- Generate some data by using a client for your particular data source to execute requests.