Starting PCAP
To start PCAP, HCP provides a utility script. This script takes no arguments and is very simple to run. Complete the following steps to start PCAP:
Log into the host on which you are running Metron.
If you are running HCP on an Ambari-managed cluster, perform the following steps. If you are running a VM or a cluster that is not managed by Ambari, skip to Step 3.
Update the
$METRON_HOME/config/pcap.properties
by changingkafka.zk
to the appropriate server.You can retrieve the appropriate server information from Ambari in Kafka service > Configs > Kafka Broker > zookeeper.connect.
On the HDFS host, create
/apps/metron/pcap
, change its ownership to metron:hadoop, and change its permissions to 775.hdfs dfs -mkdir /apps/metron/pcap hdfs dfs -chown metron:hadoop /apps/metron/pcap hdfs dfs -chmod 755 /apps/metron/pcap
Create a Metron user's home directory on HDFS and change its ownership to the Metron user.
hdfs dfs -mkdir /user/metron hdfs dfs -chown metron:hadoop /user/metron hdfs dfs -chmod 755 /user/metron
Create a PCAP topic in Kafka.
Switch to metron user:
su - metron
Create a Kafka topic named PCAP:
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh \ --zookeeper $ZOOKEEPER_HOST:2181 \ --create \ --topic pcap \ --partitions 1 \ --replication-factor 1
List all of the Kafka topics, to ensure that the new PCAP topic exists:
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER_HOST:2181 --list
Start the PCAP topology:
$METRON_HOME/bin/start_pcap_topology.sh
If HCP is installed on an Ambari-managed cluster, proceed the previous command with
su - metron
.Check the Storm topology to ensure that packets are being captured.
After Storm has captured a sufficient number of packets, you can check to ensure it is creating files on HDFS:
hadoop fs -ls /apps/metron/pcap