Starting pcap
To start pcap, HCP provides a utility script. This script takes no arguments and is very simple to run. Complete the following steps to start pcap:
Log in to the host on which you are running Metron.
If you are running HCP on an Ambari-managed cluster, perform the following steps; otherwise proceed with Step 3:
Update the
$METRON_HOME/config/pcap.properties
by changingkafka.zk
to the appropriate server.You can retrieve the appropriate server information from Ambari in Kafka service > Configs > Kafka Broker > zookeeper.connect.
On the HDFS host, create
/apps/metron/pcap
, change its ownership to metron:hadoop, and change its permissions to 775:hdfs dfs -mkdir /apps/metron/pcap hdfs dfs -chown metron:hadoop /apps/metron/pcap hdfs dfs -chmod 755 /apps/metron/pcap
Create a Metron user's home directory on HDFS and change its ownership to the Metron user:
hdfs dfs -mkdir /user/metron hdfs dfs -chown metron:hadoop /user/metron hdfs dfs -chmod 755 /user/metron
Create a pcap topic in Kafka:
Switch to metron user:
su - metron
Create a Kafka topic named
pcap
:/usr/hdp/current/kafka-broker/bin/kafka-topics.sh \ --zookeeper $ZOOKEEPER_HOST:2181 \ --create \ --topic pcap \ --partitions 1 \ --replication-factor 1
List all of the Kafka topics, to ensure that the new pcap topic exists:
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER_HOST:2181 --list
If HCP is installed on an Ambari-managed cluster, use the following command to start the pcap topology:
su - metron $METRON_HOME/bin/start_pcap_topology.sh
If HCP is installed by CLI, use the following command to start the pcap topology.
$METRON_HOME/bin/start_pcap_topology.sh
Check the Storm topology to ensure that packets are being captured.
After Storm has captured a sufficient number of packets, you can check to ensure it is creating files on HDFS:
hadoop fs -ls /apps/metron/pcap