Start Topologies and Send Data
The final step in setting up Model as a Service, is to start the topologies and send some data to test the model.
-
Start the sensor upon which the Model as a Service will run:
$METRON_HOME/bin/start_parser_topology.sh -k -$KAFKA_HOST -z $ZOOKEEPER_HOST:2181 -s $PARSER_NAME
-
Generate some legitimate data and some malicious data on the sensor.
For example:
#Legitimate example: squidclient http://yahoo.com #Malicious example: squidclient http://cnn.com
-
Send the data to Kafka:
cat /var/log/squid/access.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $KAFKA_HOST:6667 --topic squid
-
Browse the data in Elasticsearch at
http://$ELASTICSEARCH_HOST:9100/_plugin/head
to verify that it contains the appropriate documents.For the current example, you would see the following:- One from yahoo.com which does not have
is_alert
set and does haveis_malicious
set tolegit
. - One from cnn.com which does have
is_alert
set to true,is_malicious
set tomalicious
, andthreat:triage:level
set to 100.
- One from yahoo.com which does not have