Loading Telemetry Information into Zeppelin
Before you can analyze telemetry information in Zeppelin, you must first download it from Metron. Metron archives the fully parsed, enriched, and triaged telemetry for each sensor in HDFS. This archived telemetry information is simply raw JSON files which makes it simple to parse and analyze the information with Zeppelin. The following is an example of some Bro telemetry information.
%sh hdfs dfs -ls -C -R /apps/metron/indexing/indexed/bro /apps/metron/indexing/indexed/bro/enrichment-null-0-0-1484124296101.json /apps/metron/indexing/indexed/bro/enrichment-null-0-0-1484128332104.json /apps/metron/indexing/indexed/bro/enrichment-null-0-0-1484131460758.json /apps/metron/indexing/indexed/bro/enrichment-null-0-1-1484217861096.json /apps/metron/indexing/indexed/bro/enrichment-null-0-10-1484995461039.json /apps/metron/indexing/indexed/bro/enrichment-null-0-11-1485081861043.json /apps/metron/indexing/indexed/bro/enrichment-null-0-12-1485168261040.json /apps/metron/indexing/indexed/bro/enrichment-null-0-13-1485254661040.json /apps/metron/indexing/indexed/bro/enrichment-null-0-14-1485341061047.json /apps/metron/indexing/indexed/bro/enrichment-null-0-15-1485427461040.json /apps/metron/indexing/indexed/bro/enrichment-null-0-16-1485513861039.json /apps/metron/indexing/indexed/bro/enrichment-null-0-17-1485600261045.json /apps/metron/indexing/indexed/bro/enrichment-null-0-18-1485686661035.json /apps/metron/indexing/indexed/bro/enrichment-null-0-19-1485773061037.json /apps/metron/indexing/indexed/bro/enrichment-null-0-2-1484304261042.json /apps/metron/indexing/indexed/bro/enrichment-null-0-20-1485859461037.json /apps/metron/indexing/indexed/bro/enrichment-null-0-21-1485945861039.json /apps/metron/indexing/indexed/bro/enrichment-null-0-22-1486032261036.json
You can use Spark to load the archived information from HDFS into Zeppelin.
For example if you are loading information received from Bro, your command would look like the following:
%spark sqlContext.read.json("hdfs:///apps/metron/indexing/indexed/bro").cache().registerTempTable("bro")