Understanding Global Configuration

The global configuration file is a repository of properties that can be used by any configurable component in the system. The global configuration file can be used to assign a property to multiple parser topologies. For example, every message from every sensor is validated against global configuration rules. The global configuration file can also be used to assign properties to enrichments and the profiler which each use a single topology. For example, you can use the global configuration to configure the enrichment topology’s writer batching settings.

The following is an index of the global configuration properties and their associated Apache Ambari properties if they are managed by Ambari.

Table 1. Global Configuration Properties
Property Name Subsystem Type Ambari Property
es.clustername Indexing String es_cluster_name
es.ip Indexing String es_hosts
es.port Indexing String es_port
es.date.format Indexing String es_date_format
es.client.settings Indexing Object N/A
indexing.writer.elasticsearch.setDocumentId Indexing Boolean N/A
solr.zookeeper Indexing String solr_zookeeper_url
solr.commitPerBatch Indexing String N/A
solr.commit.soft Indexing String N/A
solr.commit.waitSearcher Indexing String N/A
solr.commit.waitFlush Indexing String N/A
solr.collection Indexing String N/A
solr.http.config Indexing String N/A
fieldValidations Parsing Object N/A
parser.error.topic Parsing String parser_error_topic
stellar.function.paths Stellar CSV String N/A
stellar.function.resolver.includes Stellar CSV String N/A
stellar.function.resolver.excludes Stellar CSV String N/A
profiler.period.duration Profiler Integer profiler_period_duration
profiler.period.duration.units Profiler String profiler_period_units
profiler.client.period.duration Profiler Integer profiler_period_duration
profiler.client.period.duration.units Profiler String profiler_period_units
profiler.writer.batchSize Profiler Integer N/A
profiler.writer.batchTimeout Profiler Integer N/A
update.hbase.table REST/Indexing String update_hbase_table
update.hbase.cf REST-Indexing String update_hbase_cf
user.settings.hbase.table REST/Indexing String user_settings_hbase_table
user.settings.hbase.cf REST/Indexing String user_settings_hbase_cf
geo.hdfs.file Enrichment String geo_hdfs_file
enrichment.writer.batchSize Enrichment Integer N/A
enrichment.writer.batchTimeout Enrichment Integer N/A
enrichment.list.hbase.provider.impl Enrichment String enrichment_list_hbase_provider_impl
enrichment.list.hbase.table Enrichment String enrichment_list_hbase_table
enrichment.list.hbase.cf Enrichment String enrichment_list_hbase_cf
source.type.field UI String source_type_field
threat.triage.score.field UI String threat_triage_score-_field

You can also create a validation using Stellar. The following validation uses Stellar to validate an ip_src_addr similar to the "validation":"IP"" example above:

"fieldValidations" : [
                "validation" : "STELLAR",
                "config" : {
                    "condition" : "IS_IP(ip_src_addr, 'IPV4')"