Understanding Global Configuration
The global configuration file is a repository of properties that can be used by any configurable component in the system. The global configuration file can be used to assign a property to multiple parser topologies. For example, every message from every sensor is validated against global configuration rules. The global configuration file can also be used to assign properties to enrichments and the profiler which each use a single topology. For example, you can use the global configuration to configure the enrichment toplogy's writer batching settings.
The following is an index of the global configuration properties and their associated Apache Ambari properties if they are managed by Ambari.
Important | |
---|---|
Any property that is managed by Ambari should only be modified via Ambari. Otherwise, when you restart a service, Ambari might overwrite your updates. |
Property Name | Subsystem | Type | Ambari Property |
---|---|---|---|
es.clustername | Indexing | String | es_cluster_name |
es.ip | Indexing | String | es_hosts |
es.port | Indexing | String | es_port |
es.date.format | Indexing | String | es_date_format |
fieldValidations | Parsing | Object | N/A |
parser.error.topic | Parsing | String | N/A |
stellar.function.paths | Stellar | CSV String | N/A |
stellar.function.resolver.includes | Stellar | CSV String | N/A |
stellar.function.resolver.excludes | Stellar | CSV String | N/A |
profiler.period.duration | Profiler | Integer | profiler_period_duration |
profiler.period.duration.units | Profiler | String | profiler_period_units |
profiler.writer.batchSize | Profiler | Integer | N/A |
profiler.writer.batchTimeout | Profiler | Interger | N/A |
update.hbase.table | REST/Indexing | String | update_hbase_table |
update.hbase.cf | REST-Indexing | String | update_hbase_cf |
geo.hdfs.file | Enrichment | String | geo_hdfs_file |
enrichment.writer.batchSize | Enrichment | Integer | N/A |
enrichment.writer.batchTimeout | Enrichment | String | N/A |
source.type.field | UI | String | source_type_field |
threat.triage.score.field | UI | String | threat_triage_score_field |
You can also create a validation using Stellar. The following validation uses Stellar to validate an ip_src_addr similar to the "validation":"IP"" example above:
"fieldValidations" : [ { "validation" : "STELLAR", "config" : { "condition" : "IS_IP(ip_src_addr, 'IPV4')" } } ]