kite-morphlines-maxmind

geoIP

The geoIP command (source code) returns Geolocation information for a given IP address, using an efficient in-memory Maxmind database lookup. The command stores a corresponding Jackson JsonNode Java object into the _attachment_body record field. The most recent version of the Maxmind GeoLite2 database can be downloaded as a flat data file from Maxmind.

Often, the geoIP command is combined with commands such as extractJsonPaths.

The command provides the following configuration options:


Property Name	Default	Description
inputField	n/a	The name of the input field that contains zero or more IP addresses.
database	GeoLite2-City.mmdb	The relative or absolute path of a Maxmind database file on the local file system. Example: /path/to/GeoLite2-City.mmdb

Example usage:

# extract geolocation info into a Jackson JsonNode Java object 
# and store it into the _attachment_body field:
geoIP {
  inputField : ip
  database : "target/test-classes/GeoLite2-City.mmdb"
}

# extract parts of the geolocation info from the Jackson JsonNode Java 
# object contained in the _attachment_body field and store the parts in
# the given record output fields:      
extractJsonPaths {
  flatten : false
  paths : { 
    /country/iso_code : /country/iso_code
    /country/names/en : /country/names/en
    /country/names/zh-CN : /country/names/zh-CN       
    "/subdivisions[]/names/en" : "/subdivisions[]/names/en"     
    "/subdivisions[]/iso_code" : "/subdivisions[]/iso_code"     
    /city/names/en : /city/names/en
    /postal/code : /postal/code
    /location/latitude : /location/latitude
    /location/longitude : /location/longitude
    /location/latitude_longitude : /location/latitude_longitude
    /location/longitude_latitude : /location/longitude_latitude
  } 
}

Example geoIP JSON output with extractJsonPaths:

Input: ip: 128.101.101.101

Expected output:

ip: 128.101.101.101
/country/iso_code: US
/country/names/en: United States
/country/names/zh-CN: 美国
/subdivisions[]/names/en: Minnesota
/subdivisions[]/iso_code: MN
/city/names/en: Minneapolis
/postal/code: 55455
/location/latitude: 44.9733
/location/longitude: -93.2323
/location/latitude_longitude: 44.9733,-93.2323
/location/longitude_latitude: -93.2323,44.9733

Example geoIP JSON output:

Input: ip: 128.101.101.101

Expected output:

{
   "city":{
      "geoname_id":5037649,
      "names":{
         "de":"Minneapolis",
         "en":"Minneapolis",
         "es":"Mineápolis",
         "fr":"Minneapolis",
         "ja":"ミネアポリス",
         "pt-BR":"Minneapolis",
         "ru":"Миннеаполис",
         "zh-CN":"明尼阿波利斯"
      }
   },
   "continent":{
      "code":"NA",
      "geoname_id":6255149,
      "names":{
         "de":"Nordamerika",
         "en":"North America",
         "es":"Norteamérica",
         "fr":"Amérique du Nord",
         "ja":"北アメリカ",
         "pt-BR":"América do Norte",
         "ru":"Северная Америка",
         "zh-CN":"北美洲"
      }
   },
   "country":{
      "geoname_id":6252001,
      "iso_code":"US",
      "names":{
         "de":"USA",
         "en":"United States",
         "es":"Estados Unidos",
         "fr":"États-Unis",
         "ja":"アメリカ合衆国",
         "pt-BR":"Estados Unidos",
         "ru":"США",
         "zh-CN":"美国"
      }
   },
   "location":{
      "latitude":44.9733,
      "longitude":-93.2323,
      "metro_code":"613",
      "time_zone":"America/Chicago"
      "latitude_longitude":"44.9733,-93.2323"
      "longitude_latitude":"-93.2323,44.9733"
   },
   "postal":{
      "code":"55455"
   },
   "registered_country":{
      "geoname_id":6252001,
      "iso_code":"US",
      "names":{
         "de":"USA",
         "en":"United States",
         "es":"Estados Unidos",
         "fr":"États-Unis",
         "ja":"アメリカ合衆国",
         "pt-BR":"Estados Unidos",
         "ru":"США",
         "zh-CN":"美国"
      }
   },
   "subdivisions":[
      {
         "geoname_id":5037779,
         "iso_code":"MN",
         "names":{
            "en":"Minnesota",
            "es":"Minnesota",
            "ja":"ミネソタ州",
            "ru":"Миннесота"
         }
      }
   ]
}