Cloudera Docs

MAP data type

This article describes the specifics of the MAP complex data type.

Syntax for MAP

column_name MAP < primitive_type, type >
type ::= primitive_type | complex_type

MAP data types represent sets of key-value pairs. These collections have an arbitrary number of elements, with each element being the same type. A MAP type is similar a miniature table, the table equivalent of scalar (primitive) values with two columns.

KEY

  • A scalar type, such as BIGINT, STRING, or TIMESTAMP

  • It enables you to define non-continuous sequences or categories with arbitrary names.

  • Keys in a map column may represent a numeric sequence of events during a manufacturing process or TIMESTAMP values that correspond to sensor observations.

  • Access as map_name.key.

VALUE

  • The second part of a key-value pair in a map.

  • May be a scalar, or another complex type (an ARRAY, a STRUCT, or another MAP).

  • Access as map_name.value.

  • If a map contains a STRUCT, access as map_name.value.field_name, or as map_name.field_name.

MAPs in the Dataset Fields interface

In the Dataset Fields interface, an example of a basic MAP data type may look something like the following image. Each level of a complex data type can be expanded to show component details, or collapsed.

In the example of the dataset Complex Type - Map, you can see the following:

  • Dimension Customer (a String, symbolized by A)
  • Measures Orderid (an Integer, symbolized by #)
  • Overalldiscount (a Real, symbolized by 1.2)

These are primitive types. However, Dimension Orderinfo is a Map data type, symbolized by [M].

When you click Edit Fields, you can see that:

  • Primitive types can be cast as alternate data types (such as Integer into Real).
  • Complex data type Array cannot be changed to another type.
  • Primitive components of the array may be cast as other primitive data types.
  • Cloudera Data Visualization uses complex datatypes only as Dimensions. They or their components cannot be redefined as Measurements of the dataset.

MAPs in visuals

When building a visual with complex data, you cannot use the complex type directly. However, you can add the primitive components of the complex type to the shelves of the visual.

Changing field properties

Changing field properties for a component of a complex data type is straightforward. You can change the aggregation function, type, display options, and so on, just like a standalone primitive type field.

MAPs in Expression Editor

The expression editor supports the full use of MAPs, both in the Dataset and Visual interfaces.