A brief introduction to Cloudera security features.
As a system designed to support vast amounts and types of data,
Cloudera clusters must meet ever-evolving security requirements imposed by regulating
agencies, governments, industries, and the general public. Cloudera clusters comprise both
Hadoop core and ecosystem components, all of which must be protected from a variety of threats
to ensure the confidentiality, integrity, and availability of all the cluster's services and
data. This overview provides introductions to:
Security Requirements
Goals for data management systems, such as confidentiality, integrity, and availability,
require that the system be secured across several dimensions. These can be characterized in
terms of both general operational goals and technical concepts, as shown in the figure
below:
Perimeter Access to the cluster must be protected from a variety of threats
coming from internal and external networks and from a variety of actors. Network isolation
can be provided by proper configuration of firewalls, routers, subnets, and the proper use
of public and private IP addresses, for example. Authentication mechanisms ensure that
people, processes, and applications properly identify themselves to the cluster and prove
they are who they say they are, before gaining access to the cluster.
Data Data in the cluster must always be protected from unauthorized exposure.
Similarly, communications between the nodes in the cluster must be protected. Encryption
mechanisms ensure that even if network packets are intercepted or hard-disk drives are
physically removed from the system by bad actors, the contents are not usable.
Access Access to any specific service or item of data within the cluster must be
specifically granted. Authorization mechanisms ensure that once users have authenticated
themselves to the cluster, they can only see the data and use the processes to which they
have been granted specific permission.
Visibility Visibility means that the history of data changes is transparent and
capable of meeting data governance policies. Auditing mechanisms ensure that all actions
on data and its lineage—source, changes over time, and so on—are documented as they
occur.
Securing the cluster to meet specific organizational goals involves using security features
inherent to the Hadoop ecosystem as well as using external security infrastructure. The
various security mechanisms can be applied in a range of levels.
Security Levels
The figure below shows the range of security levels that can be implemented for a Cloudera
cluster, from non-secure (0) to compliance ready (3). As the sensitivity and volume of data
on the cluster increases, so should the security level you choose for the cluster. The table below describes the levels in more detail:
Level
Security
Characteristics
0
Non-secure
No security configured. Non-secure clusters should never be used in production
environments because they are vulnerable to any and all attacks and exploits.
Recommended only for proof of concept, demo, and disposable platforms.
1
Basic security
Configured for authentication, authorization, and auditing. Authentication is
first configured to ensure that users and services can access the cluster only after
proving their identities. Next, authorization mechanisms are applied to assign
privileges to users and user groups. Auditing procedures keep track of who accesses
the cluster (and how). Recommended for development platforms.
2
Data Security and Governance
Sensitive data is encrypted. Key management systems handle encryption keys.
Auditing has been setup for data in meta stores. System metadata is reviewed and
updated regularly. Ideally, cluster has been setup so that lineage for any data
object can be traced (data governance). Recommended for staging and
production.
3
Compliance Ready
The secure CDP cluster is one in which all data, both data-at-rest and
data-in-transit, is encrypted and the key management system is fault-tolerant.
Auditing mechanisms comply with industry, government, and regulatory standards (PCI,
HIPAA, NIST, for example), and extend from CDP to the other systems that integrate
with it. Cluster administrators are well-trained, security procedures have been
certified by an expert, and the cluster can pass technical review. Recommended for
all production platforms.
Hadoop Security Architecture
The figure below is an example of some of the many components at work in a production
Cloudera enterprise cluster. The figure highlights the need to secure clusters that may
ingest data from both internal and external data feeds, and across possibly multiple
datacenters. Securing the cluster requires applying authentication and access controls
throughout these many inter- and intra-connections, as well as to all users who want to
query, run jobs, or even view the data held in the cluster.
External data streams are authenticated by mechanisms in place for Flume and Kafka.
Data from legacy databases is ingested using Sqoop. Data scientists and BI analysts can
use interfaces such as Hue to work with data on Impala or Hive, for example, to create
and submit jobs. Kerberos authentication can be leveraged to protect all these
interactions.
Encryption can be applied to data at-rest using transparent HDFS encryption with an
enterprise-grade Key Trustee
Server.
Authorization policies can be enforced using Ranger (for services such as Hive,
Impala, and Search) as well as HDFS Access Control Lists.
Auditing capabilities can be provided by using Apache
Ranger.
