Cloudera Docs

Configuring Access to Google Cloud Storage

Access to Google Cloud Storage can be configured separately for each cluster by providing the service account email address.

Access from a cluster to a Google Cloud Storage is possible through a service account. Configuring access to Google Cloud Storage involves the following steps.

Table 1. Overview of Configuring Access to Google Cloud Storage
Step Considerations
Creating a service account on Google Cloud Platform and generating a key associated with it.
  • You may need to contact your Google Cloud Platform admin in order to complete these steps.
  • If you already have a service account, you do not need to perform these steps as long as you are able to provide the service account key. If you have a service account but do not know the service account key, you should be able to generate a new key.
Creating a role on Google Cloud Platform with sufficient permissions to access storage buckets.
  • You may need to contact your Google Cloud Platform admin in order to complete these steps.
  • This is a one time operation, and the same role can be used across different service accounts and storage buckets.
Modifying permissions of the Google Cloud Storage bucket so that you can access it by using your service account key.
  • You may need to contact your Google Cloud Platform admin in order to complete these steps.
  • \You should perform these steps for each bucket that you want to access.
  • You do not need to perform these steps if your service account has project-wide access to all buckets on the account.
Configuring credentials via Cloudera Manager.
  • These configuration steps are appropriate for a single-user cluster.
  • Only one configuration per cluster is recommended; that is, you should use one service account per cluster. If required, it is possible to use multiple service accounts with the same cluster; in that case, each job-specific configuration should be changed to use the desired service account.