Configuring Extraction for Amazon S3

1. Log in to Cloudera Manager Admin Console and stop Cloudera Navigator:
   • Select Clusters > Cloudera Management Service
   • Click the Instances tab.
   • Click the checkbox next to Navigator Audit Server and Navigator Metadata Server in the Role Type list to select these roles.
   • From the Actions for Selected (2) menu button, select Stop
2. Log in to the AWS Management Console with AWS account (IAM user) and open the Simple Queue Service setup page (select Services > Messaging > Simple Queue Service. Click Create New Queue or Get Started Now if region has no configured queues.)
3. For each region that has Amazon S3 buckets, create a queue as follows:
   1. Click the Create New Queue button. Enter a Queue Name, click the Standard Queue (not FIFO), and then click Configure Queue. Configure the queue using the following settings:

      Default Visibility Timeout	10 minutes
      Message Retention Period	14 days
      Delivery Delay	0 seconds
      Receive Message Wait Time	0 seconds

   2. Select the queue you created, click the Permissions tab, click Add a Permission, and configure the following in the Add a Permision to... dialog box:

      Effect	Allow
      Principal	Everybody
      Actions	SendMessage

   3. Click the Add Conditions (optional) link open the condition fields and enter the following values:

      Qualifier	None
      Condition	ArnLike
      Key	aws:SourceArn
      Value	arn:aws:s3::*:*

   4. Click Add Condition to save the settings.
   5. Click Add Permission to save all settings for the queue.

1. Navigate to the Amazon S3 bucket for the region (Services > Storage > S3).
2. Select the bucket.
3. Click the Properties tab.
4. Click the Events settings box.
5. Click Add notification.
6. Configure event notification for the bucket as follows:

   Name	nav-send-metadata-on-change
   Events	ObjectCreated(All) ObjectRemoved(All)
   Send to	SQS Queue
   SQS queue	Enter the name of your queue

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1481678612000",
            "Effect": "Allow",
            "Action": [
                "sqs:DeleteMessage",
                "sqs:DeleteMessageBatch",
                "sqs:GetQueueAttributes",
                "sqs:ReceiveMessage"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Stmt1481678744000",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:ListAllMyBuckets",
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetBucketNotification",
                "s3:PutBucketNotification"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

• Log in to the AWS Management Console using the IAM user account associated with the target Amazon S3 bucket.
• Copy the appropriate JSON text from the table below Extraction Policies JSON Referenceand paste into the AWS Management Console policy editor for the Navigator user account on (the IAM user) through the AWS Management Console.