To configure an SQS queue suitable for Atlas extraction
Use the following information to configure an SQS queue for Atlas extraction.
Prerequisite: You must add Amazon Resource Names (ARN) of the IAM role for the user / group configured under CDP. For more information, see Onboarding CDP users and groups for cloud storage.
- Log in to the AWS Management Console
(https://aws.amazon.com/console/) with the AWS account (IAM user).
Make sure that there are adequate privileges to administer the queue and the buckets involved in metadata extraction.
- Open the Simple Queue Service setup page and select Services > Application Integration > Simple Queue Service.
- Click Create New Queue or Get Started Now, if the region has no configured queues.
- For each region that has Amazon S3 buckets, create a queue as follows:
- Click Create Queue. Enter a Queue Name,
name it, for example:
aws-s3-cdp-atlas-events
, click the Standard Queue (not FIFO), and next under configuration you must configure the queue using the following settings:Default Visibility Timeout =10 minutes
Message Retention Period =14 days
Delivery Delay = 0 seconds
Receive Message Wait Time = 0 seconds
Maximum message size = 256 KB
- Under Access Policy, choose method
Advanced and insert the access policy similar to
below:
{ "Version": "2008-10-17", "Id": "__default_policy_ID", "Statement": [ { “Xxxx”: "__owner_statement", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::00000000000:root" }, "Action": "SQS:ReceiveMessage", "Resource": "arn:aws:sqs:us-west-1:00000000000:aws-s3-cdp-atlas-events" }, { "xxxx": "__sender_statement", "Effect": "Allow", "Principal": { "Service": "s3.amazonaws.com" }, "Action": "SQS:SendMessage", "Resource": "arn:aws:sqs:us-west-1:00000000000:aws-s3-cdp-atlas-events" } ] }
- Configure at rest Encryption to meet your organization standards. In-transit
encryption is configured automatically.
- Click Create Queue.
- Click Create Queue. Enter a Queue Name,
name it, for example:
- Repeat this process for each region that has Amazon S3 buckets.
For more information about creating an SQS queue, see Creating an Amazon SQS queue.