Cloudera Docs

AWS limits

When you create your AWS account, AWS sets limits to the resources available to you. The limits can vary by region.

For example:
  • Depending on your AWS account limits, you might only be allowed to provision a certain number of CPU instances, or you might not have default access to GPU instances at all.
  • If you are planning to use Endpoint Access Gateway, CDP will create two AWS network load balancers (AWS NLB) per cluster (that is for each Data Lake and Data Hub). Make sure that your AWS NLB limits allow for the load balancer creation.
  • If you are planning to use the Machine Learning service, review Limitations on AWS in ML docs.

Make sure to review your AWS service limits before your proceed.

To view the limits set by Amazon for your account, log in to AWS and go to EC2 > Limits. The EC2 Service Limits page lists the limits for the resources available to you in your EC2 instance, including limits to the number of instances and hosts. The Networking Limits section on the page lists the VPC, subnet, and security group limits for your AWS account. For example, there is typically a limit of 5 VPCs per region.

To review what resources CDP creates on your AWS account during environment registration and later when you deploy specific CDP services, refer to AWS resources used by CDP.

If you require more resources than the limit set by Amazon, you can request Amazon to raise the limit of a resource. On the EC2 Service Limits page, click Request limit increase for the resource that you want to increase and create an AWS support case for a Service Limit Increase.

For more information about AWS limits, refer to AWS Service Limits in AWS documentation.