awslimitchecker

IMPORTANT - Breaking Changes

• This release removes the EC2 / Max spot instance requests per region limit, which has been removed by AWS, in favor of six new vCPU-based limits: All F Spot Instance Requests, All G Spot Instance Requests, All Inf Spot Instance Requests, All P Spot Instance Requests, All X Spot Instance Requests, and All Standard (A, C, D, H, I, M, R, T, Z) Spot Instance Requests.
• This release adds two new services: CertificateManager (ACM) and CloudFront.
• This release requires additional IAM permissions: acm:ListCertificates, cloudfront:ListCloudFrontOriginAccessIdentities, cloudfront:ListKeyGroups, cloudfront:ListDistributions, cloudfront:ListCachePolicies, and cloudfront:ListOriginRequestPolicies.

IMPORTANT - Seeking New Maintainer

As I commented in Issue #500, I'm looking for someone to share (and perhaps take over) maintenance of this project. awslimitchecker is, and has always been, a personal-time-only project for me; the only time I've done work on it during my day job is when my employer was experiencing an issue or requested a specific feature. Because of a variety of issues, including changing personal interests and my employer relying on this project much less (following an AWS account restructuring that largely avoids service limits), I've been spending much less time on this project than it deserves. As a result, I'm looking for someone to help with maintenance... at the very least, helping review PRs and get them to a merge-able state. If you're interested, please comment on Issue #500 or contact me directly. While I am incredibly flattered by the offers I've received for sponsorship, paid support, or other financial incentive, I'd ask that anyone who's willing to make that commitment instead dedicate a few hours to working on issues or PRs. I, for my part, will make a concerted effort to quickly merge and release any PRs that meet all of the development.pull_request_guidelines{.interpreted-text role="ref"}.

All Changes

• PR #532 - Add Quotas Service support for ECS Fargate quotas. Thanks to robpickerill for this contribution.
• PR #533 / Fixes Issue #527 - Fix Quotas Service quota names for EIPs. Thanks to robpickerill for this contribution.
• PR #534 / Fixes Issue #521 - Update Quotas Service quota names for EBS. Thanks to robpickerill for this contribution.
• PR #535 / Fixes Issue #518 - Fix EC2 Security Group counts to only include groups owned by the current account. Thanks to robpickerill for this contribution.
• PR #536 / Fixes Issue #512 - Fix CloudWatch metrics queries to get data from one minute ago, to fix bug where GetMetricData is not yet populated. Thanks to robpickerill for this contribution.
• PR #543 / Fixes Issue #538 - Fix issue with calculation of usage for EC2 Rules Per Network ACL. Thanks to jwu2 for this contribution.
• PR #537 - Use boto3 adaptive retry mode. Thanks to robpickerill for this contribution.
• PR #547 / Fixes Issue #502 - Replace EC2 / Max spot instance requests per region limit, which has been removed by AWS, with new vCPU-based spot instance requests limits. This also switches to using CloudWatch metric data to retrieve current usage. Thanks to TagadaPoe for this contribution.
• PR #546 / Fixes Issue #540 - Add support for ACM (Certificate Manager) limits. Thanks to TagadaPoe for this contribution.
• PR #545 / Fixes Issue #539 - Add support for CloudFront limits. Thanks to TagadaPoe for this contribution.
• Issue #551 - Allow custom host for Datadog metric provider.

IMPORTANT - Breaking Changes

• This release removes the EBS General Purpose (SSD) volume storage (GiB) limit in favor of General Purpose (SSD gp2) volume storage (GiB) and General Purpose (SSD gp3) volume storage (GiB) limits, to account for the new gp3 volume type and corresponding AWS service limits.
• This release removes the EBS Provisioned IOPS and Provisioned IOPS (SSD) storage (GiB) limits in favor of Provisioned IOPS (io1) and Provisioned IOPS (io2), and Provisioned IOPS SSD (io1) storage (GiB) and Provisioned IOPS SSD (io2) storage (GiB), respectively, to account for the new io2 EBS volume type and corresponding AWS service limtits.

IMPORTANT - Seeking New Maintainer

As I commented in Issue #500, I'm looking for someone to share (and perhaps take over) maintenance of this project. awslimitchecker is, and has always been, a personal-time-only project for me; the only time I've done work on it during my day job is when my employer was experiencing an issue or requested a specific feature. Because of a variety of issues, including changing personal interests and my employer relying on this project much less (following an AWS account restructuring that largely avoids service limits), I've been spending much less time on this project than it deserves. As a result, I'm looking for someone to help with maintenance... at the very least, helping review PRs and get them to a merge-able state. If you're interested, please comment on Issue #500 or contact me directly. While I am incredibly flattered by the offers I've received for sponsorship, paid support, or other financial incentive, I'd ask that anyone who's willing to make that commitment instead dedicate a few hours to working on issues or PRs. I, for my part, will make a concerted effort to quickly merge and release any PRs that meet all of the development.pull_request_guidelines{.interpreted-text role="ref"}.

All Changes

• PR #519 - Add support for gp3 EBS volume type. Thanks to spockNinja for this contribution!
• Issue #514 / PR #517 - Fix EKS Fargate profile usage check exception (KeyError: 'labels'). Thanks to jwu2 for first reporting this issue and sebasrp for the fix!
• Issue #520 / PR #522 - Add awslimitchecker to conda-forge so it can be installed via Conda. Thanks to BastianZim for this.
• PR #525 - Fix Prometheus example in docs, thanks to felixkrohn.
• PR #526 - Fix possible KeyError: 'LaunchSpecifications' in EC2 service. Thanks to nitrocode for this.
• PR #528 - Update ECS default limits. Thanks to arturpriz for this contribution.
• PR #523 - Add support for io2 type EBS volumes. Thanks to andyjp for this contribution.
• Add tox and TravisCI support for testing against Python 3.9; bump the docker, docs, and integration3 tox environments from 3.8 to 3.9.
• Bump the base Docker image from Python 3.8 to 3.9.

IMPORTANT - Breaking Changes

• This release makes significant changes to how Trusted Advisor is used; see below.
• This release requires the following new IAM permissions: eks:ListClusters, eks:DescribeCluster, eks:ListNodegroups, eks:ListFargateProfiles, eks:DescribeFargateProfile, kinesis:DescribeLimits.
• This release introduces a number of new limits, as well as new services. Please see below for details.
• This release removes the EC2/Security groups per VPC limit, which no longer exists, and adds the new EC2/VPC security groups per Region limit.

All Changes

• Issue #466 - Significant changes to Trusted Advisor support.
  • In June 2019, AWS announced the new Service Quotas service (great name) that allows us to retrieve limit/quota information from a unified API. In addition, many individual services now provide limit information via their own APIs. At this point (late 2020) all of the limit/quota information that was previously available via Trusted Advisor is now available via a combination of the individual service APIs and Service Quotas.
  • In February 2020, the layout of Trusted Advisor checks was changed, and the "Performance / Service Limits" check that we previously used to obtain limit information was moved to its own category in Trusted Advisor. While I can't confirm this, as far as I can tell, this change was only made in the standard AWS regions/partitions (i.e. not GovCloud or China).
  • awslimitchecker still has not been updated for this new Trusted Advisor layout.
  • This release disables Trusted Advisor by default outside China and GovCloud, as it provides no additional information outside of these regions/partitions.
  • If you are running in China or GovCloud and have issues with awslimitchecker retrieving information from Trusted Advisor, please open an issue.
  • My current intent is to leave Trusted Advisor support in this state until Service Quotas is available in China and GovCloud, at which point I plan on completely removing all Trusted Advisor support.
• Migrate CI builds from travis-ci.org to travis-ci.com.
• Issue #503 - Fix Units set to "None" error when retrieving load balancer data from Service Quotas. We now allow the (A|E)LB per Region quota with a unit of either "Count" (prior to November 2020) or "None" (November 2020 on).
• Issue #489 / PR #490 - Add missing RDS limits: Manual Cluster Snapshots, Custom Endpoints Per DB Cluster, DB Instance Roles, and DB Cluster Roles. Thanks to sebasrp for this contribution!
• Issue #472 / PR #494 - Add support for the EKS service, and 8 new limits for it. Thanks to sebasrp for this contribution!
• Issue #495 / PR #496 - Add support for the Kinesis service, and one new limit for it. Thanks to sebasrp for this contribution!
• PR #499 - Set quota_name for VPC "Entries per route table" limit, so that the current limit will be automatically retrieved from Service Quotas. Thanks to patuck for this contribution!
• Issue #498 - Fix multiple issues relating to VPC limits:
  • Update the EC2 / Rules per VPC security group limit to support retrieving the current limit value from Service Quotas.
  • Remove the EC2/Security groups per VPC limit, which no longer exists.
  • Add the new EC2/VPC security groups per Region limit.
• Issue #501 - Update VPC/Network interfaces per Region limit for new calculation method.
• Issue #488 / PR #491 - Update new ElastiCache default limits. Thanks to sebasrp for this contribution!

Important: This release requires new IAM permissions: sts:GetCallerIdentity and cloudwatch:GetMetricData

Important: This release includes updates for major changes to ECS limits, which includes the renaming of some existing limits.

• Issue #477 - EC2 instances running on Dedicated Hosts (tenancy "host") or single-tenant hardware (tenancy "dedicated") do not count towards On-Demand Instances limits. They were previously being counted towards these limits; they are now excluded from the count. Thanks to pritam2277 for reporting this issue and providing details and test data.
• Issue #477 - For all VPC resources that support the owner-id filter, supply that filter when describing them, set to the current account ID. This will prevent shared resources from other accounts from being counted against the limits. Thanks to pritam2277 for reporting this issue and providing details and test data.
• Issue #475 - When an Alert Provider is used, only exit non-zero if an exception is encountered. Exit zero even if there are warnings and/or criticals. Thanks to varuzam for this feature request.
• Issue #467 - Fix the Service Quotas quota name for VPC "NAT Gateways per AZ" limit. Thanks to xRokco for reporting this issue, as well as the required fix.
• Issue #457 - In the required IAM permissions, replace support:* with the specific permissions that we need.
• Issue #463 - Updates for the major changes to ECS limits in August 2020. Thanks to vincentclee for reporting this issue.
  • The EC2 Tasks per Service (desired count) limit has been replaced with Tasks per service, which measures the desired count of tasks of all launch types (EC2 or Fargate). The default value of this limit has increased from 1000 to 2000.
  • The default of Clusters has increased from 2,000 to 10,000.
  • The default of Services per Cluster has increased from 1,000 to 2,000.
  • The Fargate Tasks limit has been removed.
  • The Fargate On-Demand resource count limit has been added, with a default quota value of 500. This limit measures the number of ECS tasks and EKS pods running concurrently on Fargate. The current usage for this metric is obtained from CloudWatch.
  • The Fargate Spot resource count limit has been added, with a default quota value of 500. This limit measures the number of ECS tasks running concurrently on Fargate Spot. The current usage for this metric is obtained from CloudWatch.
• Add internal helper method to :py~._AwsService{.interpreted-text role="class"} to get Service Quotas usage information from CloudWatch.

• PR #468 - Fix transposed headings in CLI Usage documentation. Thanks to @owenmann.
• PR #470 - Fix new EBS "Active snapshots" limit (bumped from 10,000 to 100,000) and Quotas Service name. Thanks to @rashidamiri.
• Issue #464 - Fix bug where SES was causing ConnectTimeoutError in some regions. This has been added to the list of SES exceptions that we catch and silently ignore. This is a new exception thrown by regions that do not have SES support.
• Add .dockerignore file to make local builds quite a bit smaller.
• Issue #465 - Fixed via versionfinder 1.1.1.
• Internal testing changes:
  • Stop testing under Python 2.7 and Python 3.4.
  • Switch from deprecated pep8 / pytest-pep8 to pycodestyle / pytest-pycodestyle.
  • Pin pytest to <6.0.0 to avoid some breaking changes for now.
  • Switch integration test environment from Python 3.7 to Python 3.8.

• PR #458 - Fix for ZeroDivisionError on some Service Quotas limits that report as having a limit of zero. Thanks to @deimosfr.

• Fixes issue #453 - remove version constraint on dateutil dependency.
• Fixes issue #454 - remove version constraint on botocore dependency.
• Update tox docs, localdocs, and docker environments to use Python 3.8.
• Fixes issue #451 - Fix default Rules Per VPC Security Group limit.

Important: This release includes major changes to the EC2 On-Demand Instances service limits! For most users, this means the 175 Instance-type-specific limits will be removed and replaced with five (5) limits. Please see the changelog.8_0_0_vcpu_limits section below for further details, as this will especially impact anyone using limit or threshold overrides, or post-processing awslimitchecker's output. This is also a time to remind all users that this project adheres to a strict development.versioning_policy and if occasional breakage due to limit or IAM policy changes is unacceptable, you should pin to a major version.

Important: Python versions prior to 3.5, including 2.7, are now pending deprecation. As of January 1, 2020, they will no longer be tested or supported, and awslimitchecker will require Python 3.5 or newer. Please see below for details. Also take note that running via the official Docker image is a way to ensure the best version of Python is always used.

Important: This release requires a new IAM permission, servicequotas:ListServiceQuotas.

• Issue #400 / PR #434 - Support GovCloud region and alternate partitions in STS assumed roles and Trusted Advisor. Thanks to @djkiourtsis.
• Issue #432 - Update EC2 limit handling for new vCPU-based limits in regions other than cn-* and us-gov-* (which still use old per-instance-type limits). See section below <changelog.8_0_0_vcpu_limits> for further information. For regions other than cn-* and us-gov-*, this will remove all 175 Running On-Demand <type> instances and the Running On-Demand EC2 instances limit, and replace them with:
  • Running On-Demand All F instances
  • Running On-Demand All G instances
  • Running On-Demand All P instances
  • Running On-Demand All X instances
  • Running On-Demand All Standard (A, C, D, H, I, M, R, T, Z) instances
• Issue #429 - add 87 missing EC2 instance types. This will now only impact cn-* and us-gov-* regions.
• Issue #433 - Fix broken links in the docs; waffle.io and landscape.io are both gone, sadly.
• Issue #441 - Fix critical bug where awslimitchecker would die with an unhandled botocore.exceptions.ParamValidationError exception in accounts that have Trusted Advisor but do not have a "Service Limits" check in the "performance" category.
• Issue #439 - Fix unhandled exception in CloudTrail service when attempting to call GetEventSelectors on an Organization trail. When calling DescribeTrails, we will now pass includeShadowTrails as False, to not include replications of trails in different regions or organization trails in member accounts (relevant API documentation).
• Issue #438 - Per PEP 373, Python 2.7 will officially end support on January 1, 2020. As such, and in keeping with reasoning explained at python3statement.org, awslimitchecker will stop supporting and testing against Python 2.7 on January 1, 2020. At that point, all new versions will be free to use Python features introduced in 3.5. As of this version, a PendingDeprecationWarning will be emitted when running awslimitchecker under Python 2.7.
• Issue #437 - Per PEP 429, Python 3.4 reached end-of-life on March 18, 2019 and is now officially retired. Add a PendingDeprecationWarning for users running under this version, announcing that support for Python 3.4 will be removed on January 1, 2020.
• In following with the above two issues, raise a DeprecationWarning when running on any Python2 version prior to 2.7 or any Python3 version prior to 3.4, in accorance with the published end-of-life dates of those versions.
• Issue #436 - Begin testing under Python 3.8 and base our Docker image on python:3.8-alpine.
• Issue #435 - Allow configuring the botocore maximum retries for Throttling / RateExceeded errors on a per-AWS-API basis via environment variables. See the relevant sections of the CLI Usage or Python Usage documentation for further details.
• Issue #431 - Fix a major under-calculation of usage for the EC2 Rules per VPC security group limit. We were previously calculating the number of "Rules" (from port / to port / protocol combinations) in a Security Group, but the limit is actually based on the number of permissions granted. See this comment on the issue for further details.
• Issue #413 - Add support for retrieving limits from the new Service Quotas service where available. See the changelog.8_0_0_service_quotas section below for more information.
• Bump boto3 minimum version requirement from 1.4.6 to 1.9.175 and botocore minimum version requirement from 1.6.0 to 1.12.175, in order to support Service Quotas.

New EC2 vCPU Limits

AWS has announced new, completely different handling of EC2 On-Demand Instances service limits. Instead of having a limit per instance type (currently 261 limits), there will now be only five limits, based on the number of vCPUs for instance families: one each for "F", "G", "P", and "X" family instances (defaulting to a total of 128 vCPUs each) and one limit for all other "Standard" instance families (currently A, C, D, H, I, M, R, T, and Z) defaulting to a combined total of 1152 vCPUs. Please see the link, and the EC2 On-Demand Instance Limits section of the AWS FAQ for further information.

This greatly simplifies handling of the EC2 On-Demand limits, but does mean that any existing code that references EC2 Running On-Demand limit names, including any limit and/or threshold overrides, will need to be updated for this change.

This change is only going into effect in the "standard" AWS regions/partitions, i.e. not in the China partition (cn- regions) or GovCloud (us-gov- regions). It is a phased rollout from October 24 to November 7, 2019 based on the first character of your account ID (see the "How will the transition to vCPU limits happen?" entry in the FAQ linked above for exact dates). Unfortunately, there is no clear way to determine via API if a given account is using the new vCPU limits or the old per-instance-type limits. As a result, and given that this release is being made already part-way through the rollout window, the current behavior of awslimitchecker is as follows:

• When running against region names beginning with cn- or us-gov-, use the old per-instance-type limits, unless the USE_VCPU_LIMITS environment variable is set to true.
• Otherwise use the new vCPU-based limits, unless the USE_VCPU_LIMITS environment variable is set to something other than true.

As such, if you install this release before November 7, 2019 and need to force your non-China, non-GovCloud accout to use the older per-instance-type limits, setting the USE_VCPU_LIMITS environment variable to false will accomplish this until your account switches over to the new vCPU limits. Alternatively, you can leave awslimitchecker as-is and accept possibly-slightly-inaccurate limit calculations for a few days.

Please also note that with the change to vCPU limits, there is no longer an overall Running On-Demand EC2 instances limit for accounts that use the new vCPU limits.

I have not yet implemented Trusted Advisor (TA) support for these new limits, as they're presented in a different category of Trusted Advisor checks from the previous EC2 limits. I'm not going to be implementing TA for these limits, in favor of spending the time instead on implementing Service Quotas support via Issue #413.

Calculation of current usage for the vCPU limits is based on the EC2 Optimizing CPU Options documentation which specifies, "The number of vCPUs for the instance is the number of CPU cores multiplied by the threads per core." The CpuOptions field of the EC2 DescribeInstances API specifies the core and thread count for each running instance.

Service Quotas

AWS' new Service Quotas service provides a unified interface to retrieve current limits from many AWS services. These limit values are second only to the services' own APIs (for the services that provide limit information via API), and are much more current and complete than the information provided by Trusted Advisor. The introduction of Service Quotas should greatly reduce the number of limits that need to be retrieved from Trusted Advisor or specified manually.

If you currently have any Limit Overrides set (via either the CLI or Python API), please verify on the limits page whether Service Quotas data is now available for those limits. You should be able to remove manual overrides for the limits that now retrieve data from Service Quotas.

• Issue #301 - Distribute an official Docker image for awslimitchecker.
• Issue #421
  • Stop referencing deprecated botocore.vendored.requests.exceptions.ConnectTimeout in favor of new, and higher-level, botocore.exceptions.ConnectionError
  • In awslimitchecker.utils._get_latest_version, replace use of botocore.vendored.requests with urllib3.
• Issue #324 - Support loading limit overrides and/or threshold overrides from a JSON file either stored locally or in S3 via new --limit-override-json and --threshold-override-json CLI options.
• Issue #418 - Add support for sending runtime, limits, and usage to metric providers such as Datadog.
• Issue #419 - Add support for alerts/notifications of thresholds crossed or failed runs (exceptions) via alert providers such as PagerDuty.

This release removes one limit and adds two new limits!

• Issue #412 / PR #414 - Since some time in June 2019, the former ELB Service Active load balancers limit is now two separate limits, Classic load balancers and Application load balancers. Anyone who was using the "Active load balancers" limit name (e.g. in overrides or custom code) must update their code accordingly. This release removes the Active load balancers limit and adds two new limits, Classic load balancers and Application load balancers, to match how AWS now calculates and exposes these limits.
• Issue #410 - Documentation fix for missing Trusted Advisor information on Limits page.
• Fix some test failures related to exception objects in pytest 5.0.0.

• Issue #406 - Fix for unhandled exception when a Trusted Advisor check has a null timestamp.

• PR #402 - Add --skip-check command line option for ignoring specific checks based on service and check name. Thanks to @ddelnano.

• Issue #397 - Fix unhandled exception checking SES in some regions. Issue #375 in 6.0.1 handled an uncaught ClientError when checking SES in some regions, but some regions such as ap-southeast-2 are now returning a 503 Service Unavailable for SES instead. Handle this case as well. Thanks to @TimGebert for reporting the issue and bergkampsliew for verifying.

• PR #394 - Fix bug in calculation of VPC "Network interfaces per Region" limit, added in 6.1.0 (PR #379), that resulted in reporting the limit 5x lower than it actually is in some cases. Thanks to @TimGebert.

• PR #391 / Issue #390 - Update for some recently-increased DynamoDB and EFS default limits. Thanks to bergkampsliew.

• PR #387 - Fix bug in calculation of VPC "Network interfaces per Region" limit, added in 6.1.0 (PR #379). Thanks to @nadlerjessie.

• PR #381 / Issue #382 - Revised fix for Issue #375, uncaught ClientError exception when checking SES Send Quota in certain regions. Thanks to bergkampsliew.

• PR #379 - Add support for EC2/VPC Network interfaces per Region limit. Thanks to @nadlerjessie.

• Issue #375 - Fix uncaught ClientError exception when checking SES Send Quota in certain regions. Thanks to bergkampsliew for PR #376.

This release requires new IAM permissions:

• lambda:GetAccountSettings

Important: This release removes the ApiGateway APIs per account limit in favor of more-specific limits; see below.

• Issue #363 - Add support for the Lambda limits and usages.
• Clarify support for "unlimited" limits (limits where :pyawslimitchecker.limit.AwsLimit.get_limit returns None).
• Add support for 26 new EC2 instance types.
• Update default limits for ECS service.
• ApiGateway service now has three ReST API limits (Regional API keys per account, Private API keys per account, and Edge API keys per account) in place of the previous single APIs per account to reflect the current documented service limits.
• API Gateway service - add support for VPC Links per account limit.
• Add support for Network Load Balancer limits Network load balancers and Listeners per network load balancer.
• Add support for Application Load Balancer limits Certificates per application load balancer.
• Add support for Classic ELB (ELBv1) Registered instances per load balancer limit.
• Rename dev/terraform.py to dev/update_integration_iam_policy.py and move from using terraform to manage integration test IAM policy to pure Python.
• Note that I've left out the Targets per application load balancer and Targets per network load balancer limits. Checking usage for these requires iterating over DescribeTargetHealth for each target group, so I've opted to leave it out at this time for performance reasons and because I'd guess that the number of people with 500 or 1000 targets per LB is rather small. Please open an issue if you'd like to see usage calculation for these limits.

Important Note on Limit Values

awslimitchecker has had documented support for Limits that are unlimited/"infinite" since 0.5.0 by returning None from :pyawslimitchecker.limit.AwsLimit.get_limit. Until now, that edge case was only triggered when Trusted Advisor returned "Unlimited" for a limit. It will now also be returned for the Lambda service's Function Count Limit. Please be aware of this if you're using the Python API and assuming Limit values are all numeric.

If you are relying on the output format of the command line awslimitchecker script, please use the Python API instead.