AWS Builder Center

Builder Center

Q-Bits: Managing service quotas with Amazon Q Developer

Q-Bits: Managing service quotas with Amazon Q Developer

Managing AWS service quotas efficiently is crucial for maintaining reliable cloud applications. Q Developer transforms this process by providing AI-powered assistance directly within your Integrated Development Environment (IDE) making it easier to monitor, manage and request quota increases

Daniel Wirjo

Welcome to another instalment of  Q-Bits , our regular series showcasing cool ways Amazon employees are leveraging Amazon Q Developer. Today, we're diving into how Q Developer can assist with managing service quotas.

Why leverage AI to manage service quotas?

As you build on AWS, it is worth noting that AWS services have  service quotas . Unless it is otherwise noted, service quotas are allocated per region per AWS account. Managing AWS service quotas efficiently is crucial for maintaining reliable cloud applications. Q Developer transforms this process by providing AI-powered assistance directly within your Integrated Development Environment (IDE) making it easier to monitor, manage and request quota increases. Rather than manually searching through documentation, developers can leverage natural language interactions to quickly find and understand quota information, as well as build tools to manage them.
Service quotas are increasingly important if your application leverages serverless and API-based services. For example:
  • Tokens per Minute (TPM) and Requests Per Minute (RPM) for Amazon Bedrock
  • vCPU quotas for Amazon EC2
  • Concurrency limits for Amazon Lambda
  • PutEvents Transactions per Second (TPS) for Amazon EventBridge

Example finding service quotas for Bedrock models using Q Developer chat

Here's how you might use  Q Developer chat  to find Bedrock quotas:
Prompt:
plaintext
What is the TPM and RPM service quota for the 
Anthropic Claude 3.5 Sonnet model in Amazon Bedrock in us-east-1? 

Provide a concise summary.
Example response:
HTML
Based on the resources, for Anthropic Claude 3.5 Sonnet V2 in us-west-2: 
• Tokens per minute (TPM): 2,000,000
• Requests per minute (RPM): 50
These limits apply to the combined usage across Converse, ConverseStream, InvokeModel and InvokeModelWithResponseStream operations. 
These quotas are not adjustable through the standard Service Quotas dashboard.
As the answer uses generative AI, you may still need to verify the information with the cited source. However, to improve accuracy, remember to make your prompt more specific. In this example, we have asked for a very specific quota (TPM and RPM), the region and also a concise summary.

Example managing service quotas for GPU and Compute using Q Developer chat

Q Developer can help you calculate and request the right GPU quotas for your workload. For example, prompt:
plaintext
I need to launch 3 g4dn.xlarge instances, 
what quota increase should I request?
Example response:
text
Request a quota increase for "Running On-Demand G and VT instances" to 
at least 12 vCPUs (4 vCPUs × 3 instances) in your target region. 
Request a quota increase for "Running On-Demand G and VT instances" to at least 12 vCPUs (4 vCPUs × 3 instances) in your target region.
As you can see, Q Developer can help you calculate the required quota. For applicable service quotas that are adjustable Q Developer can also help generate the CLI command to  request for an increase . Follow-up prompt:
text
Generate me a CLI command to increase the quota for region ap-southeast-2
Example response:
plaintext
aws service-quotas request-service-quota-increase \
--service-code ec2 \
--quota-code L-DB2E81BA \
--desired-value 12 \
--region ap-southeast-2
This requests an increase to 12 vCPUs for G and VT instances in Sydney region. The quota code L-DB2E81BA is for "Running On-Demand G and VT instances".

Example monitoring service quotas with Amazon CloudWatch using Q Developer code recommendations

You can also use Q Developer for  code block completion . To do this, simply write a comment describing the intended code block. To try this, I created a new TypeScript CDK application and a new  test.ts  file in my existing TypeScript CDK application.
Add the comment:
text
# Monitor Lambda service quota
# Trigger a Slack notification when utilization is high
Example code output:
TypeScript
import * as cdk from 'aws-cdk-lib';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as subscriptions from 'aws-cdk-lib/aws-sns-subscriptions';
import * as chatbot from 'aws-cdk-lib/aws-chatbot';
import { Construct } from 'constructs';

export class LambdaConcurrencyMonitorStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create SNS Topic for alerts
    const alertTopic = new sns.Topic(this, 'LambdaConcurrencyAlertTopic', {
      displayName: 'Lambda Concurrency Alert Topic'
    });

    // Create CloudWatch alarm
    const concurrencyAlarm = new cloudwatch.Alarm(this, 'LambdaConcurrencyAlarm', {
      metric: new cloudwatch.Metric({
        namespace: 'AWS/Lambda',
        metricName: 'ConcurrentExecutions',
        statistic: 'Maximum',
        period: cdk.Duration.minutes(5),
        dimensionsMap: {
          // No dimensions needed for account-wide metric
        }
      }),
      threshold: 800, // Alert when reaching 80% of default 1000 limit
      evaluationPeriods: 3,
      datapointsToAlarm: 2,
      comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
      alarmDescription: 'Alert when Lambda concurrent executions are high'
    });

    // Add alarm action to publish to SNS topic
    concurrencyAlarm.addAlarmAction(new cloudwatch.SnsAction(alertTopic));

    // Create Slack integration using AWS Chatbot
    const slackChannel = new chatbot.SlackChannelConfiguration(this, 'SlackChannel', {
      slackChannelConfigurationName: 'lambda-alerts',
      slackWorkspaceId: 'YOUR_SLACK_WORKSPACE_ID', // Replace with your Slack workspace ID
      slackChannelId: 'YOUR_SLACK_CHANNEL_ID', // Replace with your Slack channel ID
      notificationTopics: [alertTopic],
    });

    // Add IAM permissions for Chatbot
    slackChannel.addToRolePolicy(new cdk.aws_iam.PolicyStatement({
      effect: cdk.aws_iam.Effect.ALLOW,
      actions: [
        'cloudwatch:Describe*',
        'cloudwatch:Get*',
        'cloudwatch:List*'
      ],
      resources: ['*']
    }));
  }
}

You now have a draft CDK stack for a simple proactive alerting system. As the prompt is very general and not very specific, you may obtain slightly different generated outputs for the same prompt. From here, you can highlight specific parts of the code and use  Q Developer inline chat . For example, “make the stack easily extensible for multiple service quotas in addition to Lambda concurrency limits” or “make the slack message more engaging with emojis”.

Conclusion

Amazon Q Developer helps builders streamline the understanding and management of service quotas. To build reliable applications, remember to regularly review your quotas, implement monitoring, and leverage the power of AI to accelerate and automate. Having a proactive approach helps prevent quota-related issues before they impact your applications.
If you’re looking for ways on how to leverage AI and Amazon Q Developer to make your teams more productive, check out Q-Bits, a series by how Amazon employees.  
# q-bits# service-quotas# amazon-q-developer
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article