CloudWatch PutMetricData AccessDenied: Fixing Monitoring Permission Gaps

Your monitoring just went silent. The custom metrics dashboard is empty, the alarms are not firing, and somewhere in your application logs you find this:

An error occurred (AccessDenied) when calling the PutMetricData operation:
User: arn:aws:sts::123456789012:assumed-role/my-app-role/i-0abc123def456
is not authorized to perform: cloudwatch:PutMetricData on resource: *

Or from a Lambda function:

[ERROR] ClientError: An error occurred (AccessDenied) when calling the
PutLogEvents operation: User: arn:aws:sts::123456789012:assumed-role/
my-lambda-role/my-function is not authorized to perform:
logs:PutLogEvents on resource: arn:aws:logs:us-east-1:123456789012:
log-group:/aws/lambda/my-function:log-stream:2026/05/24/[$LATEST]abc123

When CloudWatch rejects your metrics or logs, your observability stack goes blind. You cannot detect outages, you cannot investigate performance issues, and your alarms stop working. I have seen this cause production incidents to last hours longer than necessary because the team had no metrics to diagnose the original problem.

Here is how to systematically fix every CloudWatch permission gap.

Step 1: Identify What Is Being Denied

The error message tells you three things: the calling principal, the denied action, and the target resource. Parse these carefully because each points to a different fix.

First, confirm the calling identity:

aws sts get-caller-identity

{
    "UserId": "AROAEXAMPLEID:i-0abc123def456",
    "Account": "123456789012",
    "Arn": "arn:aws:sts::123456789012:assumed-role/my-app-role/i-0abc123def456"
}

Then check what policies are attached to this role:

aws iam list-attached-role-policies --role-name my-app-role
aws iam list-role-policies --role-name my-app-role

Inspect each policy to find what CloudWatch permissions are granted:

# For managed policies
aws iam get-policy-version \
  --policy-arn arn:aws:iam::123456789012:policy/my-monitoring-policy \
  --version-id v1 \
  --query 'PolicyVersion.Document'

# For inline policies
aws iam get-role-policy \
  --role-name my-app-role \
  --policy-name MonitoringPermissions

Root Cause 1: Missing cloudwatch:PutMetricData Permission

The most straightforward cause. The IAM role does not have cloudwatch:PutMetricData in any of its policies. This happens when a role was created for a specific purpose and monitoring was added later without updating permissions.

The minimum policy for publishing custom metrics:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData"
      ],
      "Resource": "*"
    }
  ]
}

Note that cloudwatch:PutMetricData does not support resource-level permissions — the Resource must be "*". However, you can restrict which namespaces a role can publish to using the cloudwatch:namespace condition key:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "cloudwatch:namespace": "MyApp/Production"
        }
      }
    }
  ]
}

This is actually a best practice — it prevents a role from accidentally publishing metrics to the wrong namespace or overwriting metrics from another application.

Root Cause 2: Namespace Restriction Mismatch

If your policy uses the cloudwatch:namespace condition, the namespace in your PutMetricData call must match exactly. This is case-sensitive.

Test with the CLI:

aws cloudwatch put-metric-data \
  --namespace "MyApp/Production" \
  --metric-name "RequestCount" \
  --value 1 \
  --unit Count

If the policy allows MyApp/Production but your application sends metrics to myapp/production, it will be denied. Check your application code for the exact namespace string.

A common variant of this problem occurs when teams use environment-based namespaces like MyApp/Staging and MyApp/Production but the IAM policy only allows one of them. Use a wildcard condition for flexibility:

{
  "Condition": {
    "StringLike": {
      "cloudwatch:namespace": "MyApp/*"
    }
  }
}

Root Cause 3: CloudWatch Logs Permissions Missing

CloudWatch Logs uses a completely separate set of IAM actions from CloudWatch Metrics. If your application writes both metrics and logs, you need both sets of permissions.

The full set of CloudWatch Logs permissions for an application:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams"
      ],
      "Resource": [
        "arn:aws:logs:us-east-1:123456789012:log-group:/myapp/*",
        "arn:aws:logs:us-east-1:123456789012:log-group:/myapp/*:log-stream:*"
      ]
    }
  ]
}

Unlike PutMetricData, CloudWatch Logs actions do support resource-level permissions. You can restrict access to specific log groups by ARN pattern. The resource must include both the log group and log stream ARNs.

A frequent mistake is including the log group ARN but forgetting the log stream ARN:

"Resource": "arn:aws:logs:us-east-1:123456789012:log-group:/myapp/*"

This allows CreateLogGroup and DescribeLogGroups but denies PutLogEvents because that action targets a log stream, not a log group. You need the *:log-stream:* suffix.

Root Cause 4: Lambda Execution Role Missing CloudWatch Permissions

Lambda functions automatically get CloudWatch Logs permissions through the AWSLambdaBasicExecutionRole managed policy. But if your function publishes custom metrics, you need additional permissions.

Check what your Lambda role has:

aws lambda get-function-configuration \
  --function-name my-function \
  --query 'Role'

aws iam list-attached-role-policies \
  --role-name my-lambda-role

If you see AWSLambdaBasicExecutionRole but not CloudWatchFullAccess or a custom policy, the function can write logs but not metrics.

The AWSLambdaBasicExecutionRole only includes:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

To add custom metric permissions, attach an additional policy:

aws iam put-role-policy \
  --role-name my-lambda-role \
  --policy-name CloudWatchMetrics \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "cloudwatch:PutMetricData",
          "cloudwatch:GetMetricData",
          "cloudwatch:ListMetrics"
        ],
        "Resource": "*"
      }
    ]
  }'

Root Cause 5: EC2 Instance Profile Missing Monitoring Permissions

EC2 instances use instance profiles to access AWS services. If the CloudWatch Agent is installed but the instance profile lacks the right permissions, metrics and logs will fail silently or with AccessDenied errors in the agent log.

Check the instance profile:

# Get the instance profile from the instance
aws ec2 describe-instances \
  --instance-ids i-0abc123def456 \
  --query 'Reservations[0].Instances[0].IamInstanceProfile.Arn'

# List the roles in the instance profile
aws iam get-instance-profile \
  --instance-profile-name my-instance-profile \
  --query 'InstanceProfile.Roles[*].RoleName'

The CloudWatch Agent requires this minimum policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams",
        "ec2:DescribeTags",
        "ec2:DescribeVolumes",
        "ec2:DescribeInstances",
        "ssm:GetParameter"
      ],
      "Resource": "*"
    }
  ]
}

AWS provides the managed policy CloudWatchAgentServerPolicy for this purpose:

aws iam attach-role-policy \
  --role-name my-ec2-role \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

Check the CloudWatch Agent log for specific errors:

# On Amazon Linux / RHEL
cat /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log | grep -i "error\|denied\|unauthorized"

Root Cause 6: VPC Endpoints Missing for CloudWatch

If your application runs in a private subnet without internet access, it needs VPC endpoints to reach CloudWatch and CloudWatch Logs. Without these endpoints, API calls time out rather than returning AccessDenied — but the symptom looks the same: missing metrics.

Check existing VPC endpoints:

aws ec2 describe-vpc-endpoints \
  --filters "Name=vpc-id,Values=vpc-abc123" \
  --query 'VpcEndpoints[*].{
    ServiceName: ServiceName,
    State: State,
    VpcEndpointType: VpcEndpointType
  }'

You need endpoints for both services:

# CloudWatch Metrics endpoint
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-abc123 \
  --service-name com.amazonaws.us-east-1.monitoring \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-abc123 subnet-def456 \
  --security-group-ids sg-abc123

# CloudWatch Logs endpoint
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-abc123 \
  --service-name com.amazonaws.us-east-1.logs \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-abc123 subnet-def456 \
  --security-group-ids sg-abc123

The security group on the VPC endpoint must allow inbound HTTPS (port 443) from your application's security group.

Root Cause 7: Cross-Account Observability Not Configured

If you are trying to send metrics from one account to a central monitoring account, you need CloudWatch cross-account observability. Without it, PutMetricData calls to another account's namespace will fail.

Set up the monitoring account as a sink:

# In the monitoring account
aws oam create-sink \
  --name central-monitoring \
  --tags Key=Environment,Value=Production

Then link source accounts:

# In each source account
aws oam create-link \
  --label-template '$AccountName' \
  --resource-types "AWS::CloudWatch::Metric" "AWS::Logs::LogGroup" \
  --sink-identifier arn:aws:oam:us-east-1:999999999999:sink/sink-id

Root Cause 8: Resource-Based Policy on Log Groups

CloudWatch Logs supports resource-based policies on log groups. If a log group has a restrictive policy, even a role with the correct IAM permissions may be denied.

Check the resource policy:

aws logs describe-resource-policies

# Check a specific log group's subscription filters and policies
aws logs describe-log-groups \
  --log-group-name-prefix "/myapp/" \
  --query 'logGroups[*].{
    logGroupName: logGroupName,
    retentionInDays: retentionInDays,
    kmsKeyId: kmsKeyId
  }'

If the log group is encrypted with a KMS key, the writing role also needs kms:GenerateDataKey and kms:Decrypt permissions on that key.

Prevention and Best Practices

Use the AWS managed policies as a starting point. CloudWatchAgentServerPolicy for EC2, AWSLambdaBasicExecutionRole for Lambda, then add custom metric permissions as needed.

Test metric publishing from the CLI first. Before deploying application changes, verify from the same role:

aws cloudwatch put-metric-data \
  --namespace "MyApp/Production" \
  --metric-name "TestMetric" \
  --value 1 \
  --unit Count

If this succeeds, the IAM permissions are correct and any application failures are in the code.

Set up CloudWatch Agent configuration centrally using AWS Systems Manager Parameter Store:

aws ssm put-parameter \
  --name "/cloudwatch-agent/config" \
  --type String \
  --value file://cloudwatch-agent-config.json

Monitor your monitoring. Create a CloudWatch alarm on the CloudWatch Agent's own heartbeat metric. If the agent stops publishing, you get alerted before your dashboards go dark.

Use namespace conventions consistently. Standardize on a format like Company/Application/Environment so IAM condition policies are predictable and maintainable.

Audit permissions quarterly. IAM Access Analyzer can identify roles that have CloudWatch permissions they no longer need, and roles that are missing permissions they require.

When to Call for Help

CloudWatch permission issues that persist after checking all of the above often involve complex interactions between SCPs, permission boundaries, VPC networking, and cross-account configurations. When your monitoring is down, your ability to detect and diagnose other issues is compromised — it is worth fixing quickly. We help teams build reliable observability stacks on AWS. If your monitoring has gaps or you need help designing a cross-account CloudWatch architecture, reach out for a free consultation. We will audit your monitoring permissions and ensure nothing is falling through the cracks.