[bug]: Not getting all metrics for multiple resources with tags

[shubhadaR]

We are trying to fetch metrics with tags (using aws_tag_select) for multiple resources.
However we are getting metrics only for 2 or 3 resources but not for all. Also we are getting aws_resource_info only for few resources at a time but not for all resources.
here is our config file:

region: ap-south-1
#delay_seconds: 10
#period_seconds: 600
metrics:

• aws_namespace: AWS/SQS
  use_get_metric_data: false
  aws_metric_name: ApproximateNumberOfMessagesNotVisible
  aws_dimensions: [QueueName]
  aws_tag_select:
  tag_selections:
  Monitoring: ["enabled"]
  resource_type_selection: "sqs:queue"
  resource_id_dimension: QueueName
  aws_statistics: [Average]

• aws_namespace: AWS/ELB
  use_get_metric_data: false
  aws_metric_name: RequestCount
  aws_dimensions: [AvailabilityZone, LoadBalancerName]
  aws_tag_select:
  tag_selections:
  Monitoring: ["enabled"]
  resource_type_selection: "elasticloadbalancing:loadbalancer"
  resource_id_dimension: LoadBalancerName
  aws_statistics: [Sum]

• aws_namespace: AWS/RDS
  use_get_metric_data: false
  aws_metric_name: CPUUtilization
  aws_dimensions: [DBInstanceIdentifier]
  aws_tag_select:
  tag_selections:
  Monitoring: ["enabled"]
  resource_type_selection: "rds:db"
  resource_id_dimension: DBInstanceIdentifier
  aws_statistics: [Sum]

• aws_dimensions:

  • InstanceId
    aws_metric_name: CPUUtilization
    aws_namespace: AWS/EC2
    aws_statistics:
  • Average
    aws_tag_select:
    tag_selections:
    Monitoring: ["enabled"]
    resource_type_selection: "ec2:instance"
    resource_id_dimension: InstanceId

• aws_namespace: AWS/ApplicationELB
  use_get_metric_data: false
  aws_metric_name: RequestCount
  aws_dimensions: [AvailabilityZone, LoadBalancer]
  aws_tag_select:
  tag_selections:
  Monitoring: ["enabled"]
  resource_type_selection: "elasticloadbalancing:loadbalancer"
  resource_id_dimension: LoadBalancer

[matthiasr]

That's concerning 😬 and difficult to debug without access to your environment. What happens when you look for these resources using aws resourcegroupstaggingapi get-resources?

The core of fetching the resources is here. Could you add more debug logging there to figure out where it's going wrong?

[matthiasr]

I believe this is another instance of #268. You may be interested in #571 and #648.

[matthiasr]

Or is this a problem with metrics being available only after a long delay?