Loading...
Loading...
### Terraform Version ```shell Terraform v1.9.8 on darwin_arm64 + provider registry.terraform.io/hashicorp/aws v6.7.0 Tested across multiple versions: - AWS Provider v5.0.0 through v6.7.0 (all affected) - Terraform v1.1.9 through v1.9.8 (all affected) ``` ### Terraform Configuration Files ```terraform terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } provider "aws" { region = "us-east-1" } locals { subnet_ids = [ "subnet-12345", "subnet-67890", ] } resource "aws_security_group" "test" { name = "test-resolver-endpoint" vpc_id = "vpc-12345" egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } tags = { Name = "test-resolver-endpoint" } } # This demonstrates the hash collision bug # Expected: 6 IP addresses (3 per subnet) # Actual: 2 IP addresses (1 per subnet due to hash collisions) resource "aws_route53_resolver_endpoint" "test" { direction = "OUTBOUND" security_group_ids = [aws_security_group.test.id] # Multiple auto-assigned IPs per subnet - this is the problematic case ip_address { subnet_id = local.subnet_ids[0] # First IP in subnet A } ip_address { subnet_id = local.subnet_ids[0] # Second IP in subnet A - gets deduplicated! } ip_address { subnet_id = local.subnet_ids[0] # Third IP in subnet A - gets deduplicated! } ip_address { subnet_id = local.subnet_ids[1] # First IP in subnet B } ip_address { subnet_id = local.subnet_ids[1] # Second IP in subnet B - gets deduplicated! } ip_address { subnet_id = local.subnet_ids[1] # Third IP in subnet B - gets deduplicated! } tags = { Name = "test-multiple-ips-per-subnet" } } output "ip_address_count" { value = length(aws_route53_resolver_endpoint.test.ip_address) description = "Should be 6 but will show 2 due to hash collision bug" } output "ip_addresses" { value = aws_route53_resolver_endpoint.test.ip_address } ``` ### Debug Output ``` 2025-08-05T11:53:38.760-0700 [DEBUG] ReferenceTransformer: "aws_route53_resolver_endpoint.test" references: [aws_security_group.test local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand)] 2025-08-05T11:53:38.760-0700 [DEBUG] ReferenceTransformer: "local.subnet_ids (expand)" references: [] ``` ### Expected Behavior AWS Route53 Resolver endpoints should support up to 6 IP addresses per endpoint (quotas can be increased), including multiple IP addresses per subnet with auto-assignment. When configuring multiple `ip_address` blocks for the same subnet without explicit IP addresses, AWS should auto-assign unique IP addresses within each subnet, and Terraform should track all configured IP addresses correctly. According to AWS documentation, this is a supported configuration. ```diff # aws_route53_resolver_endpoint.test will be created + resource "aws_route53_resolver_endpoint" "test" { + arn = (known after apply) + direction = "OUTBOUND" + host_vpc_id = (known after apply) + id = (known after apply) + protocols = (known after apply) + region = "us-east-1" + resolver_endpoint_type = (known after apply) + security_group_ids = (known after apply) + tags = { + "Name" = "test-multiple-ips-per-subnet" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-12345" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-12345" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-12345" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-67890" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-67890" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-67890" } } ``` ### Actual Behavior ## Affected Resource(s) * `aws_route53_resolver_endpoint` When multiple `ip_address` blocks are configured for the same subnet with auto-assigned IPs (empty `ip` field), the Terraform provider's hash function creates identical hashes, causing Terraform's Set data structure to deduplicate them. **Root Cause**: The `endpointHashIPAddress` function in `/internal/service/route53resolver/endpoint.go` generates hashes using the format `"{subnet_id}-{ip}-"`. When `ip` is empty (auto-assigned), multiple entries for the same subnet create identical hashes like `"subnet-12345-"`, causing Set deduplication. **Impact**: - Only 1 IP address is created per subnet instead of the requested multiple IPs - Forces users to specify explicit IP addresses to achieve uniqueness - Contradicts AWS best practices of letting AWS manage IP allocation - Prevents users from leveraging AWS Route53 Resolver's full high-availability capabilities No error is thrown, but terraform plan shows fewer IP addresses than configured: ```diff # aws_route53_resolver_endpoint.test will be created + resource "aws_route53_resolver_endpoint" "test" { + arn = (known after apply) + direction = "OUTBOUND" + host_vpc_id = (known after apply) + id = (known after apply) + protocols = (known after apply) + region = "us-east-1" + resolver_endpoint_type = (known after apply) + security_group_ids = (known after apply) + tags = { + "Name" = "test-multiple-ips-per-subnet" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-12345" } + ip_address { + ip = (known after apply) + ip_id = (known after apply) + ipv6 = (known after apply) + subnet_id = "subnet-67890" } } ``` ### Steps to Reproduce 1. Apply the provided configuration with multiple `ip_address` blocks for the same subnet without explicit `ip` values 2. Run `terraform plan` and observe that only 2 IP addresses are planned instead of 6 3. Run `terraform apply` and verify that only 1 IP address per subnet is created 4. Check the AWS Console to confirm only 2 IP addresses exist on the resolver endpoint 5. Check Terraform state: `terraform state show aws_route53_resolver_endpoint.test` will show only 2 IP addresses **Alternative reproduction with existing resources**: 1. Create a resolver endpoint through AWS Console with multiple IPs per subnet (this works fine) 2. Import it into Terraform using `terraform import aws_route53_resolver_endpoint.test rslvr-out-12345` 3. Run `terraform plan` - it will show a diff trying to remove the "extra" IP addresses due to the hash collision ### Additional Context ## Affected Resource(s) or Data Source(s) * `aws_route53_resolver_endpoint` ### References **AWS Service Capability vs Provider Limitation:** - AWS Route53 Resolver supports up to 6 IP addresses per endpoint ([quotas can be increased](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/DNSLimitations.html#limits-api-entities-resolver)) - AWS Route53 Resolver supports multiple IP addresses per subnet for higher query limits - AWS Route53 Resolver supports auto-assignment of IP addresses within subnets - AWS documentation encourages letting AWS manage IP allocation for operational simplicity **Business Impact:** - Prevents users from implementing multiple IP addresses per subnet for DNS resolver configurations for higher query limits - Forces manual IP management instead of leveraging AWS auto-assignment capabilities - Creates a gap between AWS service capabilities and Terraform provider support - May lead to configuration drift when users work around the limitation using AWS Console or CLI **Technical Analysis:** - Issue is specifically in the `endpointHashIPAddress` function in `/internal/service/route53resolver/endpoint.go` - Current hash format: `fmt.Fprintf(&buf, "%s-%s-", m[names.AttrSubnetID].(string), m["ip"].(string))` - When `ip` is empty (auto-assigned), identical subnets produce identical hashes: `"subnet-12345-"` - This affects both resource creation and state management (imports, refreshes, etc.) **Similar Patterns in Other Resources:** - `aws_instance` successfully handles multiple network interfaces per subnet using unique identifiers - `aws_lb_target_group_attachment` handles multiple targets per target group without hash collisions - Other AWS resources with Set-based configurations have solved similar problems **Environment Details:** - Affects all AWS regions and partitions - No workarounds exist other than manual IP specification - Issue present since the resource was first implemented ### Generative AI / LLM assisted development? n/a
Click on a version to see all relevant bugs
Terraform Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.