Parallel resource creation fails because of ServiceAssociationLinkNotReady

[dsteindo]

Terraform Version

1.1.3

Terraform Configuration Files

resource "azurerm_container_group" "jmeter_workers" {
  count               = var.JMETER_WORKERS_COUNT
  name                = "${var.PREFIX}-worker${count.index}"
  location            = azurerm_resource_group.jmeter_rg.location
  resource_group_name = azurerm_resource_group.jmeter_rg.name

  ip_address_type = "private"
  os_type         = "Linux"

  network_profile_id = azurerm_network_profile.jmeter_net_profile.id

  image_registry_credential {
    server   = data.azurerm_container_registry.jmeter_acr.login_server
    username = data.azurerm_container_registry.jmeter_acr.admin_username
    password = data.azurerm_container_registry.jmeter_acr.admin_password
  }

  container {
    name   = "jmeter"
    image  = var.JMETER_DOCKER_IMAGE
    cpu    = var.JMETER_WORKER_CPU
    memory = var.JMETER_WORKER_MEMORY

    ports {
      port     = var.JMETER_DOCKER_PORT
      protocol = "TCP"
    }

    volume {
      name                 = "jmeter"
      mount_path           = "/jmeter"
      read_only            = true
      storage_account_name = azurerm_storage_account.jmeter_storage.name
      storage_account_key  = azurerm_storage_account.jmeter_storage.primary_access_key
      share_name           = azurerm_storage_share.jmeter_share.name
    }

    commands = [
      "/bin/sh",
      "-c",
      "cp -r /jmeter/* .; /entrypoint.sh -s -J server.rmi.ssl.disable=true -Djava.rmi.server.hostname=$(ifconfig eth0 | grep 'inet addr:' | awk '{gsub(\"addr:\", \"\"); print $2}') -Jnode_index=${count.index} -Jusers_per_node=${var.USERS_PER_NODE}",
    ]
  }
}

Debug Output

azurerm_network_profile.jmeter_net_profile: Creating...
azurerm_network_profile.jmeter_net_profile: Creation complete after 1s [id=/subscriptions/HASH/resourceGroups/jmeter-performance-RG/providers/Microsoft.Network/networkProfiles/jmeter3netprofile]
azurerm_container_group.jmeter_workers[0]: Creating...
azurerm_container_group.jmeter_workers[2]: Creating...
azurerm_container_group.jmeter_workers[3]: Creating...
azurerm_container_group.jmeter_workers[1]: Creating...
azurerm_container_group.jmeter_workers[3]: Still creating... [10s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [10s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [20s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [20s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [30s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [30s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [40s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [40s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [50s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [50s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [1m0s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [1m0s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [1m10s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [1m10s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [1m20s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [1m20s elapsed]
azurerm_container_group.jmeter_workers[3]: Still creating... [1m30s elapsed]
azurerm_container_group.jmeter_workers[0]: Still creating... [1m30s elapsed]
azurerm_container_group.jmeter_workers[3]: Creation complete after 1m34s [id=/subscriptions/HASH/resourceGroups/jmeter-performance-RG/providers/Microsoft.ContainerInstance/containerGroups/jmeter3-worker3]
azurerm_container_group.jmeter_workers[0]: Creation complete after 1m34s [id=/subscriptions/HASH/resourceGroups/jmeter-performance-RG/providers/Microsoft.ContainerInstance/containerGroups/jmeter3-worker0]


|
| Error: creating/updating Container Group: (Name "jmeter3-worker1" / Resource Group "jmeter-performance-RG"): containerinstance.ContainerGroupsClient#CreateOrUpdate: Failure sending request: StatusCode=409 -- Original Error: Code="ServiceAssociationLinkNotReady" Message="Network is not ready for container group 'jmeter3-worker1'. Please try again."
| 
|   with azurerm_container_group.jmeter_workers[1],
|   on main.tf line 75, in resource "azurerm_container_group" "jmeter_workers":
|   75: resource "azurerm_container_group" "jmeter_workers" {
| 
|
|
| Error: creating/updating Container Group: (Name "jmeter3-worker2" / Resource Group "jmeter-performance-RG"): containerinstance.ContainerGroupsClient#CreateOrUpdate: Failure sending request: StatusCode=409 -- Original Error: Code="ServiceAssociationLinkNotReady" Message="Network is not ready for container group 'jmeter3-worker2'. Please try again."
| 
|   with azurerm_container_group.jmeter_workers[2],
|   on main.tf line 75, in resource "azurerm_container_group" "jmeter_workers":
|   75: resource "azurerm_container_group" "jmeter_workers" {
| 
|
##[error]Bash exited with code '1'.

Expected Behavior

4 workers for load testing should have been created

Actual Behavior

Sometimes the pipeline breaks because of the "ServiceAssociationLinkNotReady" error

Steps to Reproduce

Use the following project https://github.com/Azure-Samples/jmeter-aci-terraform
Configure that multiple workers are used, see Azure-Samples/jmeter-aci-terraform#81
Also consider that there is still an open PR to raise the terraform version used which we already integrated into our project, see Azure-Samples/jmeter-aci-terraform#84

Additional Context

Variables

TF_VAR_JMETER_DOCKER_IMAGE = justb4/jmeter:5.1.1
TF_VAR_JMETER_WORKERS_COUNT = 4
TF_VAR_PREFIX = jmeter3
TF_VAR_RESOURCE_GROUP_NAME = jmeter-performance-RG
TF_VAR_USERS_PER_NODE = 250

https://github.com/Azure-Samples/jmeter-aci-terraform/blob/main/terraform/variables.tf

References

Azure-Samples/jmeter-aci-terraform#81

[jbardin]

Hello,

This appears to be an issue or question with the AzureRM provider, not with Terraform itself. You can see existing issues and file a new one in their repository here: https://github.com/terraform-providers/terraform-provider-azurerm/issues. If you have questions about Terraform or the AzureRM provider, it's better to use the community forum where there are more people ready to help. The GitHub issues here are monitored only by a few core maintainers.

Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.