Skip to content

ELBv2 controller panic #2549

@jwilms1971

Description

@jwilms1971

I am trying to modify an existing network load balancer (created via CloudFormation) by changing the allocated subnets. Here is the YAML I submitted to the ELBv2 controller:

apiVersion: elbv2.services.k8s.aws/v1alpha1
kind: LoadBalancer
metadata:
  name: eks-v2-blueprints-cluster-nlb
  annotations:
    services.k8s.aws/adoption-policy: adopt-or-create
    services.k8s.aws/adoption-fields: |
      {
        "name": "k8s-argocd-blueprin-61bea9c603"
      }
spec:
  name: k8s-argocd-blueprin-61bea9c603
  subnets:
  - subnet-06f51576daf32f89b
  - subnet-0f92d1276ed15bf64
  - subnet-065970a909617b02a 

The controller goes into a panic:

{"level":"error","ts":"2025-07-03T06:03:01.295Z","msg":"Reconciler error","controller":"loadbalancer","controllerGroup":"elbv2.services.k8s.aws","controllerKind":"LoadBalancer","LoadBalancer":{"name":"eks-v2-blueprints-cluster-nlb","namespace":"ack-system"},"namespace":"ack-system","name":"eks-v2-blueprints-cluster-nlb","reconcileID":"6fa856d9-e31b-4c74-b45e-3c6856972a0a","error":"panic: runtime error: invalid memory address or nil pointer dereference [recovered]","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:255"}

To ensure I wasn't making any configuration mistakes I also submitted the resource in using adopt without create which yielded the following result:

$ k describe loadbalancers
Name:         eks-v2-blueprints-cluster-nlb
Namespace:    ack-system
Labels:       <none>
Annotations:  services.k8s.aws/adopted: true
              services.k8s.aws/adoption-fields:
                {
                  "name": "k8s-argocd-blueprin-61bea9c603"
                }
              services.k8s.aws/adoption-policy: adopt
API Version:  elbv2.services.k8s.aws/v1alpha1
Kind:         LoadBalancer
Metadata:
  Creation Timestamp:  2025-07-03T06:11:03Z
  Finalizers:
    finalizers.elbv2.services.k8s.aws/LoadBalancer
  Generation:        2
  Resource Version:  105113
  UID:               c8384973-f4f7-43fe-9311-e1576be9727d
Spec:
  Attributes:
    Key:            dns_record.client_routing_policy
    Value:          any_availability_zone
    Key:            access_logs.s3.prefix
    Value:
    Key:            deletion_protection.enabled
    Value:          false
    Key:            zonal_shift.config.enabled
    Value:          false
    Key:            load_balancing.cross_zone.enabled
    Value:          false
    Key:            access_logs.s3.enabled
    Value:          false
    Key:            access_logs.s3.bucket
    Value:
  Ip Address Type:  ipv4
  Name:             k8s-argocd-blueprin-61bea9c603
  Scheme:           internet-facing
  Security Groups:
    sg-0618a68283f1b8e18
    sg-0cb86f30aa77383d2
  Subnets:
    subnet-06f51576daf32f89b
    subnet-0f92d1276ed15bf64
    subnet-065970a909617b02a
  Tags:
    Key:    service.k8s.aws/resource
    Value:  LoadBalancer
    Key:    service.k8s.aws/stack
    Value:  argocd/blueprints-addon-argocd-server
    Key:    elbv2.k8s.aws/cluster
    Value:  hsbc-prod-eks-v2-blueprints-cluster
  Type:     network
Status:
  Ack Resource Metadata:
    Arn:               arn:aws:elasticloadbalancing:eu-west-2:000000000000:loadbalancer/net/k8s-argocd-blueprin-61bea9c603/5b5ece2886e7a2da  
    Owner Account ID:  000000000000
    Region:            eu-west-2
  Availability Zones:
    Subnet ID:               subnet-033ba98d248b2fcbd
    Zone Name:               eu-west-2b
    Subnet ID:               subnet-0b477b8fee674a8ac
    Zone Name:               eu-west-2c
    Subnet ID:               subnet-0e1a51c61ebb9d110
    Zone Name:               eu-west-2a
  Canonical Hosted Zone ID:  ZD4D7Y8KGAS4G
  Conditions:
    Last Transition Time:  2025-07-03T06:11:03Z
    Message:               Resource synced successfully
    Reason:
    Status:                True
    Type:                  ACK.ResourceSynced
  Created Time:            2025-07-02T23:03:35Z
  Dns Name:                k8s-argocd-blueprin-61bea9c603-5b5ece2886e7a2da.elb.eu-west-2.amazonaws.com
  State:
    Code:  active
  Vpc ID:  vpc-0cc98e76c53211f70
Events:    <none>

Metadata

Metadata

Assignees

Labels

area/adoption-annotationIssues or PRs related to ACK Adoption by Annotation featureservice/elbv2Indicates issues or PRs that are related to elbv2-controller.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions