-
Notifications
You must be signed in to change notification settings - Fork 19
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
A clear and concise description of what the bug is.
Status update by rollout controller on clusterresourcebinding is getting context cancelled. This prevents the clusterresourcebinding from getting updated with latest snapshot.
This happens when we are propagating to more than 7-8 clusters especially default rollingUpdate values.
"timestamp": 2025-10-01T19:36:09.1622327Z,
"log": E1001 19:36:09.162205 1 rollout/controller.go:699] "Failed to update binding status" err="Put \"https://10.244.0.1:443/apis/placement.kubernetes-fleet.io/v1beta1/clusterresourcebindings/followfeed-filters-indexin-prod-ltx1-k8s-ippp-test-282e5327/status\": context canceled" clusterResourceBinding="followfeed-filters-indexin-prod-ltx1-k8s-ippp-test-282e5327" condition={"type":"RolloutStarted","status":"False","observedGeneration":2,"lastTransitionTime":null,"reason":"RolloutNotStartedYet","message":"The resources cannot be updated to the latest because of the rollout strategy"}
This doesn't seem to be API server issue because we don't wee it happening with other service calling PUT.
Environment
Please provide the following:
- Hub cluster details: v0.9.5
- Member cluster details: v0.9.5
To Reproduce
Steps to reproduce the behavior:
- Create CRP with 7-8 spec.policy.clusterNames
- Have a default value of rollingUpdate
- Change resource to create new snapshot
Expected behavior
A clear and concise description of what you expected to happen.
- I expect even with default rollingUpdate strategy and rolling to multiple clusters, context should not get cancelled.
- One status update failure should not impact status update on another resourceBinding. All goroutines in the errgroup use the same derived context (cctx). When any single updateBindingStatus call fails and returns an error, the
cctxcontext is immediately cancelled. this cancellation affects all other goroutines that are still running, causing them to receive a cancelled context. Code reference
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working