-
Notifications
You must be signed in to change notification settings - Fork 19
Open
Description
We ran into an incident today due to Kubefleet hub agent failing to drain a large workqueue.
We bumped our worker count from 40 → 80 but we were still seeing extremely slow recovery ( O(hours) ).
It turns out Hub agent actually uses 1/10 of workers specified:
kubefleet/cmd/hubagent/workload/setup.go
Line 518 in 39ed26b
| ConcurrentPlacementWorker: int(math.Ceil(float64(opts.MaxConcurrentClusterPlacement) / 10)), |
This is a big no-no in controller development. Controllers should respect the user input and not mess with it as it breaks the expectation between the user and the system.
Metadata
Metadata
Assignees
Labels
No labels