✨ Allow RFC6902 patches and strategic merge patches for provider manifests

[hjoshi123]

What this PR does / why we need it:

Allow JSON merge patches and RFC6902 for provider manifests, this makes it possible to modify provider manifests consistently across releases, for example, if the user needs to change labels/annotations on objects, example usage:

patches:
    - patch: |-
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: not-used
        spec:
          template:
            metadata:
              annotations:
                cluster-autoscaler.kubernetes.io/safe-to-evict: "true"        
      target:
        kind: Deployment
        name: hello-world
        namespace: sample

This still uses the same patch library. This PR doesn't remove the existing field but introduces a new one and in the future versions of the operator we can deprecate the existing manifestPatches field in favor of a more generic patches field.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #792

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign furkatgofurov7 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Welcome @hjoshi123!

It looks like this is your first PR to kubernetes-sigs/cluster-api-operator 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api-operator has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @hjoshi123. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-sigs-cluster-api-operator ready!

Name	Link
🔨 Latest commit	d09d80c
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-cluster-api-operator/deploys/6933cdc567ffe300078fc6d6
😎 Deploy Preview	https://deploy-preview-937--kubernetes-sigs-cluster-api-operator.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[nalum]

Looking good, just a few comments.

One question I have on the merge patch is around the requirement for the apiVersion key. I know you don't likely have an answer for the reasoning behind that but is that still the case for the new patching setup?

[nalum]

This looks like it will fail the test based on the existing test.

I would update this to use it's own set of patch data as you want to test all patch types that this supports.

[hjoshi123]

Yes I am writing more tests. The one I added was just a basic test to see if merge patch works.. and I ran the tests in local and it does patch and tests pass.. I will add more for RFC6902 patches

[hjoshi123]

I did add a test regarding jsonpatch and it passed.. let me know if the code looks good.

[nalum]

@hjoshi123 thanks for this, it's looking better. I would suggest a change though, this does not match the existing test structure.

Could you update to follow that?

As an example:

testCases := []struct {
	name               string
	objectsToPatchYaml string
	patches            []*operatorv1.Patch
  expectedOutput     string
}{
	{
		name:               "strategic merge test",
		objectsToPatchYaml: testObjectsToPatchYaml,
		expectedOutput: expectedTestPatchedObjectsYaml,
		patches: []*operatorv1.Patch{
			{
				Patch: addServiceAccoungPatchRBAC,
				Target: &operatorv1.PatchSelector{
					Group: "rbac.authorization.k8s.io",
					Kind:  "ClusterRoleBinding",
				},
			},
      // ... leaving out for brevity
    },
  },
  {
		name:               "rfc6902 patch test add",
		objectsToPatchYaml: testObjectsToPatchYaml,
    expectedOutput: expectedRFC6902Output,
		patches: []*operatorv1.Patch{
			{
				Patch: rfc6902PatchAdd,
				Target: &operatorv1.PatchSelector{
					Group: "rbac.authorization.k8s.io",
					Kind:  "ClusterRoleBinding",
				},
			},
      // ... leaving out for brevity
		},
	},
}

for _, tc := range testCases {
	t.Run(tc.name, func(t *testing.T) {
		g := NewWithT(t)

		objectToPatch, err := utilyaml.ToUnstructured([]byte(tc.objectsToPatchYaml))
		g.Expect(err).NotTo(HaveOccurred())

		result, err := ApplyGenericPatches(objectToPatch, tc.patches)
		if tc.expectedError {
			g.Expect(err).To(HaveOccurred())
		}

		g.Expect(err).NotTo(HaveOccurred())

		resultYaml, err := utilyaml.FromUnstructured(result)
		g.Expect(err).NotTo(HaveOccurred())

		g.Expect(string(resultYaml)).To(Equal(tc.expectedOutput))
	})
}

I would also add an additional test that shows the output of the existing spec.manifestPatches matches the expected output of that and the output of the new spec.patches.

[nalum]

And a test that checks that either spec.patches or spec.manifestPatches is set, not both of them.

[hjoshi123]

Yup will definitely add more tests.. just wanted to get the feedback on the logic itself.. and the tests are failing because of outdated generate files and log package (will fix it later this evening)
The way I wrote the logic if both are put in then spec.patches gets the priority. We can change the behavior but I feel if we document it, it should be fine.

[richardcase]

The way I wrote the logic if both are put in then spec.patches gets the priority.

It should be one or the other. If a user sets both, then the webhook validation should fail. The reason is that this makes the migration and deprecation of the old field easier.

[hjoshi123]

@richardcase I did add the kubebuilder validations to check if both of them are present to error out... I am not sure about how to test it.. Should it be part of phases_test?

@nalum I did fix the test to match the existing behavior for both merge and json patches.

[nalum]

Should also update the patching provider manifests page.

[richardcase]

/ok-to-test