Skip to content

Commit e3c0b30

Browse files
Callisto13Callisto13
authored
feat: Add MountPoint option to AdditionalVolumes (#254)
And update templates to use a dedicated binary image for `kernel.image` and an additional volume to provide the modules. In line with changes to our image-builder pipeline, which removes the embedded kernel modules from the OS image, this feature adds an option to include modules as an additional volume. Old images will remain available to not break those using earlier versions.
1 parent 37ae130 commit e3c0b30

9 files changed

+103
-46
lines changed

‎config/crd/bases/infrastructure.cluster.x-k8s.io_microvmmachines.yaml‎

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,11 @@ spec:
6868
kernel cmdline. Each MicroVM provider has its own recommended list,
6969
they will be used automatically. This field is for additional values.
7070
type: object
71+
labels:
72+
additionalProperties:
73+
type: string
74+
description: Labels allow you to include extra data on the Microvm
75+
type: object
7176
memoryMb:
7277
description: MemoryMb is the amount of memory in megabytes that the
7378
microvm will be allocated.
@@ -119,7 +124,12 @@ spec:
119124
description: ID is a unique identifier for this volume.
120125
type: string
121126
image:
122-
description: Image is the container image to use for the volume.
127+
description: Image is the container image to use as the source
128+
for the volume.
129+
type: string
130+
mountPoint:
131+
description: MountPoint specifies the guest mountpoint for the
132+
volume. This will only be applied to additional volumes.
123133
type: string
124134
readOnly:
125135
default: false
@@ -163,7 +173,12 @@ spec:
163173
description: ID is a unique identifier for this volume.
164174
type: string
165175
image:
166-
description: Image is the container image to use for the volume.
176+
description: Image is the container image to use as the source
177+
for the volume.
178+
type: string
179+
mountPoint:
180+
description: MountPoint specifies the guest mountpoint for the
181+
volume. This will only be applied to additional volumes.
167182
type: string
168183
readOnly:
169184
default: false

‎config/crd/bases/infrastructure.cluster.x-k8s.io_microvmmachinetemplates.yaml‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,12 @@ spec:
103103
recommended list, they will be used automatically. This
104104
field is for additional values.
105105
type: object
106+
labels:
107+
additionalProperties:
108+
type: string
109+
description: Labels allow you to include extra data on the
110+
Microvm
111+
type: object
106112
memoryMb:
107113
description: MemoryMb is the amount of memory in megabytes
108114
that the microvm will be allocated.
@@ -155,8 +161,13 @@ spec:
155161
description: ID is a unique identifier for this volume.
156162
type: string
157163
image:
158-
description: Image is the container image to use for the
159-
volume.
164+
description: Image is the container image to use as the
165+
source for the volume.
166+
type: string
167+
mountPoint:
168+
description: MountPoint specifies the guest mountpoint
169+
for the volume. This will only be applied to additional
170+
volumes.
160171
type: string
161172
readOnly:
162173
default: false
@@ -203,8 +214,13 @@ spec:
203214
description: ID is a unique identifier for this volume.
204215
type: string
205216
image:
206-
description: Image is the container image to use for
207-
the volume.
217+
description: Image is the container image to use as
218+
the source for the volume.
219+
type: string
220+
mountPoint:
221+
description: MountPoint specifies the guest mountpoint
222+
for the volume. This will only be applied to additional
223+
volumes.
208224
type: string
209225
readOnly:
210226
default: false

‎docs/compatibility.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ This table shows the compatibility between CAPMVM and Flintlock versions.
44

55
| CAPMVM | Flintlock |
66
| ----------- | ---------------------- |
7+
| `v0.8.0` | `>= v0.5.0` |
78
| `v0.7.0` | `>= v0.1.0` |
89
| `v0.6.0` | `v0.1.0` |
910
| `v0.5.x` | `v0.1.0-alpha.9` |

‎docs/development-with-tilt.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -101,13 +101,14 @@ Create the declaration for your cluster. We will use the template in the repo.
101101
3. Create a cluster declaration from the template
102102

103103
```bash
104-
export KUBERNETES_VERSION=v1.20.0
104+
export KUBERNETES_VERSION=v1.23.5
105105
export CLUSTER_NAME=mvm-test
106106
export CONTROL_PLANE_MACHINE_COUNT=1
107107
export WORKER_MACHINE_COUNT=1
108108
export CONTROL_PLANE_VIP=192.168.8.15
109-
export MVM_ROOT_IMAGE=docker.io/richardcase/ubuntu-bionic-test:cloudimage_v0.0.1
110-
export MVM_KERNEL_IMAGE=docker.io/richardcase/ubuntu-bionic-kernel:0.0.11
109+
export MVM_ROOT_IMAGE=ghcr.io/weaveworks-liquidmetal/capmvm-k8s-os:1.23.5
110+
export MVM_KERNEL_IMAGE=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77
111+
export MVM_KERNEL_MODULES_IMAGE=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77
111112
# NOTE: change 192.168.8.2 to be the IP address from step 2
112113
export HOST_ENDPOINT=192.168.8.2:9090
113114

‎go.mod‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,11 @@ require (
77
github.com/onsi/ginkgo v1.16.5
88
github.com/onsi/gomega v1.20.0
99
github.com/spf13/pflag v1.0.5
10-
github.com/weaveworks-liquidmetal/controller-pkg/client v0.0.0-20221118161315-83de77687232
11-
github.com/weaveworks-liquidmetal/controller-pkg/services/microvm v0.0.0-20221118161315-83de77687232
12-
github.com/weaveworks-liquidmetal/controller-pkg/types/microvm v0.0.0-20221118161315-83de77687232
13-
github.com/weaveworks-liquidmetal/flintlock/api v0.0.0-20221117153111-bd29de31356f
14-
github.com/weaveworks-liquidmetal/flintlock/client v0.0.0-20221117153111-bd29de31356f
10+
github.com/weaveworks-liquidmetal/controller-pkg/client v0.0.0-20230113164813-f1ccd249c9ba
11+
github.com/weaveworks-liquidmetal/controller-pkg/services/microvm v0.0.0-20230113164813-f1ccd249c9ba
12+
github.com/weaveworks-liquidmetal/controller-pkg/types/microvm v0.0.0-20230113164813-f1ccd249c9ba
13+
github.com/weaveworks-liquidmetal/flintlock/api v0.0.0-20230109155544-97f4f12ca184
14+
github.com/weaveworks-liquidmetal/flintlock/client v0.0.0-20230109155544-97f4f12ca184
1515
github.com/yitsushi/macpot v1.0.2
1616
google.golang.org/grpc v1.50.1
1717
google.golang.org/protobuf v1.28.1

‎go.sum‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -610,16 +610,16 @@ github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1
610610
github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
611611
github.com/valyala/fastjson v1.6.3 h1:tAKFnnwmeMGPbwJ7IwxcTPCNr3uIzoIj3/Fh90ra4xc=
612612
github.com/valyala/fastjson v1.6.3/go.mod h1:CLCAqky6SMuOcxStkYQvblddUtoRxhYMGLrsQns1aXY=
613-
github.com/weaveworks-liquidmetal/controller-pkg/client v0.0.0-20221118161315-83de77687232 h1:ODvIKIfUzOYEOsMiUkIOSjKF7CzveE29/L46GsDLp40=
614-
github.com/weaveworks-liquidmetal/controller-pkg/client v0.0.0-20221118161315-83de77687232/go.mod h1:0snthlwNQ731b4CPJtSZ8I76FgS02wAOfyar9ss+X+U=
615-
github.com/weaveworks-liquidmetal/controller-pkg/services/microvm v0.0.0-20221118161315-83de77687232 h1:Nsd0XLPPizVY8oXR+D34GR5JzTpu4XGC3oVFaxudH5Q=
616-
github.com/weaveworks-liquidmetal/controller-pkg/services/microvm v0.0.0-20221118161315-83de77687232/go.mod h1:jld9hBa9YZ4ujs9+mpQ+QWWRq/aQ+t7Rlv9C70I3bes=
617-
github.com/weaveworks-liquidmetal/controller-pkg/types/microvm v0.0.0-20221118161315-83de77687232 h1:896AcgVPh5xcPCAnvC5WR5wyPFOz53+rndKsoeFQ6Vg=
618-
github.com/weaveworks-liquidmetal/controller-pkg/types/microvm v0.0.0-20221118161315-83de77687232/go.mod h1:KPlbfZsmo9xlfzl98cFBqGYc9hpgSYljICR+LrqKJUA=
619-
github.com/weaveworks-liquidmetal/flintlock/api v0.0.0-20221117153111-bd29de31356f h1:xHtKo4eiR1pRZUrXxlGF3KDuoa6xYjBGuPP47M9eVWM=
620-
github.com/weaveworks-liquidmetal/flintlock/api v0.0.0-20221117153111-bd29de31356f/go.mod h1:JPML9O56MoPKGX97jfj++BtuFFS84jm4T+jWQBjO5Uc=
621-
github.com/weaveworks-liquidmetal/flintlock/client v0.0.0-20221117153111-bd29de31356f h1:xlBe6YZx8/cQMVAh0FaQ1IYaqeZPjP0kTOg3dHTcSdQ=
622-
github.com/weaveworks-liquidmetal/flintlock/client v0.0.0-20221117153111-bd29de31356f/go.mod h1:d65mpsT+pbMnMJZhUqeNtt7lcQnsX8cowl9dTRkniSI=
613+
github.com/weaveworks-liquidmetal/controller-pkg/client v0.0.0-20230113164813-f1ccd249c9ba h1:Hestt4C2MDRG1xEb2kVbCetmlBnPulUab8LuBtzXsl4=
614+
github.com/weaveworks-liquidmetal/controller-pkg/client v0.0.0-20230113164813-f1ccd249c9ba/go.mod h1:0snthlwNQ731b4CPJtSZ8I76FgS02wAOfyar9ss+X+U=
615+
github.com/weaveworks-liquidmetal/controller-pkg/services/microvm v0.0.0-20230113164813-f1ccd249c9ba h1:WreSgeHqwWSyvNHvsk1OHr9zx/a1XkQEkhWyuam3XCU=
616+
github.com/weaveworks-liquidmetal/controller-pkg/services/microvm v0.0.0-20230113164813-f1ccd249c9ba/go.mod h1:JPXXxIrs6HC1qmy4//kioH4myd4h1G9b02yvVl0GYEg=
617+
github.com/weaveworks-liquidmetal/controller-pkg/types/microvm v0.0.0-20230113164813-f1ccd249c9ba h1:czFGmavIfOHuyVt+UInQBfF7tCCI3Ov/ZigJh37qzok=
618+
github.com/weaveworks-liquidmetal/controller-pkg/types/microvm v0.0.0-20230113164813-f1ccd249c9ba/go.mod h1:KPlbfZsmo9xlfzl98cFBqGYc9hpgSYljICR+LrqKJUA=
619+
github.com/weaveworks-liquidmetal/flintlock/api v0.0.0-20230109155544-97f4f12ca184 h1:o0CJeY1Dbq+xxw5mx32S+5nOft5nf2NZMjIbHlLvFpQ=
620+
github.com/weaveworks-liquidmetal/flintlock/api v0.0.0-20230109155544-97f4f12ca184/go.mod h1:JPML9O56MoPKGX97jfj++BtuFFS84jm4T+jWQBjO5Uc=
621+
github.com/weaveworks-liquidmetal/flintlock/client v0.0.0-20230109155544-97f4f12ca184 h1:7vrApNvF61104ULFq5lzH1IGfIE+94MwV/SmXlYvss0=
622+
github.com/weaveworks-liquidmetal/flintlock/client v0.0.0-20230109155544-97f4f12ca184/go.mod h1:d65mpsT+pbMnMJZhUqeNtt7lcQnsX8cowl9dTRkniSI=
623623
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
624624
github.com/yitsushi/macpot v1.0.2 h1:LSiBfVKRhzrvUTBDO2GZYAQvS+gG9wdzPaaiNeR29KM=
625625
github.com/yitsushi/macpot v1.0.2/go.mod h1:7KBzokvUNbcsR1VcmKwmYRWB2FyAWcIll4L93b2A8q4=

‎templates/cluster-template-cilium.yaml‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ metadata:
4444
name: "${CLUSTER_NAME}-control-plane"
4545
spec:
4646
replicas: ${CONTROL_PLANE_MACHINE_COUNT}
47-
version: "${KUBERNETES_VERSION:=v1.21.8}"
47+
version: "${KUBERNETES_VERSION:=v1.23.5}"
4848
machineTemplate:
4949
infrastructureRef:
5050
kind: MicrovmMachineTemplate
@@ -76,10 +76,14 @@ spec:
7676
memoryMb: 2048
7777
rootVolume:
7878
id: root
79-
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.21.8}"
79+
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.23.5}"
8080
kernel:
8181
filename: "boot/vmlinux"
82-
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/flintlock-kernel:5.10.77}"
82+
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77}"
83+
volumes:
84+
- id: modules
85+
image: "${MVM_KERNEL_MODULES_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77}"
86+
mountPoint: /lib/modules/5.10.77
8387
kernelCmdline: {}
8488
networkInterfaces:
8589
- guestDeviceName: "eth1"
@@ -97,7 +101,7 @@ spec:
97101
template:
98102
spec:
99103
clusterName: "${CLUSTER_NAME}"
100-
version: "${KUBERNETES_VERSION:=v1.21.8}"
104+
version: "${KUBERNETES_VERSION:=v1.23.5}"
101105
bootstrap:
102106
configRef:
103107
name: "${CLUSTER_NAME}-md-0"
@@ -119,10 +123,14 @@ spec:
119123
memoryMb: 2048
120124
rootVolume:
121125
id: root
122-
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.21.8}"
126+
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.23.5}"
123127
kernel:
124128
filename: "boot/vmlinux"
125-
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/flintlock-kernel:5.10.77}"
129+
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77}"
130+
volumes:
131+
- id: modules
132+
image: "${MVM_KERNEL_MODULES_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77}"
133+
mountPoint: /lib/modules/5.10.77
126134
kernelCmdline: {}
127135
networkInterfaces:
128136
- guestDeviceName: "eth1"

‎templates/cluster-template-flannel.yaml‎

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ metadata:
4444
name: "${CLUSTER_NAME}-control-plane"
4545
spec:
4646
replicas: ${CONTROL_PLANE_MACHINE_COUNT}
47-
version: "${KUBERNETES_VERSION:=v1.21.8}"
47+
version: "${KUBERNETES_VERSION:=v1.23.5}"
4848
machineTemplate:
4949
infrastructureRef:
5050
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
@@ -74,17 +74,21 @@ metadata:
7474
spec:
7575
template:
7676
spec:
77+
rootVolume:
78+
id: root
79+
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.23.5}"
7780
kernel:
7881
filename: boot/vmlinux
79-
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/flintlock-kernel:5.10.77}"
82+
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77}"
83+
volumes:
84+
- id: modules
85+
image: "${MVM_KERNEL_MODULES_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77}"
86+
mountPoint: /lib/modules/5.10.77
8087
kernelCmdline: {}
8188
memoryMb: 2048
8289
networkInterfaces:
8390
- guestDeviceName: eth1
8491
type: macvtap
85-
rootVolume:
86-
id: root
87-
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.21.8}"
8892
vcpu: 2
8993
---
9094
apiVersion: cluster.x-k8s.io/v1beta1
@@ -99,7 +103,7 @@ spec:
99103
template:
100104
spec:
101105
clusterName: "${CLUSTER_NAME}"
102-
version: "${KUBERNETES_VERSION:=v1.21.8}"
106+
version: "${KUBERNETES_VERSION:=v1.23.5}"
103107
bootstrap:
104108
configRef:
105109
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
@@ -117,17 +121,21 @@ metadata:
117121
spec:
118122
template:
119123
spec:
124+
rootVolume:
125+
id: root
126+
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.23.5}"
120127
kernel:
121128
filename: boot/vmlinux
122-
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/flintlock-kernel:5.10.77}"
129+
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77}"
130+
volumes:
131+
- id: modules
132+
image: "${MVM_KERNEL_MODULES_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77}"
133+
mountPoint: /lib/modules/5.10.77
123134
kernelCmdline: {}
124135
memoryMb: 2048
125136
networkInterfaces:
126137
- guestDeviceName: eth1
127138
type: macvtap
128-
rootVolume:
129-
id: root
130-
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.21.8}"
131139
vcpu: 2
132140
---
133141
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1

‎templates/cluster-template.yaml‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ metadata:
4040
name: "${CLUSTER_NAME}-control-plane"
4141
spec:
4242
replicas: ${CONTROL_PLANE_MACHINE_COUNT}
43-
version: "${KUBERNETES_VERSION:=v1.21.8}"
43+
version: "${KUBERNETES_VERSION:=v1.23.5}"
4444
machineTemplate:
4545
infrastructureRef:
4646
kind: MicrovmMachineTemplate
@@ -72,10 +72,14 @@ spec:
7272
memoryMb: 2048
7373
rootVolume:
7474
id: root
75-
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.21.8}"
75+
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.23.5}"
7676
kernel:
7777
filename: "boot/vmlinux"
78-
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/flintlock-kernel:5.10.77}"
78+
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77}"
79+
volumes:
80+
- id: modules
81+
image: "${MVM_KERNEL_MODULES_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77}"
82+
mountPoint: /lib/modules/5.10.77
7983
kernelCmdline: {}
8084
networkInterfaces:
8185
- guestDeviceName: "eth1"
@@ -93,7 +97,7 @@ spec:
9397
template:
9498
spec:
9599
clusterName: "${CLUSTER_NAME}"
96-
version: "${KUBERNETES_VERSION:=v1.21.8}"
100+
version: "${KUBERNETES_VERSION:=v1.23.5}"
97101
bootstrap:
98102
configRef:
99103
name: "${CLUSTER_NAME}-md-0"
@@ -115,10 +119,14 @@ spec:
115119
memoryMb: 2048
116120
rootVolume:
117121
id: root
118-
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.21.8}"
122+
image: "${MVM_ROOT_IMAGE:=ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes:1.23.5}"
119123
kernel:
120124
filename: "boot/vmlinux"
121-
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/flintlock-kernel:5.10.77}"
125+
image: "${MVM_KERNEL_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-bin:5.10.77}"
126+
volumes:
127+
- id: modules
128+
image: "${MVM_KERNEL_MODULES_IMAGE:=ghcr.io/weaveworks-liquidmetal/kernel-modules:5.10.77}"
129+
mountPoint: /lib/modules/5.10.77
122130
kernelCmdline: {}
123131
networkInterfaces:
124132
- guestDeviceName: "eth1"

0 commit comments

Comments
 (0)