Skip to content

Commit 1d3ab97

Browse files
taylorificfacebook-github-bot
taylorific
authored and
facebook-github-bot
committed
Make non-dokken kitchen file (facebook#295)
Summary: Make non-dokken kitchen file per facebook#290 Translated the Dokken config in `.kitchen.yml` and translated it to Vagrant in `.kitchen.vagrant.yml` along with an accompanying `TESTING.md` readme. # Testing Performed Following setup and configuration guide in `TESTING.md` configured an Ubuntu 24.04 host and tried the following platforms: - centos-stream-10 - segfaults loading OS on VirtualBox 7.x - no joy :-( - centos-stream-9 - converges successfully - debian-12, ubuntu-2204 and ubuntu-2404 - all complain about missing hddtemp package (and yes, doesn't exist on these platforms afaik), workaround to exclude from jaymzh applied, and restarting ejabberd service without a public ip, so ejabberd is skipped when running under vagrant - ubuntu-2204 and ubuntu-2404 have an issue with systemd-timesyncd being masked when chrony is installed (workaround applied) All test platforms besides centos-stream-10 now converge properly. Verified `kitchen test` works properly once `vagrant` user was added to passwordless sudo. Also verified that when I set "export CHEF_VERSION=18.4.2" it used cinc 18.4.2 instead of the latest. Pull Request resolved: facebook#295 Differential Revision: D71134838 fbshipit-source-id: 3022efa5234baa20f5d3ecb44f5a23ca82ee4bed
1 parent 3170bb1 commit 1d3ab97

File tree

6 files changed

+295
-7
lines changed

6 files changed

+295
-7
lines changed

‎.kitchen.vagrant.yml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
driver:
3+
name: vagrant
4+
box_auto_update: true
5+
box_auto_prune: true
6+
7+
provisioner:
8+
name: chef_zero
9+
product_name: cinc
10+
# our cookbooks should be compatible with the latest Chef,
11+
# but we need to ensure they still work earlier versions of 18.
12+
product_version: <%= ENV['CHEF_VERSION'] || 'latest' %>
13+
install_script: https://omnitruck.cinc.sh/install.sh
14+
chef_root: /opt/cinc
15+
16+
verifier:
17+
name: inspec
18+
19+
lifecycle:
20+
post_create:
21+
- remote: |
22+
bash -xc '
23+
# Some cookbooks assume /var/chef has been created (fb_storage)
24+
sudo mkdir -p /var/chef
25+
'
26+
27+
platforms:
28+
# We will grab boxes from bento on Vagrant Cloud
29+
# for a set of common platforms
30+
- name: centos-stream-10 # Requires VirtualBox 7.1 or higher
31+
lifecycle:
32+
post_create:
33+
- remote: |
34+
bash -xc '
35+
# enable EPEL (for stuff like hddtemp)
36+
sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm
37+
'
38+
- name: centos-stream-9
39+
lifecycle:
40+
post_create:
41+
- remote: |
42+
bash -xc '
43+
# enable EPEL (for stuff like hddtemp)
44+
sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
45+
'
46+
- name: debian-12
47+
- name: ubuntu-22.04
48+
- name: ubuntu-24.04
49+
50+
suites:
51+
- name: default
52+
attributes:
53+
# Add vagrant to passwordless sudo
54+
fb_sudo:
55+
users:
56+
vagrant:
57+
admin: "ALL=(ALL:ALL) NOPASSWD: ALL"
58+
# No /boot/efi directory for bento boxes
59+
fb_grub:
60+
force_both_efi_and_bios: false
61+
run_list:
62+
- recipe[ci_fixes]
63+
- recipe[fb_init_sample]
64+
- recipe[test_services]

‎TESTING.md‎

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# Testing with Test Kitchen
2+
3+
The default **Test Kitchen** configuration uses **Dokken**, which is ideal for
4+
running tests in **GitHub Actions** in a CI environment. However, **Dokken**
5+
is not always sufficent for local testing, especially when a full-fledged
6+
virtual machine is required.
7+
8+
To enable local testing, the **Dokken** configuration has been duplicated into
9+
**.kitchen.vagrant.yml**, allowing you to run tests using **Vagrant** and
10+
VirtualBox on a **Cinc Workstation** setup.
11+
12+
## Setting up for Local Testing
13+
14+
Follow the "Configuring" guide below to configure your system for
15+
Vagrant-based testing. To ensure that Test Kitchen uses the Vagrant
16+
configuration instead of Dokken, set the following environment variable:
17+
18+
```bash
19+
export KITCHEN_YAML=.kitchen.vagrant.yml
20+
```
21+
22+
## Listing Available Test Platforms
23+
24+
To see the available test platforms, run:
25+
26+
```bash
27+
KITCHEN_YAML=.kitchen.vagrant.yml kitchen list
28+
```
29+
30+
Example output:
31+
32+
```bash
33+
$ KITCHEN_YAML=.kitchen.vagrant.yml kitchen list
34+
Instance Driver Provisioner Verifier Transport Last Action Last Error
35+
default-centos-stream-9 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
36+
default-debian-12 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
37+
default-ubuntu-2204 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
38+
default-ubuntu-2404 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
39+
```
40+
41+
## Running a Test on a Specific Platform
42+
43+
To test on a specific platform, use the `kitchen converge` command:
44+
45+
```bash
46+
KITCHEN_YAML=.kitchen.vagrant.yml kitchen converge centos-stream-9
47+
```
48+
49+
Example output:
50+
51+
```bash
52+
KITCHEN_YAML=.kitchen.vagrant.yml kitchen converge centos-stream-9
53+
-----> Starting Test Kitchen (v3.6.0)
54+
-----> Creating <default-centos-stream-9>...
55+
Checking for updates to 'bento/centos-stream-9'
56+
Latest installed version: 202502.21.0
57+
Version constraints: > 202502.21.0
58+
Provider: virtualbox
59+
Architecture: "amd64"
60+
Box 'bento/centos-stream-9' (v202502.21.0) is running the latest version.
61+
The following boxes will be kept...
62+
....
63+
64+
Recipe: fb_init_sample::default
65+
* fb_helpers_reboot[process deferred reboots] action process_deferred (up to date)
66+
67+
Running handlers:
68+
Running handlers complete
69+
Infra Phase complete, 300/585 resources updated in 02 minutes 41 seconds
70+
[2025-03-09T18:05:51+00:00] WARN: This release of Cinc Client became end of life (EOL) on May 1st 2024. Please update to a supported release to receive new features, bug fixes, and security updates.
71+
Downloading files from <default-centos-stream-9>
72+
Finished converging <default-centos-stream-9> (6m24.22s).
73+
-----> Test Kitchen is finished. (7m3.84s)
74+
```
75+
76+
## Limitations and Workarounds
77+
78+
### CentOS Stream 10/RHEL 10 require x86-64-v3 instructions
79+
80+
CentOS Stream 10 and RHEL require x86-64-v3 capabilities to be exposed in
81+
guest VMs. Make sure you are using at least VirtualBox 7.1, as it now
82+
supports x86-64-v3 instructions and that you are running on a supported
83+
host CPU.
84+
85+
If CentOS Stream 10/RHEL10 kernel panics on boot, likely it is due to this
86+
issue. Make sure you're running on at least version 7.1 of VirtualBox and
87+
that your host CPU supports x86-64-v3 instructions.
88+
89+
### Cleaning Up: Stopping and Destroying VMs
90+
91+
Unlike with the Dokken driver, running `kitchen destroy` **without parameters**
92+
will **not** automatically stop and remove running VMs. Instead, you must
93+
specify the instance explicitly.
94+
95+
```bash
96+
KITCHEN_YAML=.kitchen.vagrant.yml kitchen destroy centos-stream-9
97+
````
98+
99+
Example output:
100+
101+
```bash
102+
$ KITCHEN_YAML=.kitchen.vagrant.yml kitchen destroy centos-stream-9
103+
-----> Starting Test Kitchen (v3.6.0)
104+
-----> Destroying <default-centos-stream-9>...
105+
==> default: Forcing shutdown of VM...
106+
==> default: Destroying VM and associated drives...
107+
Vagrant instance <default-centos-stream-9> destroyed.
108+
Finished destroying <default-centos-stream-9> (0m3.74s).
109+
-----> Test Kitchen is finished. (0m4.47s)
110+
```
111+
112+
If a VM is still lingering, you can manually identify and remove it:
113+
114+
1. Find the VM ID:
115+
116+
```bash
117+
vagrant global-status
118+
```
119+
120+
2. Force destroy the VM:
121+
122+
```bash
123+
vagrant destroy -f <VM-ID>
124+
```
125+
126+
### Testing with Different Cinc Client Versions
127+
128+
Similar to the Dokken configuration, if you need to test with a **specific
129+
version** of the Cinc Client (rather than latest), set the `CHEF_VERSION`
130+
environment variable before running Test Kitchen:
131+
132+
```bash
133+
export CHEF_VERSION="18.2.7"
134+
KITCHEN_YAML=.kitchen.vagrant.yml kitchen converge centos-stream-9
135+
```
136+
137+
This will ensure that the specified verison is installed insdie the test VM.
138+
139+
140+
## Configuring an Ubuntu 24.04 Host for Local Testing.
141+
142+
To set up your **Ubuntu 24.04** machine for local testing, ensure you are
143+
running a **64-bit AMD64-compatible host** with **CPU virtualization enabled**
144+
in your system's BIOS.
145+
146+
You'll need to install **VirtualBox, Vagrant, and Cinc Workstation**.
147+
148+
Install [Virtualbox](https://www.virtualbox.org/wiki/Linux_Downloads)
149+
150+
```bash
151+
# Add Oracle VirtualBox's GPG key:
152+
wget -O- https://www.virtualbox.org/download/oracle_vbox_2016.asc \
153+
| sudo gpg --yes --output /usr/share/keyrings/oracle-virtualbox-2016.gpg --dearmor
154+
155+
# Add the repository to Apt sources.list.d
156+
echo \
157+
"deb [arch=amd64 signed-by=/usr/share/keyrings/oracle-virtualbox-2016.gpg] https://download.virtualbox.org/virtualbox/debian \
158+
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") contrib" | \
159+
sudo tee /etc/apt/sources.list.d/virtualbox.list > /dev/null
160+
161+
# Install the latest version of Oracle VirtualBox
162+
sudo apt-get update
163+
sudo apt-get install virtualbox
164+
165+
# Recommended: Reboot to complete configuration
166+
sudo reboot
167+
168+
# Verify that Oracle VirtualBox was installed properly
169+
# Check the version
170+
VBoxManage --version
171+
172+
# Check that the VirtualBox Kernel Modules are loaded properly
173+
# Expected output should include vboxdrv, vboxnetflt and vboxnetadp
174+
$ lsmod | grep vbox
175+
vboxnetadp 28672 0
176+
vboxnetflt 32768 0
177+
vboxdrv 696320 2 vboxnetadp,vboxnetflt
178+
```
179+
180+
Install [Vagrant](https://developer.hashicorp.com/vagrant/downloads#linux):
181+
182+
```bash
183+
# Add HashiCorp's GPG key:
184+
wget -O - https://apt.releases.hashicorp.com/gpg \
185+
| sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
186+
187+
# Add the repository to Apt sources.list.d
188+
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" \
189+
| sudo tee /etc/apt/sources.list.d/hashicorp.list
190+
191+
# Install the latest version of HashiCorp Vagrant
192+
sudo apt update
193+
sudo apt install vagrant
194+
195+
# Verify vagrant is installed properly - check the vagrant version
196+
vagrant --version
197+
```
198+
199+
Install Cinc Workstation
200+
201+
```bash
202+
# Download and install cinc-workstatus via omnibus installer
203+
curl -L https://omnitruck.cinc.sh/install.sh \
204+
| sudo bash -s -- -P cinc-workstation -v 25
205+
206+
# Verify cinc-workstation is installed properly
207+
# Check the cinc version
208+
cinc --version
209+
210+
# Initialize [cinc-workstation for your shell](https://docs.chef.io/workstation/getting_started/):
211+
echo 'eval "$(chef shell-init bash)"' >> ~/.bashrc
212+
source ~/.bashrc
213+
214+
# Ensure ruby is pointing at the cinc-workstation ruby
215+
$ which ruby
216+
/opt/cinc-workstation/embedded/bin/ruby
217+
```

‎cookbooks/fb_ejabberd/recipes/default.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@
4444

4545
service 'ejabberd' do
4646
# if you try to restart ejabberd, often times epmd will still
47-
# be holding it's port open. If you stop epmd.service (which doesn't
47+
# be holding its port open. If you stop epmd.service (which doesn't
4848
# stop its socket), and restart ejabberd, that'll start everything
4949
# up properly
5050
restart_command '

‎cookbooks/fb_init_sample/recipes/default.rb‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,9 @@
9595
include_recipe 'fb_hdparm'
9696
include_recipe 'fb_sdparm'
9797
include_recipe 'fb_nscd'
98-
include_recipe 'fb_hddtemp'
98+
if fedora_derived?
99+
include_recipe 'fb_hddtemp'
100+
end
99101
end
100102
include_recipe 'fb_postfix'
101103
# HERE: nfs

‎cookbooks/fb_systemd/attributes/default.rb‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,20 +54,19 @@
5454
loader['default'] = "#{node['machine_id']}-*"
5555
end
5656

57-
# Starting from 18.04, Ubuntu uses networkd, resolved and timesyncd by default,
57+
# Starting from 18.04, Ubuntu uses networkd and resolved by default,
5858
# so default to enabling them there to prevent breakage
5959
if node.ubuntu? &&
6060
FB::Version.new(node['platform_version']) >= FB::Version.new('18.04')
6161
enable_networkd = true
6262
enable_resolved = true
6363
enable_nss_resolve = true
64-
enable_timesyncd = true
6564
else
6665
enable_networkd = false
6766
enable_resolved = false
6867
enable_nss_resolve = false
69-
enable_timesyncd = false
7068
end
69+
enable_timesyncd = false
7170

7271
# This enables a workaround in Fedora systemd-nspawn containers
7372
# so that tmpfiles can be created

‎cookbooks/test_services/recipes/default.rb‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,14 @@
3939
if node.ubuntu18? || node.ubuntu20?
4040
include_recipe 'fb_smokeping'
4141
end
42-
node.default['fb_ejabberd']['config']['hosts'] << 'sample.com'
43-
include_recipe 'fb_ejabberd'
42+
# ejabberd has issues restarting while running in Test Kitchen
43+
# if running behind a NAT, so don't run while using VirtualBox
44+
# https://github.com/openspace42/aenigma/issues/48
45+
# https://github.com/test-kitchen/test-kitchen/issues/458
46+
unless ENV['TEST_KITCHEN'] || node['virtualization']['system'] == 'vbox'
47+
node.default['fb_ejabberd']['config']['hosts'] << 'sample.com'
48+
include_recipe 'fb_ejabberd'
49+
end
4450
include_recipe 'fb_influxdb'
4551
end
4652

0 commit comments

Comments
 (0)