Skip to content

Commit b018f1d

Browse files
authored
Merge pull request #186 from lool/optional-dtb
Optional DTB
2 parents 26be435 + 64079e3 commit b018f1d

File tree

2 files changed

+156
-89
lines changed

2 files changed

+156
-89
lines changed

README.md

Lines changed: 28 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
[![daily build status](https://img.shields.io/github/actions/workflow/status/qualcomm-linux/qcom-deb-images/build-daily.yml?label=daily%20build)](https://github.com/qualcomm-linux/qcom-deb-images/actions/workflows/build-daily.yml)
55
[![mainline build status](https://img.shields.io/github/actions/workflow/status/qualcomm-linux/qcom-deb-images/linux.yml?label=weekly%20mainline%20build)](https://github.com/qualcomm-linux/qcom-deb-images/actions/workflows/linux.yml)
66

7-
A collection of recipes to build Qualcomm Linux images for deb based operating systems. The current focus of this project is to provide mainline centric images for Qualcomm® IoT platforms as to demonstrate the state of upstream open source software, help developers getting started, and support continuous development and continuous testing efforts.
7+
A collection of recipes to build Qualcomm Linux images for deb based operating systems. The current focus of this project is to provide mainline-centric images for Qualcomm® IoT platforms as to demonstrate the state of upstream open source software, help developers getting started, and support continuous development and continuous testing efforts.
88

99
Initially, this repository provides [debos](https://github.com/go-debos/debos) recipes based on Debian trixie for boards such as the Qualcomm RB3 Gen 2.
1010

@@ -16,7 +16,7 @@ On standard Linux distros like Debian, firmware updates are generally delivered
1616

1717
### Firmware delivery
1818

19-
On a Desktop system, its usually GNOME Software which monitors LVFS for any firmware updates and pushes to fwupd if any. On a headless system like most embedded devices, the fwupdmgr command line tool can be used to monitor LVFS for firmware updates as follows:
19+
On a Desktop system, it's usually GNOME Software which monitors LVFS for any firmware updates and pushes to fwupd if any. On a headless system like most embedded devices, the fwupdmgr command-line tool can be used to monitor LVFS for firmware updates as follows:
2020

2121
```bash
2222
# Download latest metadata from LVFS
@@ -31,10 +31,10 @@ fwupdmgr update
3131

3232
### Firmware on devices supported by Qualcomm Linux
3333

34-
The firmware on Qualcomm devices is expected to support UEFI UpdateCapsule plugin for fwupd daemon. However, currently firmware for Qualcomm devices in not available in LVFS which is a work in progress as of now. In order to play with UEFI firmware capsule updates, one can use fwupdtool to locally update firmware like on RB1 as follows:
34+
The firmware on Qualcomm devices is expected to support UEFI UpdateCapsule plugin for fwupd daemon. However, currently firmware for Qualcomm devices is not available in LVFS which is a work in progress as of now. In order to play with UEFI firmware capsule updates, one can use fwupdtool to locally update firmware like on RB1 as follows:
3535

3636
```bash
37-
# Transfer U-Boot firmware cabinet archive build from scripts/build-u-boot-rb1.sh to RB1
37+
# Transfer U-Boot firmware cabinet archive built by scripts/build-u-boot-rb1.sh to RB1
3838
sudo fwupdtool install u-boot.cab
3939
# It will ask for a reboot for the UEFI firmware capsule update to happen
4040
```
@@ -86,7 +86,7 @@ To build flashable assets for all supported boards, follow these steps:
8686

8787
1. build disk and filesystem images from the root filesystem tarball
8888
```bash
89-
# the default is to build an UFS image
89+
# the default is to build a UFS image
9090
debos debos-recipes/qualcomm-linux-debian-image.yaml
9191
9292
# (optional) if you want SD card images or support for eMMC boards, run
@@ -100,12 +100,15 @@ To build flashable assets for all supported boards, follow these steps:
100100
101101
# (optional) if you've built U-Boot for the RB1, run this instead:
102102
#debos -t u_boot_rb1:u-boot/rb1-boot.img debos-recipes/qualcomm-linux-debian-flash.yaml
103+
104+
# (optional) build only a subset of boards:
105+
#debos -t target_boards:qcs615-adp-air-sa6155p,qcs6490-rb3gen2-vision-kit debos-recipes/qualcomm-linux-debian-flash.yaml
103106
```
104107

105108
1. enter Emergency Download Mode (see section below) and flash the resulting images with QDL
106109
```bash
107110
# for RB3 Gen2 Vision Kit or UFS boards in general
108-
cd flash_rb3gen2-vision-kit
111+
cd flash_qcs6490-rb3gen2-vision-kit
109112
qdl --storage ufs prog_firehose_ddr.elf rawprogram[0-9].xml patch[0-9].xml
110113
111114
# for RB1 or eMMC boards in general
@@ -114,7 +117,7 @@ To build flashable assets for all supported boards, follow these steps:
114117

115118
### Debos tips
116119

117-
By default, debos will try to pick a fast build backend. It will prefer to use its KVM backend (`-b kvm`) when available, and otherwise an UML environment (`-b uml`). If none of these work, a solid backend is QEMU (`-b qemu`). Because the target images are arm64, building under QEMU can be really slow, especially when building from another architecture such as amd64.
120+
By default, debos will try to pick a fast build backend. It will prefer to use its KVM backend (`-b kvm`) when available, and otherwise a UML environment (`-b uml`). If none of these work, a solid backend is QEMU (`-b qemu`). Because the target images are arm64, building under QEMU can be really slow, especially when building from another architecture such as amd64.
118121

119122
To build large images, the debos resource defaults might not be sufficient. Consider raising the default debos memory and scratchsize settings. This should provide a good set of minimum defaults:
120123
```bash
@@ -125,32 +128,42 @@ debos --fakemachine-backend qemu --memory 1GiB --scratchsize 4GiB debos-recipes/
125128

126129
A few options are provided in the debos recipes; for the root filesystem recipe:
127130
- `localdebs`: path to a directory with local deb packages to install (NB: debos expects relative pathnames)
128-
- `xfcedesktop`: install a Xfce desktop environment; default: console only environment
131+
- `xfcedesktop`: install an Xfce desktop environment; default: console only environment
129132

130133
For the image recipe:
131-
- `dtb`: override the firmware provided device tree with one from the linux kernel, e.g. `qcom/qcs6490-rb3gen2.dtb`; default: don't override
132-
- `imagetype`: either `ufs` (the default) or (`sdcard`); UFS images are named disk-ufs.img and use 4096 bytes sectors and SD card images are named disk-sdcard.img and use 512 bytes sectors
133-
- `imagesize`: set the output disk image size; default: `4.5GiB`
134+
- `dtb`: override the firmware provided device tree with one from the Linux kernel, e.g. `qcom/qcs6490-rb3gen2.dtb`; default: don't override
135+
- `imagetype`: either `ufs` (the default) or `sdcard`; UFS images are named disk-ufs.img and use 4096-byte sectors and SD card images are named disk-sdcard.img and use 512-byte sectors
136+
- `imagesize`: set the output disk image size; default: `4GiB`
134137
135138
For the flash recipe:
136139
- `u_boot_rb1`: prebuilt U-Boot binary for RB1 in Android boot image format -- see below (NB: debos expects relative pathnames)
140+
- `target_boards`: comma-separated list of board names to build (default: `all`). Accepted values are the board names defined in the flash recipe, e.g. `qcs615-adp-air-sa6155p`, `qcs6490-rb3gen2-vision-kit`, `qcs8300-ride`, `qcs9100-ride-r3`, `qrb2210-rb1`.
141+
142+
Note: Boards whose required device tree (.dtb) is not present in `dtbs.tar.gz` are automatically skipped during flash asset generation.
143+
144+
Deprecated flash options:
145+
- `build_qcs615`, `build_qcm6490`, `build_qcs8300`, `build_qcs9100`, `build_rb1`: these per-family/per-board toggles are deprecated and will be removed. Use `target_boards` instead to select which boards to build.
137146
138147
Here are some example invocations:
139148
```bash
140-
# build the root filesystem with Xfce and a kernel from experimental
141-
debos -t xfcedesktop:true -t experimentalkernel:true debos-recipes/qualcomm-linux-debian-rootfs.yaml
149+
# build the root filesystem with Xfce
150+
debos -t xfcedesktop:true debos-recipes/qualcomm-linux-debian-rootfs.yaml
142151
143152
# build an image where systemd overrides the firmware device tree with the one
144153
# for RB3 Gen2
145154
debos -t dtb:qcom/qcs6490-rb3gen2.dtb debos-recipes/qualcomm-linux-debian-image.yaml
146155
147156
# build an SD card image
148157
debos -t imagetype:sdcard debos-recipes/qualcomm-linux-debian-image.yaml
158+
159+
# build flash assets for a subset of boards
160+
# (see flash recipe for accepted board names)
161+
debos -t target_boards:qcs615-adp-air-sa6155p,qcs6490-rb3gen2-vision-kit debos-recipes/qualcomm-linux-debian-flash.yaml
149162
```
150163
151164
### Flashing tips
152165
153-
The `disk-sdcard.img` disk image can simply be written to a SD card, albeit most Qualcomm boards boot from internal storage by default. With an SD card, the board will use boot firmware from internal storage (eMMC or UFS) and do an EFI boot from the SD card if the firmware can't boot from internal storage.
166+
The `disk-sdcard.img` disk image can simply be written to an SD card, albeit most Qualcomm boards boot from internal storage by default. With an SD card, the board will use boot firmware from internal storage (eMMC or UFS) and do an EFI boot from the SD card if the firmware can't boot from internal storage.
154167

155168
For UFS boards, if there is no need to update the boot firmware, the `disk-ufs.img` disk image can also be flashed on the first LUN of the internal UFS storage with [qdl](https://github.com/linux-msm/qdl). Create a `rawprogram-ufs.xml` file as follows:
156169
```xml
@@ -177,7 +190,7 @@ To enter EDL mode:
177190
1. connect a cable from the flashing host to the USB type-C port on the board
178191
1. run qdl to flash the board
179192
180-
NB: It's also possible to run qdl from the host while the baord is not connected, and starting the board directly in EDL mode.
193+
NB: It's also possible to run qdl from the host while the board is not connected, then start the board directly in EDL mode.
181194

182195
## Development
183196

debos-recipes/qualcomm-linux-debian-flash.yaml

Lines changed: 128 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@
88
{{- end -}}
99
{{- $buildid := or .buildid "" }}
1010

11+
{{- $target_boards := or .target_boards "all" }}
12+
1113
architecture: arm64
1214

1315
actions:
@@ -166,11 +168,52 @@ actions:
166168
# path to unpacked qcom-ptool tarball
167169
QCOM_PTOOL="$(ls -d "${ROOTDIR}/../qcom-ptool.tar.gz.d/qcom-ptool-"*)"
168170
171+
# build a list of supported dtbs from the dtbs tarball
172+
dtbs_file="build/dtbs.txt"
173+
: >"${dtbs_file}"
174+
if [ -f "${ARTIFACTDIR}/dtbs.tar.gz" ]; then
175+
tar -tzf "${ARTIFACTDIR}/dtbs.tar.gz" --wildcards '*.dtb' \
176+
>>"$dtbs_file"
177+
fi
178+
179+
# optional list of board names to build
180+
targets_file="build/targets.txt"
181+
rm -f "${targets_file}"
182+
case "{{ $target_boards }}" in
183+
all)
184+
# no override; build all possible boards (default)
185+
touch "${targets_file}"
186+
{{- range $board := $boards }}
187+
echo "{{ $board.name }}" >>"${targets_file}"
188+
{{- end }}
189+
;;
190+
*)
191+
echo "{{ $target_boards }}" |
192+
tr ',' '\n' |
193+
sed '/^$/d' |
194+
sort -u >"${targets_file}"
195+
;;
196+
esac
197+
169198
{{- range $board := $boards }}
170199
### board: {{ $board.name }}
171200
### platform: {{ $board.platform }}
172201
### silicon family: {{ $board.silicon_family }}
173202

203+
# set skip_board if board isn't listed
204+
skip_board=false
205+
if ! grep -Fxq "{{ $board.name }}" "${targets_file}"; then
206+
skip_board=true
207+
echo "Skipping board {{ $board.name }}: not in target list"
208+
fi
209+
{{- if $board.dtb }}
210+
# set skip_board if board has a dtb and dtb isn't present
211+
if ! grep -Fxq "{{ $board.dtb }}" "${dtbs_file}"; then
212+
skip_board=true
213+
echo "Skipping board {{ $board.name }}: dtb not available"
214+
fi
215+
{{- end }}
216+
174217
# unpack boot binaries
175218
mkdir -v build/{{ $board.name }}_boot-binaries
176219
unzip "${ROOTDIR}/../{{ $board.boot_binaries_download.filename }}" \
@@ -264,83 +307,94 @@ actions:
264307
"${QCOM_PTOOL}/ptool.py" -x ptool-partitions.xml
265308
)
266309

267-
# create board-specific flash directory
268-
flash_dir="${ARTIFACTDIR}/flash_{{ $board.name }}"
269-
rm -rf "${flash_dir}"
270-
mkdir -v "${flash_dir}"
271-
# copy platform partition files
272-
cp --preserve=mode,timestamps -v build/ptool/{{ $board.platform }}/* \
273-
"${flash_dir}"
274-
# adjust paths in contents.xml to use board-specific flash_* subdir
275-
if [ -e "${flash_dir}/contents.xml" ]; then
276-
# one set of backslashes for shell quoting, another one for sed
277-
# parsing
278-
windows_path=".\\\\flash_{{ $board.name }}\\\\"
279-
linux_path="./flash_{{ $board.name }}/"
280-
sed -i \
281-
-e "s:<windows_root_path>.*:<windows_root_path>${windows_path}</windows_root_path>:" \
282-
-e "s:<linux_root_path>.*:<linux_root_path>${linux_path}</linux_root_path>:" \
283-
"${flash_dir}/contents.xml"
310+
if [ "${skip_board}" = false ]; then
311+
# create board-specific flash directory
312+
flash_dir="${ARTIFACTDIR}/flash_{{ $board.name }}"
313+
rm -rf "${flash_dir}"
314+
mkdir -v "${flash_dir}"
315+
# copy platform partition files
316+
cp --preserve=mode,timestamps -v \
317+
build/ptool/{{ $board.platform }}/* "${flash_dir}"
318+
# adjust paths in contents.xml to use board-specific flash_* subdir
319+
if [ -e "${flash_dir}/contents.xml" ]; then
320+
# one set of backslashes for shell quoting, another one for sed
321+
# parsing
322+
windows_path=".\\\\flash_{{ $board.name }}\\\\"
323+
linux_path="./flash_{{ $board.name }}/"
324+
sed -i \
325+
-e "s:<windows_root_path>.*:<windows_root_path>${windows_path}</windows_root_path>:" \
326+
-e "s:<linux_root_path>.*:<linux_root_path>${linux_path}</linux_root_path>:" \
327+
"${flash_dir}/contents.xml"
328+
fi
329+
# remove BLANK_GPT and WIPE_PARTITIONS files as it's common for
330+
# people to run "qdl rawprogram*.xml", mistakingly including these;
331+
# perhaps ptool should have a flag not to generate these; note that
332+
# there are wipe_rawprogram*.xml files still
333+
rm -v "${flash_dir}"/rawprogram*_BLANK_GPT.xml
334+
rm -v "${flash_dir}"/rawprogram*_WIPE_PARTITIONS.xml
335+
# copy silicon family boot binaries; these shouldn't ship partition
336+
# files, but make sure not to accidentally clobber any such file
337+
find build/{{ $board.name }}_boot-binaries \
338+
-not -name 'gpt_*' \
339+
-not -name 'patch*.xml' \
340+
-not -name 'rawprogram*.xml' \
341+
-not -name 'wipe*.xml' \
342+
-not -name 'zeros_*' \
343+
\( \
344+
-name LICENSE \
345+
-or -name Qualcomm-Technologies-Inc.-Proprietary \
346+
-or -name 'prog_*' \
347+
-or -name '*.bin' \
348+
-or -name '*.elf' \
349+
-or -name '*.fv' \
350+
-or -name '*.mbn' \
351+
\) \
352+
-exec cp --preserve=mode,timestamps -v '{}' "${flash_dir}" \;
353+
fi
354+
{{- if $board.u_boot_file }}
355+
if [ "${skip_board}" = false ]; then
356+
# copy U-Boot binary to boot.img;
357+
# qcom-ptool/platforms/*/partitions.conf uses filename=boot.img
358+
# boot_a and boot_b partitions
359+
cp --preserve=mode,timestamps -v \
360+
"${ARTIFACTDIR}/{{ $board.u_boot_file }}" "${flash_dir}/boot.img"
284361
fi
285-
# remove BLANK_GPT and WIPE_PARTITIONS files as it's common for people
286-
# to run "qdl rawprogram*.xml", mistakingly including these; perhaps
287-
# ptool should have a flag not to generate these; note that there are
288-
# wipe_rawprogram*.xml files still
289-
rm -v "${flash_dir}"/rawprogram*_BLANK_GPT.xml
290-
rm -v "${flash_dir}"/rawprogram*_WIPE_PARTITIONS.xml
291-
# copy silicon family boot binaries; these shouldn't ship partition
292-
# files, but make sure not to accidentally clobber any such file
293-
find build/{{ $board.name }}_boot-binaries \
294-
-not -name 'gpt_*' \
295-
-not -name 'patch*.xml' \
296-
-not -name 'rawprogram*.xml' \
297-
-not -name 'wipe*.xml' \
298-
-not -name 'zeros_*' \
299-
\( \
300-
-name LICENSE \
301-
-or -name Qualcomm-Technologies-Inc.-Proprietary \
302-
-or -name 'prog_*' \
303-
-or -name '*.bin' \
304-
-or -name '*.elf' \
305-
-or -name '*.fv' \
306-
-or -name '*.mbn' \
307-
\) \
308-
-exec cp --preserve=mode,timestamps -v '{}' "${flash_dir}" \;
309-
{{- if $board.u_boot_file }}
310-
# copy U-Boot binary to boot.img;
311-
# qcom-ptool/platforms/*/partitions.conf uses filename=boot.img
312-
# boot_a and boot_b partitions
313-
cp --preserve=mode,timestamps -v "${ARTIFACTDIR}/{{ $board.u_boot_file }}" \
314-
"${flash_dir}/boot.img"
315-
{{- end }}
362+
{{- end }}
316363

317-
{{- if $board.cdt_download }}
318-
# unpack board CDT
319-
unzip "${ROOTDIR}/../{{ $board.cdt_download.filename }}" \
320-
-d build/{{ $board.name }}_cdt
321-
# copy just the CDT data; no partition or flashing files
322-
cp --preserve=mode,timestamps -v build/{{ $board.name }}_cdt/{{ $board.cdt_filename }} \
323-
"${flash_dir}"
324-
{{- end }}
364+
{{- if $board.cdt_download }}
365+
if [ "${skip_board}" = false ]; then
366+
# unpack board CDT
367+
unzip "${ROOTDIR}/../{{ $board.cdt_download.filename }}" \
368+
-d build/{{ $board.name }}_cdt
369+
# copy just the CDT data; no partition or flashing files
370+
cp --preserve=mode,timestamps -v \
371+
build/{{ $board.name }}_cdt/{{ $board.cdt_filename }} \
372+
"${flash_dir}"
373+
fi
374+
{{- end }}
325375

326-
{{- if $board.dtb }}
327-
# generate a dtb.bin FAT partition with just a single dtb for the current
328-
# board; long-term this should really be a set of dtbs and overlays as to
329-
# share dtb.bin across boards
330-
dtb_bin="${flash_dir}/dtb.bin"
331-
rm -f "${dtb_bin}"
332-
# dtb.bin is only used in UFS based boards at the moment and UFS uses a
333-
# 4k sector size, so pass -S 4096
334-
# in qcom-ptool/platforms/*/partitions.conf, dtb_a and _b partitions
335-
# are provisioned with 64MiB; create a 4MiB FAT that will comfortably fit
336-
# in these and hold the target device tree, which is 4096 KiB sized
337-
# blocks for mkfs.vfat's last argument
338-
mkfs.vfat -S 4096 -C "${dtb_bin}" 4096
339-
# extract board device tree from the root filesystem provided tarball
340-
tar -C build -xvf "${ARTIFACTDIR}/dtbs.tar.gz" "{{ $board.dtb }}"
341-
# copy into the FAT as combined-dtb.dtb
342-
mcopy -vmp -i "${dtb_bin}" "build/{{ $board.dtb }}" ::/combined-dtb.dtb
343-
{{- end }}
376+
{{- if $board.dtb }}
377+
if [ "${skip_board}" = false ]; then
378+
# generate a dtb.bin FAT partition with just a single dtb for the
379+
# current board; long-term this should really be a set of dtbs and
380+
# overlays as to share dtb.bin across boards
381+
dtb_bin="${flash_dir}/dtb.bin"
382+
rm -f "${dtb_bin}"
383+
# dtb.bin is only used in UFS based boards at the moment and UFS
384+
# uses a 4k sector size, so pass -S 4096 in
385+
# qcom-ptool/platforms/*/partitions.conf, dtb_a and _b partitions
386+
# are provisioned with 64MiB; create a 4MiB FAT that will
387+
# comfortably fit in these and hold the target device tree, which is
388+
# 4096 KiB sized blocks for mkfs.vfat's last argument
389+
mkfs.vfat -S 4096 -C "${dtb_bin}" 4096
390+
# extract board device tree from the root filesystem provided
391+
# tarball
392+
tar -C build -xvf "${ARTIFACTDIR}/dtbs.tar.gz" "{{ $board.dtb }}"
393+
# copy into the FAT as combined-dtb.dtb
394+
mcopy -vmp -i "${dtb_bin}" "build/{{ $board.dtb }}" \
395+
::/combined-dtb.dtb
396+
fi
397+
{{- end }}
344398
{{- end }}
345399

346400
# cleanup

0 commit comments

Comments
 (0)