Canoeboot 20231026 released!

Return to index

Article published by: Leah Rowe in GNU Leah Mode™

Date of publication: 26 October 2023

• <<
• Next >
• >>

Canoeboot provides boot firmware for supported x86/ARM machines, starting a bootloader that then loads your operating system. It replaces proprietary BIOS/UEFI firmware on x86 machines, and provides an improved configuration on ARM-based chromebooks supported (U-Boot bootloader, instead of Google’s depthcharge bootloader). On x86 machines, the GRUB and SeaBIOS coreboot payloads are officially supported, provided in varying configurations per machine. It provides an automated build system for the configuration and installation of coreboot ROM images, making coreboot easier to use for non-technical people. You can find the list of supported hardware in Canoeboot documentation.

Canoeboot’s main benefit is higher boot speed, better security and more customisation options compared to most proprietary firmware. As a libre software project, the code can be audited, and coreboot does regularly audit code. The other main benefit is freedom to study, adapt and share the code, a freedom denied by most boot firmware, but not Canoeboot!

GRUB payloads in Canoeboot have a newer version of libgcrypt which supports argon2 these days, so you can easily use LUKS2.

You can find information about using the build system in the Canoeboot build instructions and in the cbmk maintenance manual.

Changes include things like vastly reduced code complexity (while not sacrificing functionality), greater speed (at compiling, and boot speeds are higher when you use the GRUB payload), many bug fixes and more.

In addition to coreboot firmware, the Canoeboot build system (cbmk) can now build serprog firmware, specifically pico-serprog and stm32-vserprog, on all devices that these projects support.

The serprog protocol is supported by flashrom, to provide SPI flashing. It can be used to set up an external SPI flasher, for flashing Canoeboot externally.

Alper Nebi Yasak submitted patches that update the U-Boot revision to the latest revision as of this release.

In Canoeboot 20231026 (this release):

• Coreboot (default): commit ID d862695f5f432b5c78dada5f16c293a4c3f9fce6, 12 October 2023
• Coreboot (cros): MERGED WITH coreboot/default (see above)
• Coreboot (fam15h_udimm): commit ID 1c13f8d85c7306213cd525308ee8973e5663a3f8, 16 June 2021
• GRUB: commit ID e58b870ff926415e23fc386af41ff81b2f588763, 3 October 2023
• SeaBIOS: commit ID 1e1da7a963007d03a4e0e9a9e0ff17990bb1608d, 24 August 2023
• U-Boot: commit ID 4459ed60cb1e0562bc5b40405e2b4b9bbf766d57, 2 October 2023

The resources/scripts/ directory is now script/, and what was resources/ now only contains configuration data plus code patches for various projects, so it has been renamed to config/ - I considered splitting patches into patch/, but the current directory structure for patches is not a problem so I left it alone.

Also, the IFD/GbE files have been moved here, under config/ifd/. These can always be ge-generated if the user wants to, using ich9gen, or using a combination of bincfg and ifdtool from coreboot, and nvmutil (to change the mac address) from Canoeboot.

Sic

• Most logic has been unified in single scripts that perform once type of task each, instead of multiple scripts performing the same type of talk; for example, defconfig-based projects now handled with the same scripts, and preparing trees for them is done the same. These unifications have been done carefully and incrementally, with great thought so as to prevent spaghetti. The code is clean, and small.
• GitHub is no longer used on main Git repository links, instead only as backup
• Backup repositories now defined, for all main repos under config/git/
• Single-tree projects are no longer needlessly re-downloaded when they already have been downloaded.
• GRUB LUKS2 support now available, with argon2 key derivation; previously, only PBKDF2 worked so most LUKS2 setups were unbootable in Canoeboot. This is fixed.
• Vastly reduced number of modules in GRUB, keeping only what is required.
• Use --mtime and option options in GNU Tar (if it is actually GNU Tar), when creating Tar archives. This results in partially reproducible source archives, and consistent hashes were seen in testing, but not between distros.
• Always re-inialitise .git within cbmk, for the build system itself, if Git history was removed as in releases. This work around some build systems like coreboot that use Git extensively, and are error-prone without it.
• More robust makefile handling in source trees; if one doesn’t exist, error out but also check other makefile name combinations, and only error out if the command was to actually build.
• ROMs build script: support the “all” argument, even when getopt options are used e.g. -k
• Disabled the pager in grub.cfg, because it causes trouble in some non-interactive setups where the user sees an errant message on the screen and has to press enter. This fixes boot interruptions in some cases, allowing normal use of the machine. The pager was initially enabled many years ago, to make use of cat a bit easier in the GRUB shell, but the user can just enable the pager themselves if they really want to.
• U-Boot can now be compiled standalone, without using the ROMs build script, because crossgcc handling is provided for U-Boot now in addition to coreboot.
• All helper scripts are now under include/, and main scripts in script/, called by the main build script
• Generally purge unused variables in shell scripts
• Simplified initialisation of variables in shell scripts, using the setvars function defined under include/err.sh
• Support patch subdirectories, when applying patches. This is done recursively, making it possible to split up patch files into smaller sets inside sub directories, per each source tree (or target of each source tree, where a project is multi-tree within cbmk)
• SPDX license headers now used, almost universally, in all parts of cbmk.
• Files such as those under config/git are now concatenated, traversing recursively through the target directory; files first, then directories in order, and for each directory, follow the same pattern until all files are concatenated. This same logic is also used for patches. This now enables use of subdirectories, in some config/patch directories.
• General code cleanup on util/nvmutil
• Git histories are more thoroughly deleted, in third party source trees during release time.
• Symlinks in release archives are no longer hard copies; the symlinks are re-created by the release script, because it clones the current cbmk work directory via Git (local git clone), rather than just using cp to copy links.
• Properly output to stderr, on printf commands in scripts where it is either a warning prior to calling err, or just something that belongs on the error output (instead of standard output).
• Don’t use the -B option in make commands.
• SECURITY: Use sha512sum (not sha1sum) when verifying certain downloads. This reduces the chance for collisions, during checksum verification.
• Set GRUB timout to 5s by default, but allow override and set to 10s or 15s on some motherboards.
• Support both curl and wget, where files are downloaded outside of Git; defer to Wget when Curl fails, and try each program three times before failing. This results in more resilient downloading, on wobbly internet connections.
• Don’t clone Git repositories into /tmp, because it might be a tmpfs with little memory available; clone into tmp/gitclone instead, within cbmk, and mv it to avoid unnecessary additional writes (mv is much more efficient than cp, for this purpose).
• Coreboot builds: automatically run make-oldconfig, to mitigate use of raw coreboot config where a revision was updated but the config was untouched. This may still result in a confirmation dialog, and it’s still recommended that the configs be updated per revision (or switch them to defconfigs).
• Vastly simplified directory structure; resources/scripts/ is now script/, and resources/ was renamed to config/; ifd and gbe files were also moved to config/ifd/. Commands are now 1-argument instead of 2, for example the ./build boot roms command is now ./build roms.
• memtest86plus: only build it on 64-bit hosts, for now (32-bit building is broken on a lot of distros nowadays, and cbmk doesn’t properly handle cross compilation except on coreboot or U-Boot)
• (courtesy of Riku Viitanen) don’t use cat on loops that handle lines of text. Instead, use the read command that is built into sh, reading each line. This is more efficient, and provides more robust handling on lines with spaces in them.
• ALL projects now have submodules downloaded at build time, not just multi tree projects such as coreboot - and a few projects under config/git have had certain depend items removed, if a given project already defines it under .gitmodules (within its repository).
• Improved cbutils handling; it’s now even less likely to needlessly re-build if it was already built.
• The release build script no longer archives what was already built, but instead builds from scratch, creating an archive from source downloads first before building the ROM archives. This saves time because it enables a single build test per release, whereas at was previously necessary to test the Git repository and then the release archive. Testing both is still desired, but this behaviour also means that whatever is built at release time is guaranteed to be the same as what the user would build (from archives).
• Improved handling of target.cfg files in multi-tree projects coreboot, SeaBIOS and U-Boot. Unified to all such projects, under one script, and with improved error handling.
• GRUB payload: all ROM images now contain the same ELF, with all keymaps inserted. This speeds up the build process, and enables easier configuration when changing the keyboard layout because less re-flashing is needed.
• Running as root is now forbidden, for most commands; cbmk will exit with non-zero status if you try. The ./build dependencies x commands still work as root (they’re the only commands available as root).
• Enabled memtest86plus on more boards, where it wasn’t previously enabled.
• Only enable SeaBIOS as first payload on desktops, but still enable GRUB as second payload where GRUB is known to work (on each given host). The text mode and coreboot framebuffer modes are provided in each case, where feasible.
• The list command has been mostly unified, making it easier to tell (from cbmk) what commands are available, without having to manually poke around under script/.
• The -T0 flag is now used, universally, on xz commands. This makes xz run on multiple threads, greatly speeding up the creation of large tar archives.
• File names as arguments now universally have quotes wrapped around them, and similar auditing has been done to all variables used as arguments everywhere in cbmk. There were cases where multiple arguments were wrongly quoted then treated as a single argument, and vice versa. This is now fixed.
• Re-wrote .gitcheck; now, a global git name/email config is always required. The only behaviour (setting local config, and unsetting) was quite error-prone under fault conditions, where cleanup may not have been provided, or when execution was interrupted, resulting sometimes in accidentally committing to cbmk.git as author named cbmkplaceholder.
• Scripts no longer directly exit with non-zero status, under fault conditions; instead, x_ or err is used to provide such behaviour. This results in all exits from cbmk being consolidated to err, under fault conditions. - zero exits are also consolidated, going only through the main script, which has its own exit function called cbmk_exit that provides TMPDIR cleanup.
• Clean error handling implemented, with an err function (and functions that use it) inside include/err.sh; there is also x_ which can be used to run a command and exit automatically with non-zero status, useful because it provides more verbose output than if you just relied on set -e, and it still works when a script does not use set -e - however, it is not used on all functions, because it works by executing $@ directly, which can break depending on arguments. Therefore, some scripts just default to || err for providing breakage in scripts.
• Memtest 6.2 now used (instead of 5.x releases). This is essentially a re-write, and it works on the coreboot framebuffer, whereas previous revisions only worked on text mode setups.
• NO MAKEFILE. The Makefile in cbmk has been removed. It was never meaningfully used because all it did was run cbmk commands, without implementing any logic itself. A Makefile may be added again in the future, but with a view to installing just the build system onto the host system, to then build ROM images under any number of directories. Cbmk’s design is strictly no-Makefile, but it uses Makefiles provided by third party source trees when building them.
• Safer GRUB configuration file handling between GRUB memdisk and coreboot CBFS; it is no longer possible to boot without a GRUB config, because the one in GRUB memdisk is provided as a failsafe, overridden by inserting one in CBFS, but there is no config in CBFS by default anymore.
• The build system warns users about elf/ vs bin/, when it comes to flashing coreboot ROM images; it tells them to use bin/ because those images do contain payloads, whereas the ones under elf/ do not.
• VASTLY more efficient build process; all coreboot ROMs without payload are now cached under elf/, as are payloads, then they are joined separately by the usual ROMs build script, and these cached ROMs contain many changes in them that were previously handled by moverom in the main ROM build script. Under the new design, repetitive steps are avoided; payloads are inserted into a copy of the cached ROMs under TMPDIR, before being copied for keymaps and small files; this eliminates delays caused by slow compression (LZMA is always used, when inserting payloads). After crossgcc and the payloads are compiled, the ROM with coreboot builds in under a minute, whereas it would have previously taken several minutes on most Canoeboot-supported hardware.
• VASTLY reduced GRUB payload size; modules that aren’t needed have been removed resulting in much smaller GRUB payloads, that also boot faster.
• ALL defconfig creation, updating and modification are handled by the same script that also handles compiling, as mentioned in the bullet-point below.
• ALL main source trees are now compiled, downloaded, configured and cleaned using the same script. The download (Git) logic is a separate file under include/ and its functions are called by the main build script, which provides a stub for this.
• Scripts are no longer executed directly, ever, except the main script. All scripts are otherwise executed from script/, inheriting the TMPDIR variable set (and exported) by cbmk.
• Coreboot, U-Boot and SeaBIOS are now downloaded, configured and compiled using the exact same script. Although these codebases differ wildly, their build systems use the same design, and they are compatible from a user-interface perspective.
• Vastly improved /tmp handling; a universal TMPDIR is set (environmental variable) and exported to all child processes running cbmk scripts. On exit, the main tmp directory is purged, cleaning all tmp directories under it.
• General simplification of coding style on all shell scripts.
• Fixed some variable initialisations in the coreboot ROM image build script
• Don’t enable u-boot on QEMU x86 images (due to buggy builds, untested)
• Fixed coreboot-version file inserted into coreboot trees, when compiled on Canoeboot release archives.
• Very general auditing has been done, finding and fixing bugs.
• Many scripts that were separate are now unified. For example: the scripts handling defconfigs files on SeaBIOS, u-Boot and coreboot have now been merged into a single script, performing the same work better in less code.
• Ditto many other scripts; repeated logic unified, logic generalised. The logic for downloading coreboot and u-boot was unified into one script, basing off of the coreboot one, and then expanding to also cover SeaBIOS. Most building (e.g. handling of Makefiles) is now done in a single script.
• Far superior error handling; in many scripts, the -e option in sh was heavily relied upon to catch errors, but now errors are handled much more verbosely. Many fault conditions previously did not make cbmk exit at all, let alone with non-zero status, and zero status was sometimes being returned under some edge cases that were tested. Error handling is more robust now.
• ROM images (of coreboot) build much faster: no-payload coreboot ROMs are cached on disk, as are payloads, where previously only the latter was cached. These cached images have as much inserted into them as possible, to eliminate redundant steps in the build process. The elf directory contains these, and the existing bin directory still holds the full ROM images (containing payloads) when compiled.
• GRUB payload: vastly reduced the size of the payload, by eliminating GRUB modules that were not needed. About 100KB of compressed space saved in flash!
• GRUB payload: argon2 key derivation supported - this means LUKS2 decryption is now possible in GRUB. This work was performed by Nicholas Johnson, rebasing from Axel’s AUR patch for GRUB 2.06 (Canoeboot currently uses GRUB 2.12).
• The new coding style is now used on many more scripts, including the build/boot/roms_helper script - the new style is much cleaner, mandating that logic be top-down, with a main() function defined.
• All GRUB keymaps now included; a single grub.elf is now used on all ROM images. The grub.cfg goes in GRUB memdisk now, but can be overridden by inserting a grub.cfg in CBFS; many behaviours are also controlled this way, for example to change keymaps and other behaviours. This results in much faster builds, because a different GRUB payload doesn’t have to be added to each new ROM image; such takes time, due to time-expensive LZMA compression. This, plus the optimised set of GRUB modules, also makes GRUB itself load much faster. All of the fat has been trimmed, though still quite a lot more than a Crumb.
• A lot of scripts have been removed entirely, and their logic not replaced; in many cases, Canoeboot’s build system contained logic that had gone unused for many years.
• More reliable configs now used on desktop motherboards: SeaBIOS-only for start, but GRUB still available where feasible (in the SeaBIOS menu). This makes it more fool proof for a user who might use integrated graphics and then switch to a graphics card; the very same images will work.
• TMPDIR environmental variable now set, and exported from main parent process when running cbmk; child processes inherit it, and a single tmp dir is used. This is then automatically cleaned, upon exit from cbmk; previously, cbmk did not cleanly handle /tmp at all, but now it’s pretty reliable.

All of the following are believed to boot, but if you have any issues, please contact the Canoeboot project. They are:

• ASUS KFSN4-DRE motherboard
• ASUS KGPE-D16 motherboard

• Gigabyte GA-G41M-ES2L motherboard
• Acer G43T-AM3
• Intel D510MO and D410PT motherboards
• Apple iMac 5,2
• ASUS KCMA-D8 motherboard

• Dell Latitude E6400 (easy to flash, no disassembly, similar hardware to X200/T400)
• ThinkPad X60 / X60S / X60 Tablet
• ThinkPad T60 (with Intel GPU)
• Lenovo ThinkPad X200 / X200S / X200 Tablet
• Lenovo ThinkPad X301
• Lenovo ThinkPad R400
• Lenovo ThinkPad T400 / T400S
• Lenovo ThinkPad T500
• Lenovo ThinkPad W500
• Lenovo ThinkPad R500
• Apple MacBook1,1 and MacBook2,1

• ASUS Chromebook Flip C101 (gru-bob)
• Samsung Chromebook Plus (v1) (gru-kevin)

You can find this release on the downloads page. At the time of this announcement, some of the rsync mirrors may not have it yet, so please check another one if your favourite one doesn’t have it.

Besides deblobbing, there are two critical differences in how Canoeboot’s build system works in this release.

• Single-tree git submodules are not downloaded in Canoeboot; none of them are used in the Canoeboot release, but using them simplified config/git/ because many of those entries were defined as submodules by each given project; in some serprog-related repositories, proprietary drivers get downloaded that are never actually compiled or executed in any way. Rather than deblob these in Canoeboot, the Canoeboot build system simply skips downloading those repositories altogether.

The Canoeboot build system is about 1250 sloc when counting shell scripts of the build system; considerably smaller than older revisions, accounting for an approximate 50% reduction in the amount of code.

That ~1250 sloc in Canoeboot is with all the extra features such as serprog integration and U-Boot support (on actual motherboards, that you can flash it with). The build system in Canoeboot 20231026 is extremely efficient.

The following binary blobs were overlooked, and are still present in the release archive for Canoeboot 20231101 and 20231026; this mistake was corrected, in the Canoeboot 20231103 release, so you should use that if you don’t want these files. They are, thus:

• src/coreboot/default/3rdparty/stm/Test/FrmPkg/Core/Init/Dmar.h
• src/coreboot/fam15h_rdimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_1gb.absf
• src/coreboot/fam15h_rdimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_2gb.absf
• src/coreboot/fam15h_udimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_1gb.absf
• src/coreboot/fam15h_udimm/src/vendorcode/intel/fsp1_0/baytrail/absf/minnowmax_2gb.absf
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_err.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gap.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gatt.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gattc.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_gatts.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_hci.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_l2cap.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_ranges.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/ble_types.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_error.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_error_sdm.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_error_soc.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_nvic.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_sdm.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_soc.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf_svc.h
• src/pico-sdk/lib/tinyusb/hw/mcu/nordic/nrf5x/s140_nrf52_6.1.1_API/include/nrf52/nrf_mbr.h

Thanks go to Craig Topham, who reported this issue.

The Canoeboot 20231026 and 20231101 release tarballs will not be altered, but errata has now been added to the announcement pages for those releases, to let people know of the above issue.

You are advised, therefore, to use the Canoeboot 20231103 release.

This file was also overlooked, and is still present in the release tarball:

• src/vendorcode/amd/agesa/f12/Proc/GNB/Nb/Family/LN/F12NbSmuFirmware.h

This has now been removed, in the Canoeboot git repository (cbmk.git), and this file will absent, in the next release after Canoeboot 20231107. Thanks go to Denis Carikli who reported this. The patch to fix it is here:

https://codeberg.org/canoeboot/cbmk/commit/70d0dbec733c5552f8cd6fb711809935c8f3d2f3

SPDX-License-Identifier: GFDL-1.3-or-later

Markdown file for this page: https://canoeboot.org/news/canoeboot20231026.md

Site map

This HTML page was generated by the Canoeboot Static Site Generator.