Toward the unification of kselftests and KUnit
The kernel project, for many years, lacked a formal testing setup; it was often joked that testing was the project's main reason for keeping users around.
长期以来,内核项目一直缺乏正式的测试体系;人们常开玩笑说,保持用户留存的主要原因就是测试本身。
While many types of kernel testing can only be done in the presence of specific hardware, there are other parts of the kernel that could be more widely tested.
虽然许多类型的内核测试只能在特定硬件环境下进行,但内核的其他部分是可以更广泛进行测试的。
Over time, though, the kernel has gained two separate testing frameworks and a growing body of automated tests to go with them. These two frameworks — kselftests and KUnit — take different approaches to the testing problem; now this patch series from Thomas Weißschuh aims to bring them together.
不过,随着时间推移,内核逐渐引入了两个独立的测试框架,并配套建立了日益丰富的自动化测试体系。这两个框架是 kselftests 和 KUnit,它们对测试问题采取了不同的方法;而现在 Thomas Weißschuh 的这一系列补丁则旨在将它们融合在一起。
Kselftests and KUnit
Kselftests 与 KUnit
Kselftests was first added by Frederic Weisbecker in 2012, with the first test being focused on the handling of breakpoints on the x86 architecture. These self tests run in user space, exercising the normal kernel system-call interface. Over the years, kselftests has grown a test-harness structure based around the Test Anything Protocol (TAP) and a set of functions, macros, and makefile support for the creation of tests.
kselftests 由 Frederic Weisbecker 于 2012 年首次引入,第一个测试聚焦于 x86 架构下断点的处理。这些自测在用户态,调用普通的内核系统调用接口。多年发展以来,kselftests 已建立起基于 Test Anything Protocol(TAP)的测试框架,并配套了一整套函数、宏和 makefile 支持,以便编写测试用例。
In current kernels, the kselftests directory has over 100 subdirectories, each containing tests for a specific subsystem. There are tests focused on system calls, architecture support, sysctl knobs, kernel behavior (such as the sealing of system mappings), /proc files, and a handful of device drivers, among other things. While this set of tests is not (and probably can never be) complete, a successful kselftests run is enough to give confidence that significant parts of the kernel are working as expected.
在当前的内核版本中,kselftests 目录下已有 100 多个子目录,每个目录包含某个子系统的测试内容。测试主题包括系统调用、架构支持、sysctl 选项、内核行为(例如系统映射的封闭)、/proc 文件,以及少量设备驱动等内容。虽然这些测试集合并不(也可能永远不会)完整,但一次成功的 kselftests 足以让人有信心:内核的关键部分正在按预期工作。
KUnit is a different beast; its tests are built as kernel modules and run within the kernel itself. That gives KUnit tests the ability to verify the operation of individual kernel functions that are not reachable from user space. KUnit tests can be loaded into a running kernel and run at any time; they can also be built into the kernel and run automatically at every boot. KUnit, too, provides a set of supporting functions to make tests easier; it is built around a version of TAP called KTAP.
KUnit 则完全不同:它将测试构建为内核模块,并在内核内部。因此,KUnit 能够测试那些用户态无法触及的内核函数。KUnit 测试可以被加载到正在的内核中随时执行,也可以内置于内核,在每次启动时自动。KUnit 同样提供了一系列辅助函数以简化测试过程;它以名为 KTAP 的 TAP 变体为基础构建。
KUnit was added in 2019 by Felix Guo. It, too, has accumulated a growing set of tests over the years; the DRM (graphics) subsystem has quite a few tests, and there is a significant set of tests covering the generic support routines found in the kernel's lib/ directory. See, for example, this set of hash-table tests.
KUnit 于 2019 年由 Felix Guo 添加。近年来,其测试套件也在不断扩充;DRM(图形)子系统拥有相当多的测试,同时还包括一套涵盖内核 lib/ 目录中通用支持例程的测试。例如,用于哈希表的那套测试便是其中之一。
The kernel's two test suites are aimed at different aspects of the testing problem, so it is not entirely surprising that they ended up with different designs. Kselftests pokes at the kernel from the outside, ensuring that its user-space ABI behaves as expected, while KUnit works on the inside, testing components that are not reachable from user space. This separation has allowed each test suite to grow within its targeted space, but it has also led to some occasional frustrations.
内核的这两套测试框架各自针对不同的测试方面,因此它们拥有不同的设计也就不足为奇。kselftests 从内核“外部”触及,确保用户态 ABI 的行为符合预期,而 KUnit 则“内部”,测试那些用户态无法访问的组件。这样的分工允许各自体系在其领域内成长,但也带来了一些不便和挫折。
Perhaps the twain shall meet
或许,两种方式终会汇合
Weißschuh's work appears to be driven by some specific needs felt by embedded developers. The KUnit tests are relatively easy to run on a new embedded system; they can be built into the kernel that is run on those systems, and require no extra support there. The kselftests tests, instead, need a fully working user space on the target system to run. Both that user space and the tests themselves must be loaded onto the target separately from the kernel, making the testing task harder in general.
Weißschuh 的工作似乎是由嵌入式开发者的一些特定需求驱动的。在新的嵌入式系统上 KUnit 测试相对容易:它们可以编译进在该系统上的内核,无需额外支持。相比之下,kselftests 则要求目标系统拥有完备的用户态环境,且测试程序需脱离内核单独加载到目标系统上,因此其测试流程更加复杂。
The solution that Weißschuh is pursuing is to integrate the two test suites and, in particular, to make it possible to build the kselftests into the kernel and run them from the KUnit framework. A successful solution would allow a kernel and a full set of tests to be loaded onto a target system as a single binary, easing the process of getting the kernel into a fully working state on a new system. There are, naturally, a few obstacles that need to be overcome to get there.
Weißschuh 正在寻求的解决方案是整合两套测试体系,尤其是实现将 kselftests 构建进内核,并通过 KUnit 框架。如果成功,将可以将内核与完整测试集打包成单一二进制加载到目标系统,从而简化新系统上内核达到可用状态的流程。当然,这个过程中还存在一些需要克服的障碍。
Kselftests are designed to be run as standalone programs, not as functions within kernel modules like KUnit tests. Weißschuh's series continues to build those tests separately, but the resulting binaries are linked into the new kunit-uapi module, which then runs them in a separate kernel thread. This code is based on the user-mode-helper functionality that was first introduced as part of bpfilter, though a number of changes needed to be made.
kselftests 设计为独立程序,而不是像 KUnit 那样作为内核模块里的函数来调用。Weißschuh 的补丁系列保留了这一编译方式,将生成的二进制文件链接到新模块 kunit‑uapi 中,并在内核的独立线程里它们。这段代码借用了最早随 bpfilter 引入的 user‑mode‑helper 功能,但也进行了不少改动。
A user-space program expects to have a C library available to it; that is part of the user-space setup that Weißschuh is trying to avoid having to do. Fortunately, since the 5.1 release, the kernel happens to have its own minimal C-library implementation in the form of nolibc. It is not a complete implementation, but it contains enough support to run much of the kselftest suite. The building of those tests, though, had to be modified to use nolibc rather than the system's C library.
用户空间的程序通常依赖 C 库,但 Weißschuh 正试图避开为此准备完整用户空间环境。幸运的是,自 5.1 版本以来,内核附带了一个名为 nolibc 的简易 C 库实现。虽然功能不完整,但足以支持大部分 kselftests。测试构建流程也因此被修改为使用 nolibc 而不是系统 C 库。
The kunit-uapi module has to do the rest of the work of creating a sufficient environment for each kselftest, as well as actually running the tests. That requires access to functionality that the kernel does not currently export to modules — kernel functions like kernel_execve(), replace_fd(), create_pipe_files(), and do_exit(). The series includes a patch exporting those symbols, among others. The export is limited to the kunit-uapi module using the new EXPORT_SYMBOL_GPL_FOR_MODULES() macro that was just added for 6.16; it is the current form of the restricted-namespace feature that was first proposed in 2024.
kunit‑uapi 模块还需完成为每个 kselftest 创建足够环境的其他工作,并真正执行测试。但这需要调用当前未向模块导出的内核函数,如 kernel_execve()、replace_fd()、create_pipe_files() 和 do_exit()。为此,补丁系列中新增了对这些符号的导出支持,且仅限于 kunit‑uapi 模块使用,采用的是 6.16 版本新增的 EXPORT_SYMBOL_GPL_FOR_MODULES() 宏形式,这是 2024 年首次提出的受限命名空间功能的当前实现方式。
The newly exported functions are used to set up the standard input, output, and error streams for each test (the input is, for all practical purposes, set to /dev/null), mount /proc, run the test itself, and clean up afterward. The TAP output from the tests is passed back into the KUnit framework to be reported with the rest of the test results.
新导出的函数用于为每个测试配置标准输入/输出/错误流(输入基本设置为 /dev/null)、挂载 /proc、执行测试主体,并完成清理。测试执行过程输出的 TAP 格式信息将被回传到 KUnit 框架,以便与其他测试结果一并报告。
All of this work sets the stage for packaging the existing kselftests, but stops short of that goal. Instead, the only tests enabled at the end of the series are a simple example test and a test verifying the /proc mount. Bringing the existing tests in will require adding a bit of glue for each, causing it to be embedded in the loadable module and run at the right time. A more automated way of incorporating the tests is on the wishlist for the future, but does not exist now.
以上所有工作为打包现有 kselftests 奠定了基础,但尚未完全达成此目标。系列补丁目前只启用了一个简单示例测试和一个 /proc 挂载验证测试。要将现有测试接入,需要为每个测试添加“粘合”代码,使其能够嵌入模块并在合适时间。未来的计划表中希望实现测试接入的自动化,但目前尚未实现。
The incorporation of the actual tests may be waiting for a consensus on the surrounding framework. As of this writing, the series is in its fourth revision, and the most significant concerns would appear to have been addressed; the most recent comments are mostly focused on relatively small issues. Assuming that the biggest problems nearly been overcome, the core framework may find its way into the mainline relatively quickly; the job of integrating all of the actual tests will likely be next.
实际测试接入可能还需要围绕框架达成共识。截至目前,该补丁系列已迭代到第四版,重大问题似乎已基本解决;最近的讨论主要集中在一些细节上。如果最大的障碍已经清除,那么核心框架有望尽快合并到主线;接下来可能就是接入所有实际测试的工作了。

