diff --git a/Documentation/dev-tools/kunit/faq.rst b/Documentation/dev-tools/kunit/faq.rst index fae426f..b1341c1 100644 --- a/Documentation/dev-tools/kunit/faq.rst +++ b/Documentation/dev-tools/kunit/faq.rst @@ -25,19 +25,21 @@ disqualifying any of them from being considered unit testing frameworks. Does KUnit support running on architectures other than UML? =========================================================== -Yes, mostly. +Yes. KUnit can run on any architecture, though the kunit.py tool can only +build and run kernels for some architectures (of which UML is the default). -For the most part, the KUnit core framework (what we use to write the tests) -can compile to any architecture. It compiles like just another part of the -kernel and runs when the kernel boots, or when built as a module, when the -module is loaded. However, there is infrastructure, like the KUnit Wrapper -(``tools/testing/kunit/kunit.py``) that might not support some architectures -(see :ref:`kunit-on-qemu`). +You can build and run tests without kunit.py at all on any architecture by +enabling ``CONFIG_KUNIT=y`` and booting the kernel. +See Documentation/dev-tools/kunit/run_manual.rst for more details. -In short, yes, you can run KUnit on other architectures, but it might require -more work than using KUnit on UML. +Alternatively, kunit.py supports many common architectures using +cross-compilers and the qemu emulator. This can be done using the ``--arch`` +parameter when running the tests, and the ``--cross_compile`` parameter +when building (if the architecture is not supported by the host compiler). +See :ref:`kunit-on-qemu` for more details. -For more information, see :ref:`kunit-on-non-uml`. +When writing tests targeting other architectures, it's worth keeping the tips +on the :ref:`kunit-on-non-uml` page in mind. .. _kinds-of-tests: @@ -78,27 +80,30 @@ things to try. down where an issue is occurring. (If you think the parser is at fault, you can run it manually against ``stdin`` or a file with ``kunit.py parse``.) 3. Running the UML kernel directly can often reveal issues or error messages, - ``kunit_tool`` ignores. This should be as simple as running ``./vmlinux`` - after building the UML kernel (for example, by using ``kunit.py build``). + ``kunit_tool`` ignores. This should be as simple as runningi the ``vmlinux`` + binary in the output directory (by default ``./.kunit/vmlinux``) after + building the UML kernel (for example, by using ``kunit.py build``). Note that UML has some unusual requirements (such as the host having a tmpfs filesystem mounted), and has had issues in the past when built statically and the host has KASLR enabled. (On older host kernels, you may need to run ``setarch `uname -m` -R ./vmlinux`` to disable KASLR.) -4. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test +4. Try running KUnit on a different architecture by using the ``--arch`` + option. On an x86_64 host, using ``--arch=x86_64`` is a good first step. +5. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config around, so you can see what config was used after running ``kunit.py run``. It also preserves any config changes you might make, so you can enable/disable things with ``make ARCH=um menuconfig`` or similar, and then re-run kunit_tool. -5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This +6. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This may help clean up any residual config items which could be causing problems. -6. Finally, try running KUnit outside UML. KUnit and KUnit tests can be - built into any kernel, or can be built as a module and loaded at runtime. - Doing so should allow you to determine if UML is causing the issue you're - seeing. When tests are built-in, they will execute when the kernel boots, and +7. Finally, try running KUnit manually, instead of via ``kunit.py``. KUnit can + be built into any kernel, or can be built as a module and loaded at runtime. + When tests are built-in, they will execute when the kernel boots, and modules will automatically execute associated tests when loaded. Test results can be collected from ``/sys/kernel/debug/kunit//results``, and - can be parsed with ``kunit.py parse``. For more details, see :ref:`kunit-on-qemu`. + can be parsed with ``kunit.py parse``. For more details, see + Documentation/dev-tools/kunit/run_manual.rst If none of the above tricks help, you are always welcome to email any issues to kunit-dev@googlegroups.com. diff --git a/Documentation/dev-tools/kunit/style.rst b/Documentation/dev-tools/kunit/style.rst index eac81a7..449f9f8 100644 --- a/Documentation/dev-tools/kunit/style.rst +++ b/Documentation/dev-tools/kunit/style.rst @@ -164,9 +164,10 @@ This Kconfig entry must: * be visible only if ``CONFIG_KUNIT_ALL_TESTS`` is not enabled. * have a default value of ``CONFIG_KUNIT_ALL_TESTS``. * have a brief description of KUnit in the help text. - -If we are not able to meet above conditions (for example, the test is unable to -be built as a module), Kconfig entries for tests should be tristate. +* depend on the feature being tested, rather than selecting it (so that + enabling ``CONFIG_KUNIT_ALL_TESTS`` does not enable unrelated functionality). +* be ``tristate``, unless there is a specific reason that the test cannot be + built as a module. For example, a Kconfig entry might look like: diff --git a/include/kunit/test.h b/include/kunit/test.h index e52452e..da5312e 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -285,6 +285,7 @@ struct kunit_suite { struct string_stream *log; int suite_init_err; bool is_init; + enum kunit_status status; }; /* Stores an array of suites, end points one past the end */ diff --git a/lib/kunit/debugfs.c b/lib/kunit/debugfs.c index 9c326f1..442b2ce 100644 --- a/lib/kunit/debugfs.c +++ b/lib/kunit/debugfs.c @@ -76,18 +76,30 @@ static int debugfs_print_results(struct seq_file *seq, void *v) seq_puts(seq, "KTAP version 1\n"); seq_puts(seq, "1..1\n"); - /* Print suite header because it is not stored in the test logs. */ - seq_puts(seq, KUNIT_SUBTEST_INDENT "KTAP version 1\n"); - seq_printf(seq, KUNIT_SUBTEST_INDENT "# Subtest: %s\n", suite->name); - seq_printf(seq, KUNIT_SUBTEST_INDENT "1..%zd\n", kunit_suite_num_test_cases(suite)); - - kunit_suite_for_each_test_case(suite, test_case) - debugfs_print_result(seq, test_case->log); + if (suite->status != KUNIT_SKIPPED) { + /* Print suite header because it is not stored in the test logs. */ + seq_puts(seq, + KUNIT_SUBTEST_INDENT "KTAP version 1\n"); + seq_printf(seq, + KUNIT_SUBTEST_INDENT "# Subtest: %s\n", + suite->name); + seq_printf(seq, + KUNIT_SUBTEST_INDENT "1..%zd\n", + kunit_suite_num_test_cases(suite)); + + kunit_suite_for_each_test_case(suite, test_case) + debugfs_print_result(seq, test_case->log); + } debugfs_print_result(seq, suite->log); - seq_printf(seq, "%s %d %s\n", - kunit_status_to_ok_not_ok(success), 1, suite->name); + if (suite->status != KUNIT_SKIPPED) + seq_printf(seq, "%s %d %s\n", + kunit_status_to_ok_not_ok(success), 1, suite->name); + else + seq_printf(seq, "%s %d %s # SKIP %s\n", + kunit_status_to_ok_not_ok(success), 1, suite->name, + suite->status_comment); return 0; } diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c index 0bae7b7..b8ded54 100644 --- a/lib/kunit/kunit-example-test.c +++ b/lib/kunit/kunit-example-test.c @@ -591,5 +591,34 @@ static struct kunit_suite example_init_test_suite = { */ kunit_test_init_section_suites(&example_init_test_suite); +/* + * This test should always be skipped. + */ +static void example_skip_suite_test(struct kunit *test) +{ + /* This line should never be seen */ + KUNIT_FAIL(test, "You should not see a this."); +} + +static struct kunit_case example_skip_suite_test_cases[] = { + KUNIT_CASE(example_skip_suite_test), + {} +}; + +static int example_skip_suite_init(struct kunit_suite *suite) +{ + kunit_mark_skipped(suite, "Test suite expected to be skipped"); + return 0; +} + +static struct kunit_suite example_test_skip_suite = { + .name = "example_skip_suite", + .suite_init = example_skip_suite_init, + .test_cases = example_skip_suite_test_cases, +}; + +/* This registers a test suite that will be skipped */ +kunit_test_suite(example_test_skip_suite); + MODULE_DESCRIPTION("Example KUnit test suite"); MODULE_LICENSE("GPL v2"); diff --git a/lib/kunit/string-stream.c b/lib/kunit/string-stream.c index 0d8f1b3..51ba40e 100644 --- a/lib/kunit/string-stream.c +++ b/lib/kunit/string-stream.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include "string-stream.h" @@ -74,7 +75,8 @@ int string_stream_vadd(struct string_stream *stream, /* Append newline if necessary. */ if (frag_container->fragment[result_len - 1] != '\n') - result_len = strlcat(frag_container->fragment, "\n", buf_len); + result_len += strscpy(frag_container->fragment + result_len, + "\n", buf_len - result_len); } else { result_len = vsnprintf(frag_container->fragment, buf_len, fmt, args); } @@ -118,15 +120,18 @@ char *string_stream_get_string(struct string_stream *stream) { struct string_stream_fragment *frag_container; size_t buf_len = stream->length + 1; /* +1 for null byte. */ + struct seq_buf sb; char *buf; buf = kzalloc(buf_len, stream->gfp); if (!buf) return NULL; + seq_buf_init(&sb, buf, buf_len); + spin_lock(&stream->lock); list_for_each_entry(frag_container, &stream->fragments, node) - strlcat(buf, frag_container->fragment, buf_len); + seq_buf_puts(&sb, frag_container->fragment); spin_unlock(&stream->lock); return buf; diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 99773e0..09e3dab 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -214,12 +214,18 @@ enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite) const struct kunit_case *test_case; enum kunit_status status = KUNIT_SKIPPED; + if (suite->status == KUNIT_SKIPPED) + return KUNIT_SKIPPED; + if (suite->suite_init_err) return KUNIT_FAILURE; kunit_suite_for_each_test_case(suite, test_case) { - if (test_case->status == KUNIT_FAILURE) + if (test_case->status == KUNIT_FAILURE) { + /* Update the kunit_suite status also */ + suite->status = KUNIT_FAILURE; return KUNIT_FAILURE; + } else if (test_case->status == KUNIT_SUCCESS) status = KUNIT_SUCCESS; } @@ -795,12 +801,20 @@ int kunit_run_tests(struct kunit_suite *suite) /* Taint the kernel so we know we've run tests. */ add_taint(TAINT_TEST, LOCKDEP_STILL_OK); + if (suite->status == KUNIT_SKIPPED) + goto suite_end; + if (suite->suite_init) { suite->suite_init_err = suite->suite_init(suite); if (suite->suite_init_err) { + suite->status = KUNIT_FAILURE; kunit_err(suite, KUNIT_SUBTEST_INDENT "# failed to initialize (%d)", suite->suite_init_err); goto suite_end; + + } else if (suite->status == KUNIT_SKIPPED) { + /* Skip this kunit suite */ + goto suite_end; } } @@ -825,6 +839,7 @@ static void kunit_init_suite(struct kunit_suite *suite) kunit_debugfs_create_suite(suite); suite->status_comment[0] = '\0'; suite->suite_init_err = 0; + suite->status = KUNIT_SUCCESS; if (suite->log) string_stream_clear(suite->log); diff --git a/rust/kernel/kunit.rs b/rust/kernel/kunit.rs index cdee5f2..91eaff8 100644 --- a/rust/kernel/kunit.rs +++ b/rust/kernel/kunit.rs @@ -288,6 +288,7 @@ macro_rules! kunit_unsafe_test_suite { log: ::core::ptr::null_mut(), suite_init_err: 0, is_init: false, + status: kernel::bindings::kunit_status_KUNIT_SUCCESS, }; #[used(compiler)] diff --git a/tools/testing/kunit/configs/all_tests.config b/tools/testing/kunit/configs/all_tests.config index bccc2c7..6825c2e 100644 --- a/tools/testing/kunit/configs/all_tests.config +++ b/tools/testing/kunit/configs/all_tests.config @@ -21,6 +21,7 @@ CONFIG_VFAT_FS=y CONFIG_PCI=y CONFIG_USB4=y CONFIG_I2C=y +CONFIG_GPIOLIB=y CONFIG_NET=y CONFIG_MCTP=y diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index ac3f715..91d234a 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -126,7 +126,7 @@ def _list_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest) lines.pop() # Filter out any extraneous non-test output that might have gotten mixed in. - return [l for l in output if re.match(r'^[^\s.]+\.[^\s.]+$', l)] + return [l for l in lines if re.match(r'^[^\s.]+\.[^\s.]+$', l)] def _list_tests_attr(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest) -> Iterable[str]: args = ['kunit.action=list_attr'] diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index da88c3a..85ae217 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -979,6 +979,18 @@ class KUnitMainTest(unittest.TestCase): self.linux_source_mock.run_kernel.assert_called_once_with( args=['kunit.action=list'], build_dir='.kunit', filter_glob='suite*', filter='', filter_action=None, timeout=300) + def test_list_tests_with_prefix(self): + want = ['suite.test1', 'suite.test2', 'suite2.test1'] + self.linux_source_mock.run_kernel.return_value = [ + '[ 0.100000] TAP version 14', + '[ 0.200000] suite.test1', + '[ 0.200000] suite.test2', + '[ 0.300000] suite2.test1'] + + got = kunit._list_tests(self.linux_source_mock, + kunit.KunitExecRequest(None, None, None, False, False, '.kunit', 300, 'suite*', '', None, None, 'suite', False, False, False)) + self.assertEqual(got, want) + @mock.patch.object(kunit, '_list_tests') def test_run_isolated_by_suite(self, mock_tests): mock_tests.return_value = ['suite.test1', 'suite.test2', 'suite2.test1']