| 1 | |
| 2 |
|
| 3 | ==========================
|
| 4 | Frequently Asked Questions
|
| 5 | ==========================
|
| 6 |
|
| 7 | How is this different from Autotest, kselftest, and so on?
|
| 8 | ==========================================================
|
| 9 | KUnit is a unit testing framework. Autotest, kselftest (and some others) are
|
| 10 | not.
|
| 11 |
|
| 12 | A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is supposed to
|
| 13 | test a single unit of code in isolation and hence the name *unit test*. A unit
|
| 14 | test should be the finest granularity of testing and should allow all possible
|
| 15 | code paths to be tested in the code under test. This is only possible if the
|
| 16 | code under test is small and does not have any external dependencies outside of
|
| 17 | the test's control like hardware.
|
| 18 |
|
| 19 | There are no testing frameworks currently available for the kernel that do not
|
| 20 | require installing the kernel on a test machine or in a virtual machine. All
|
| 21 | testing frameworks require tests to be written in userspace and run on the
|
| 22 | kernel under test. This is true for Autotest, kselftest, and some others,
|
| 23 | disqualifying any of them from being considered unit testing frameworks.
|
| 24 |
|
| 25 | Does KUnit support running on architectures other than UML?
|
| 26 | ===========================================================
|
| 27 |
|
| 28 | Yes. KUnit can run on any architecture, though the kunit.py tool can only
|
| 29 | build and run kernels for some architectures (of which UML is the default).
|
| 30 |
|
| 31 | You can build and run tests without kunit.py at all on any architecture by
|
| 32 | enabling ``CONFIG_KUNIT=y`` and booting the kernel.
|
| 33 | See Documentation/dev-tools/kunit/run_manual.rst for more details.
|
| 34 |
|
| 35 | Alternatively, kunit.py supports many common architectures using
|
| 36 | cross-compilers and the qemu emulator. This can be done using the ``--arch``
|
| 37 | parameter when running the tests, and the ``--cross_compile`` parameter
|
| 38 | when building (if the architecture is not supported by the host compiler).
|
| 39 | See :ref:`kunit-on-qemu` for more details.
|
| 40 |
|
| 41 | When writing tests targeting other architectures, it's worth keeping the tips
|
| 42 | on the :ref:`kunit-on-non-uml` page in mind.
|
| 43 |
|
| 44 | |
| 45 |
|
| 46 | What is the difference between a unit test and other kinds of tests?
|
| 47 | ====================================================================
|
| 48 | Most existing tests for the Linux kernel would be categorized as an integration
|
| 49 | test, or an end-to-end test.
|
| 50 |
|
| 51 | - A unit test is supposed to test a single unit of code in isolation. A unit
|
| 52 | test should be the finest granularity of testing and, as such, allows all
|
| 53 | possible code paths to be tested in the code under test. This is only possible
|
| 54 | if the code under test is small and does not have any external dependencies
|
| 55 | outside of the test's control like hardware.
|
| 56 | - An integration test tests the interaction between a minimal set of components,
|
| 57 | usually just two or three. For example, someone might write an integration
|
| 58 | test to test the interaction between a driver and a piece of hardware, or to
|
| 59 | test the interaction between the userspace libraries the kernel provides and
|
| 60 | the kernel itself. However, one of these tests would probably not test the
|
| 61 | entire kernel along with hardware interactions and interactions with the
|
| 62 | userspace.
|
| 63 | - An end-to-end test usually tests the entire system from the perspective of the
|
| 64 | code under test. For example, someone might write an end-to-end test for the
|
| 65 | kernel by installing a production configuration of the kernel on production
|
| 66 | hardware with a production userspace and then trying to exercise some behavior
|
| 67 | that depends on interactions between the hardware, the kernel, and userspace.
|
| 68 |
|
| 69 | KUnit is not working, what should I do?
|
| 70 | =======================================
|
| 71 |
|
| 72 | Unfortunately, there are a number of things which can break, but here are some
|
| 73 | things to try.
|
| 74 |
|
| 75 | 1. Run ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output``
|
| 76 | parameter. This might show details or error messages hidden by the kunit_tool
|
| 77 | parser.
|
| 78 | 2. Instead of running ``kunit.py run``, try running ``kunit.py config``,
|
| 79 | ``kunit.py build``, and ``kunit.py exec`` independently. This can help track
|
| 80 | down where an issue is occurring. (If you think the parser is at fault, you
|
| 81 | can run it manually against ``stdin`` or a file with ``kunit.py parse``.)
|
| 82 | 3. Running the UML kernel directly can often reveal issues or error messages,
|
| 83 | ``kunit_tool`` ignores. This should be as simple as runningi the ``vmlinux``
|
| 84 | binary in the output directory (by default ``./.kunit/vmlinux``) after
|
| 85 | building the UML kernel (for example, by using ``kunit.py build``).
|
| 86 | Note that UML has some unusual requirements (such as the host having a tmpfs
|
| 87 | filesystem mounted), and has had issues in the past when built statically and
|
| 88 | the host has KASLR enabled. (On older host kernels, you may need to run
|
| 89 | ``setarch `uname -m` -R ./vmlinux`` to disable KASLR.)
|
| 90 | 4. Try running KUnit on a different architecture by using the ``--arch``
|
| 91 | option. On an x86_64 host, using ``--arch=x86_64`` is a good first step.
|
| 92 | 5. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test
|
| 93 | (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config
|
| 94 | around, so you can see what config was used after running ``kunit.py run``.
|
| 95 | It also preserves any config changes you might make, so you can
|
| 96 | enable/disable things with ``make ARCH=um menuconfig`` or similar, and then
|
| 97 | re-run kunit_tool.
|
| 98 | 6. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This
|
| 99 | may help clean up any residual config items which could be causing problems.
|
| 100 | 7. Finally, try running KUnit manually, instead of via ``kunit.py``. KUnit can
|
| 101 | be built into any kernel, or can be built as a module and loaded at runtime.
|
| 102 | When tests are built-in, they will execute when the kernel boots, and
|
| 103 | modules will automatically execute associated tests when loaded. Test results
|
| 104 | can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and
|
| 105 | can be parsed with ``kunit.py parse``. For more details, see
|
| 106 | Documentation/dev-tools/kunit/run_manual.rst
|
| 107 |
|
| 108 | If none of the above tricks help, you are always welcome to email any issues to
|
| 109 | [email protected].
|