| 1 | |
| 2 |
|
| 3 | ==================================
|
| 4 | Device Initialization (devinit)
|
| 5 | ==================================
|
| 6 | The devinit process is complex and subject to change. This document provides a high-level
|
| 7 | overview using the Ampere GPU family as an example. The goal is to provide a conceptual
|
| 8 | overview of the process to aid in understanding the corresponding kernel code.
|
| 9 |
|
| 10 | Device initialization (devinit) is a crucial sequence of register read/write operations
|
| 11 | that occur after a GPU reset. The devinit sequence is essential for properly configuring
|
| 12 | the GPU hardware before it can be used.
|
| 13 |
|
| 14 | The devinit engine is an interpreter program that typically runs on the PMU (Power Management
|
| 15 | Unit) microcontroller of the GPU. This interpreter executes a "script" of initialization
|
| 16 | commands. The devinit engine itself is part of the VBIOS ROM in the same ROM image as the
|
| 17 | FWSEC (Firmware Security) image (see fwsec.rst and vbios.rst) and it runs before the
|
| 18 | nova-core driver is even loaded. On an Ampere GPU, the devinit ucode is separate from the
|
| 19 | FWSEC ucode. It is launched by FWSEC, which runs on the GSP in 'heavy-secure' mode, while
|
| 20 | devinit runs on the PMU in 'light-secure' mode.
|
| 21 |
|
| 22 | Key Functions of devinit
|
| 23 | ------------------------
|
| 24 | devinit performs several critical tasks:
|
| 25 |
|
| 26 | 1. Programming VRAM memory controller timings
|
| 27 | 2. Power sequencing
|
| 28 | 3. Clock and PLL (Phase-Locked Loop) configuration
|
| 29 | 4. Thermal management
|
| 30 |
|
| 31 | Low-level Firmware Initialization Flow
|
| 32 | --------------------------------------
|
| 33 | Upon reset, several microcontrollers on the GPU (such as PMU, SEC2, GSP, etc.) run GPU
|
| 34 | firmware (gfw) code to set up the GPU and its core parameters. Most of the GPU is
|
| 35 | considered unusable until this initialization process completes.
|
| 36 |
|
| 37 | These low-level GPU firmware components are typically:
|
| 38 |
|
| 39 | 1. Located in the VBIOS ROM in the same ROM partition (see vbios.rst and fwsec.rst).
|
| 40 | 2. Executed in sequence on different microcontrollers:
|
| 41 |
|
| 42 | - The devinit engine typically but not necessarily runs on the PMU.
|
| 43 | - On an Ampere GPU, the FWSEC typically runs on the GSP (GPU System Processor) in
|
| 44 | heavy-secure mode.
|
| 45 |
|
| 46 | Before the driver can proceed with further initialization, it must wait for a signal
|
| 47 | indicating that core initialization is complete (known as GFW_BOOT). This signal is
|
| 48 | asserted by the FWSEC running on the GSP in heavy-secure mode.
|
| 49 |
|
| 50 | Runtime Considerations
|
| 51 | ----------------------
|
| 52 | It's important to note that the devinit sequence also needs to run during suspend/resume
|
| 53 | operations at runtime, not just during initial boot, as it is critical to power management.
|
| 54 |
|
| 55 | Security and Access Control
|
| 56 | ---------------------------
|
| 57 | The initialization process involves careful privilege management. For example, before
|
| 58 | accessing certain completion status registers, the driver must check privilege level
|
| 59 | masks. Some registers are only accessible after secure firmware (FWSEC) lowers the
|
| 60 | privilege level to allow CPU (LS/low-secure) access. This is the case, for example,
|
| 61 | when receiving the GFW_BOOT signal.
|