projects/linux·blob master · refs

244 lines · 10.1 KiB

1============================
2 Core Driver Infrastructure
3============================
4
5GPU Hardware Structure
6======================
7
8Each ASIC is a collection of hardware blocks. We refer to them as
9"IPs" (Intellectual Property blocks). Each IP encapsulates certain
10functionality. IPs are versioned and can also be mixed and matched.
11E.g., you might have two different ASICs that both have System DMA (SDMA) 5.x IPs.
12The driver is arranged by IPs. There are driver components to handle
13the initialization and operation of each IP. There are also a bunch
14of smaller IPs that don't really need much if any driver interaction.
15Those end up getting lumped into the common stuff in the soc files.
16The soc files (e.g., vi.c, soc15.c nv.c) contain code for aspects of
17the SoC itself rather than specific IPs. E.g., things like GPU resets
18and register access functions are SoC dependent.
19
20An APU contains more than just CPU and GPU, it also contains all of
21the platform stuff (audio, usb, gpio, etc.). Also, a lot of
22components are shared between the CPU, platform, and the GPU (e.g.,
23SMU, PSP, etc.). Specific components (CPU, GPU, etc.) usually have
24their interface to interact with those common components. For things
25like S0i3 there is a ton of coordination required across all the
26components, but that is probably a bit beyond the scope of this
27section.
28
29With respect to the GPU, we have the following major IPs:
30
31GMC (Graphics Memory Controller)
32 This was a dedicated IP on older pre-vega chips, but has since
33 become somewhat decentralized on vega and newer chips. They now
34 have dedicated memory hubs for specific IPs or groups of IPs. We
35 still treat it as a single component in the driver however since
36 the programming model is still pretty similar. This is how the
37 different IPs on the GPU get the memory (VRAM or system memory).
38 It also provides the support for per process GPU virtual address
39 spaces.
40
41IH (Interrupt Handler)
42 This is the interrupt controller on the GPU. All of the IPs feed
43 their interrupts into this IP and it aggregates them into a set of
44 ring buffers that the driver can parse to handle interrupts from
45 different IPs.
46
47PSP (Platform Security Processor)
48 This handles security policy for the SoC and executes trusted
49 applications, and validates and loads firmwares for other blocks.
50
51SMU (System Management Unit)
52 This is the power management microcontroller. It manages the entire
53 SoC. The driver interacts with it to control power management
54 features like clocks, voltages, power rails, etc.
55
56DCN (Display Controller Next)
57 This is the display controller. It handles the display hardware.
58 It is described in more details in :ref:`Display Core <amdgpu-display-core>`.
59
60SDMA (System DMA)
61 This is a multi-purpose DMA engine. The kernel driver uses it for
62 various things including paging and GPU page table updates. It's also
63 exposed to userspace for use by user mode drivers (OpenGL, Vulkan,
64 etc.)
65
66GC (Graphics and Compute)
67 This is the graphics and compute engine, i.e., the block that
68 encompasses the 3D pipeline and shader blocks. This is by far the
69 largest block on the GPU. The 3D pipeline has tons of sub-blocks. In
70 addition to that, it also contains the CP microcontrollers (ME, PFP, CE,
71 MEC) and the RLC microcontroller. It's exposed to userspace for user mode
72 drivers (OpenGL, Vulkan, OpenCL, etc.). More details in :ref:`Graphics (GFX)
73 and Compute <amdgpu-gc>`.
74
75VCN (Video Core Next)
76 This is the multi-media engine. It handles video and image encode and
77 decode. It's exposed to userspace for user mode drivers (VA-API,
78 OpenMAX, etc.)
79
80It is important to note that these blocks can interact with each other. The
81picture below illustrates some of the components and their interconnection:
82
83.. kernel-figure:: amd_overview_block.svg
84
85In the diagram, memory-related blocks are shown in green. Notice that specific
86IPs have a green square that represents a small hardware block named 'hub',
87which is responsible for interfacing with memory. All memory hubs are connected
88in the UMCs, which in turn are connected to memory blocks. As a note,
89pre-vega devices have a dedicated block for the Graphic Memory Controller
90(GMC), which was replaced by UMC and hubs in new architectures. In the driver
91code, you can identify this component by looking for the suffix hub, for
92example: gfxhub, dchub, mmhub, vmhub, etc. Keep in mind that the component's
93interaction with the memory block may vary across architectures. For example,
94on Navi and newer, GC and SDMA are both attached to GCHUB; on pre-Navi, SDMA
95goes through MMHUB; VCN, JPEG, and VPE go through MMHUB; DCN goes through
96DCHUB.
97
98There is some protection for certain memory elements, and the PSP plays an
99essential role in this area. When a specific firmware is loaded into memory,
100the PSP takes steps to ensure it has a valid signature. It also stores firmware
101images in a protected memory area named Trusted Memory Area (TMR), so the OS or
102driver can't corrupt them at runtime. Another use of PSP is to support Trusted
103Applications (TA), which are basically small applications that run on the
104trusted processor and handles a trusted operation (e.g., HDCP). PSP is also
105used for encrypted memory for content protection via Trusted Memory Zone (TMZ).
106
107Another critical IP is the SMU. It handles reset distribution, as well as
108clock, thermal, and power management for all IPs on the SoC. SMU also helps to
109balance performance and power consumption.
110
111.. _pipes-and-queues-description:
112
113GFX, Compute, and SDMA Overall Behavior
114=======================================
115
116.. note:: For simplicity, whenever the term block is used in this section, it
117 means GFX, Compute, and SDMA.
118
119GFX, Compute and SDMA share a similar form of operation that can be abstracted
120to facilitate understanding of the behavior of these blocks. See the figure
121below illustrating the common components of these blocks:
122
123.. kernel-figure:: pipe_and_queue_abstraction.svg
124
125In the central part of this figure, you can see two hardware elements, one called
126**Pipes** and another called **Queues**; it is important to highlight that Queues
127must be associated with a Pipe and vice-versa. Every specific hardware IP may have
128a different number of Pipes and, in turn, a different number of Queues; for
129example, GFX 11 has two Pipes and two Queues per Pipe for the GFX front end.
130
131Pipe is the hardware that processes the instructions available in the Queues;
132in other words, it is a thread executing the operations inserted in the Queue.
133One crucial characteristic of Pipes is that they can only execute one Queue at
134a time; no matter if the hardware has multiple Queues in the Pipe, it only runs
135one Queue per Pipe.
136
137Pipes have the mechanics of swapping between queues at the hardware level.
138Nonetheless, they only make use of Queues that are considered mapped. Pipes can
139switch between queues based on any of the following inputs:
140
1411. Command Stream;
1422. Packet by Packet;
1433. Other hardware requests the change (e.g., MES).
144
145Queues within Pipes are defined by the Hardware Queue Descriptors (HQD).
146Associated with the HQD concept, we have the Memory Queue Descriptor (MQD),
147which is responsible for storing information about the state of each of the
148available Queues in the memory. The state of a Queue contains information such
149as the GPU virtual address of the queue itself, save areas, doorbell, etc. The
150MQD also stores the HQD registers, which are vital for activating or
151deactivating a given Queue. The scheduling firmware (e.g., MES) is responsible
152for loading HQDs from MQDs and vice versa.
153
154The Queue-switching process can also happen with the firmware requesting the
155preemption or unmapping of a Queue. The firmware waits for the HQD_ACTIVE bit
156to change to low before saving the state into the MQD. To make a different
157Queue become active, the firmware copies the MQD state into the HQD registers
158and loads any additional state. Finally, it sets the HQD_ACTIVE bit to high to
159indicate that the queue is active. The Pipe will then execute work from active
160Queues.
161
162Driver Structure
163================
164
165In general, the driver has a list of all of the IPs on a particular
166SoC and for things like init/fini/suspend/resume, more or less just
167walks the list and handles each IP.
168
169Some useful constructs:
170
171KIQ (Kernel Interface Queue)
172 This is a control queue used by the kernel driver to manage other gfx
173 and compute queues on the GFX/compute engine. You can use it to
174 map/unmap additional queues, etc. This is replaced by MES on
175 GFX 11 and newer hardware.
176
177IB (Indirect Buffer)
178 A command buffer for a particular engine. Rather than writing
179 commands directly to the queue, you can write the commands into a
180 piece of memory and then put a pointer to the memory into the queue.
181 The hardware will then follow the pointer and execute the commands in
182 the memory, then returning to the rest of the commands in the ring.
183
184.. _amdgpu_memory_domains:
185
186Memory Domains
187==============
188
189.. kernel-doc:: include/uapi/drm/amdgpu_drm.h
190 :doc: memory domains
191
192Buffer Objects
193==============
194
195.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
196 :doc: amdgpu_object
197
198.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
199 :internal:
200
201PRIME Buffer Sharing
202====================
203
204.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
205 :doc: PRIME Buffer Sharing
206
207.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
208 :internal:
209
210MMU Notifier
211============
212
213.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
214 :doc: MMU Notifier
215
216.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
217 :internal:
218
219AMDGPU Virtual Memory
220=====================
221
222.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
223 :doc: GPUVM
224
225.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
226 :internal:
227
228Interrupt Handling
229==================
230
231.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
232 :doc: Interrupt Handling
233
234.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
235 :internal:
236
237IP Blocks
238=========
239
240.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h
241 :doc: IP Blocks
242
243.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h
244 :identifiers: amd_ip_block_type amd_ip_funcs DC_FEATURE_MASK DC_DEBUG_MASK