| 1 | |
| 2 |
|
| 3 | =====================================
|
| 4 | Asynchronous Transfers/Transforms API
|
| 5 | =====================================
|
| 6 |
|
| 7 | |
| 8 |
|
| 9 | 1. INTRODUCTION
|
| 10 |
|
| 11 | 2 GENEALOGY
|
| 12 |
|
| 13 | 3 USAGE
|
| 14 | 3.1 General format of the API
|
| 15 | 3.2 Supported operations
|
| 16 | 3.3 Descriptor management
|
| 17 | 3.4 When does the operation execute?
|
| 18 | 3.5 When does the operation complete?
|
| 19 | 3.6 Constraints
|
| 20 | 3.7 Example
|
| 21 |
|
| 22 | 4 DMAENGINE DRIVER DEVELOPER NOTES
|
| 23 | 4.1 Conformance points
|
| 24 | 4.2 "My application needs exclusive control of hardware channels"
|
| 25 |
|
| 26 | 5 SOURCE
|
| 27 |
|
| 28 | 1. Introduction
|
| 29 | ===============
|
| 30 |
|
| 31 | The async_tx API provides methods for describing a chain of asynchronous
|
| 32 | bulk memory transfers/transforms with support for inter-transactional
|
| 33 | dependencies. It is implemented as a dmaengine client that smooths over
|
| 34 | the details of different hardware offload engine implementations. Code
|
| 35 | that is written to the API can optimize for asynchronous operation and
|
| 36 | the API will fit the chain of operations to the available offload
|
| 37 | resources.
|
| 38 |
|
| 39 | 2.Genealogy
|
| 40 | ===========
|
| 41 |
|
| 42 | The API was initially designed to offload the memory copy and
|
| 43 | xor-parity-calculations of the md-raid5 driver using the offload engines
|
| 44 | present in the Intel(R) Xscale series of I/O processors. It also built
|
| 45 | on the 'dmaengine' layer developed for offloading memory copies in the
|
| 46 | network stack using Intel(R) I/OAT engines. The following design
|
| 47 | features surfaced as a result:
|
| 48 |
|
| 49 | 1. implicit synchronous path: users of the API do not need to know if
|
| 50 | the platform they are running on has offload capabilities. The
|
| 51 | operation will be offloaded when an engine is available and carried out
|
| 52 | in software otherwise.
|
| 53 | 2. cross channel dependency chains: the API allows a chain of dependent
|
| 54 | operations to be submitted, like xor->copy->xor in the raid5 case. The
|
| 55 | API automatically handles cases where the transition from one operation
|
| 56 | to another implies a hardware channel switch.
|
| 57 | 3. dmaengine extensions to support multiple clients and operation types
|
| 58 | beyond 'memcpy'
|
| 59 |
|
| 60 | 3. Usage
|
| 61 | ========
|
| 62 |
|
| 63 | 3.1 General format of the API
|
| 64 | -----------------------------
|
| 65 |
|
| 66 | ::
|
| 67 |
|
| 68 | struct dma_async_tx_descriptor *
|
| 69 | async_<operation>(<op specific parameters>, struct async_submit_ctl *submit)
|
| 70 |
|
| 71 | 3.2 Supported operations
|
| 72 | ------------------------
|
| 73 |
|
| 74 | ======== ====================================================================
|
| 75 | memcpy memory copy between a source and a destination buffer
|
| 76 | memset fill a destination buffer with a byte value
|
| 77 | xor xor a series of source buffers and write the result to a
|
| 78 | destination buffer
|
| 79 | xor_val xor a series of source buffers and set a flag if the
|
| 80 | result is zero. The implementation attempts to prevent
|
| 81 | writes to memory
|
| 82 | pq generate the p+q (raid6 syndrome) from a series of source buffers
|
| 83 | pq_val validate that a p and or q buffer are in sync with a given series of
|
| 84 | sources
|
| 85 | datap (raid6_recov_datap) recover a raid6 data block and the p block
|
| 86 | from the given sources
|
| 87 | 2data (raid6_recov_2data) recover 2 raid6 data blocks from the given
|
| 88 | sources
|
| 89 | ======== ====================================================================
|
| 90 |
|
| 91 | 3.3 Descriptor management
|
| 92 | -------------------------
|
| 93 |
|
| 94 | The return value is non-NULL and points to a 'descriptor' when the operation
|
| 95 | has been queued to execute asynchronously. Descriptors are recycled
|
| 96 | resources, under control of the offload engine driver, to be reused as
|
| 97 | operations complete. When an application needs to submit a chain of
|
| 98 | operations it must guarantee that the descriptor is not automatically recycled
|
| 99 | before the dependency is submitted. This requires that all descriptors be
|
| 100 | acknowledged by the application before the offload engine driver is allowed to
|
| 101 | recycle (or free) the descriptor. A descriptor can be acked by one of the
|
| 102 | following methods:
|
| 103 |
|
| 104 | 1. setting the ASYNC_TX_ACK flag if no child operations are to be submitted
|
| 105 | 2. submitting an unacknowledged descriptor as a dependency to another
|
| 106 | async_tx call will implicitly set the acknowledged state.
|
| 107 | 3. calling async_tx_ack() on the descriptor.
|
| 108 |
|
| 109 | 3.4 When does the operation execute?
|
| 110 | ------------------------------------
|
| 111 |
|
| 112 | Operations do not immediately issue after return from the
|
| 113 | async_<operation> call. Offload engine drivers batch operations to
|
| 114 | improve performance by reducing the number of mmio cycles needed to
|
| 115 | manage the channel. Once a driver-specific threshold is met the driver
|
| 116 | automatically issues pending operations. An application can force this
|
| 117 | event by calling async_tx_issue_pending_all(). This operates on all
|
| 118 | channels since the application has no knowledge of channel to operation
|
| 119 | mapping.
|
| 120 |
|
| 121 | 3.5 When does the operation complete?
|
| 122 | -------------------------------------
|
| 123 |
|
| 124 | There are two methods for an application to learn about the completion
|
| 125 | of an operation.
|
| 126 |
|
| 127 | 1. Call dma_wait_for_async_tx(). This call causes the CPU to spin while
|
| 128 | it polls for the completion of the operation. It handles dependency
|
| 129 | chains and issuing pending operations.
|
| 130 | 2. Specify a completion callback. The callback routine runs in tasklet
|
| 131 | context if the offload engine driver supports interrupts, or it is
|
| 132 | called in application context if the operation is carried out
|
| 133 | synchronously in software. The callback can be set in the call to
|
| 134 | async_<operation>, or when the application needs to submit a chain of
|
| 135 | unknown length it can use the async_trigger_callback() routine to set a
|
| 136 | completion interrupt/callback at the end of the chain.
|
| 137 |
|
| 138 | 3.6 Constraints
|
| 139 | ---------------
|
| 140 |
|
| 141 | 1. Calls to async_<operation> are not permitted in IRQ context. Other
|
| 142 | contexts are permitted provided constraint #2 is not violated.
|
| 143 | 2. Completion callback routines cannot submit new operations. This
|
| 144 | results in recursion in the synchronous case and spin_locks being
|
| 145 | acquired twice in the asynchronous case.
|
| 146 |
|
| 147 | 3.7 Example
|
| 148 | -----------
|
| 149 |
|
| 150 | Perform a xor->copy->xor operation where each operation depends on the
|
| 151 | result from the previous operation::
|
| 152 |
|
| 153 | #include <linux/async_tx.h>
|
| 154 |
|
| 155 | static void callback(void *param)
|
| 156 | {
|
| 157 | complete(param);
|
| 158 | }
|
| 159 |
|
| 160 | #define NDISKS 2
|
| 161 |
|
| 162 | static void run_xor_copy_xor(struct page **xor_srcs,
|
| 163 | struct page *xor_dest,
|
| 164 | size_t xor_len,
|
| 165 | struct page *copy_src,
|
| 166 | struct page *copy_dest,
|
| 167 | size_t copy_len)
|
| 168 | {
|
| 169 | struct dma_async_tx_descriptor *tx;
|
| 170 | struct async_submit_ctl submit;
|
| 171 | addr_conv_t addr_conv[NDISKS];
|
| 172 | struct completion cmp;
|
| 173 |
|
| 174 | init_async_submit(&submit, ASYNC_TX_XOR_DROP_DST, NULL, NULL, NULL,
|
| 175 | addr_conv);
|
| 176 | tx = async_xor(xor_dest, xor_srcs, 0, NDISKS, xor_len, &submit);
|
| 177 |
|
| 178 | submit.depend_tx = tx;
|
| 179 | tx = async_memcpy(copy_dest, copy_src, 0, 0, copy_len, &submit);
|
| 180 |
|
| 181 | init_completion(&cmp);
|
| 182 | init_async_submit(&submit, ASYNC_TX_XOR_DROP_DST | ASYNC_TX_ACK, tx,
|
| 183 | callback, &cmp, addr_conv);
|
| 184 | tx = async_xor(xor_dest, xor_srcs, 0, NDISKS, xor_len, &submit);
|
| 185 |
|
| 186 | async_tx_issue_pending_all();
|
| 187 |
|
| 188 | wait_for_completion(&cmp);
|
| 189 | }
|
| 190 |
|
| 191 | See include/linux/async_tx.h for more information on the flags. See the
|
| 192 | ops_run_* and ops_complete_* routines in drivers/md/raid5.c for more
|
| 193 | implementation examples.
|
| 194 |
|
| 195 | 4. Driver Development Notes
|
| 196 | ===========================
|
| 197 |
|
| 198 | 4.1 Conformance points
|
| 199 | ----------------------
|
| 200 |
|
| 201 | There are a few conformance points required in dmaengine drivers to
|
| 202 | accommodate assumptions made by applications using the async_tx API:
|
| 203 |
|
| 204 | 1. Completion callbacks are expected to happen in tasklet context
|
| 205 | 2. dma_async_tx_descriptor fields are never manipulated in IRQ context
|
| 206 | 3. Use async_tx_run_dependencies() in the descriptor clean up path to
|
| 207 | handle submission of dependent operations
|
| 208 |
|
| 209 | 4.2 "My application needs exclusive control of hardware channels"
|
| 210 | -----------------------------------------------------------------
|
| 211 |
|
| 212 | Primarily this requirement arises from cases where a DMA engine driver
|
| 213 | is being used to support device-to-memory operations. A channel that is
|
| 214 | performing these operations cannot, for many platform specific reasons,
|
| 215 | be shared. For these cases the dma_request_channel() interface is
|
| 216 | provided.
|
| 217 |
|
| 218 | The interface is::
|
| 219 |
|
| 220 | struct dma_chan *dma_request_channel(dma_cap_mask_t mask,
|
| 221 | dma_filter_fn filter_fn,
|
| 222 | void *filter_param);
|
| 223 |
|
| 224 | Where dma_filter_fn is defined as::
|
| 225 |
|
| 226 | typedef bool (*dma_filter_fn)(struct dma_chan *chan, void *filter_param);
|
| 227 |
|
| 228 | When the optional 'filter_fn' parameter is set to NULL
|
| 229 | dma_request_channel simply returns the first channel that satisfies the
|
| 230 | capability mask. Otherwise, when the mask parameter is insufficient for
|
| 231 | specifying the necessary channel, the filter_fn routine can be used to
|
| 232 | disposition the available channels in the system. The filter_fn routine
|
| 233 | is called once for each free channel in the system. Upon seeing a
|
| 234 | suitable channel filter_fn returns DMA_ACK which flags that channel to
|
| 235 | be the return value from dma_request_channel. A channel allocated via
|
| 236 | this interface is exclusive to the caller, until dma_release_channel()
|
| 237 | is called.
|
| 238 |
|
| 239 | The DMA_PRIVATE capability flag is used to tag dma devices that should
|
| 240 | not be used by the general-purpose allocator. It can be set at
|
| 241 | initialization time if it is known that a channel will always be
|
| 242 | private. Alternatively, it is set when dma_request_channel() finds an
|
| 243 | unused "public" channel.
|
| 244 |
|
| 245 | A couple caveats to note when implementing a driver and consumer:
|
| 246 |
|
| 247 | 1. Once a channel has been privately allocated it will no longer be
|
| 248 | considered by the general-purpose allocator even after a call to
|
| 249 | dma_release_channel().
|
| 250 | 2. Since capabilities are specified at the device level a dma_device
|
| 251 | with multiple channels will either have all channels public, or all
|
| 252 | channels private.
|
| 253 |
|
| 254 | 5. Source
|
| 255 | ---------
|
| 256 |
|
| 257 | include/linux/dmaengine.h:
|
| 258 | core header file for DMA drivers and api users
|
| 259 | drivers/dma/dmaengine.c:
|
| 260 | offload engine channel management routines
|
| 261 | drivers/dma/:
|
| 262 | location for offload engine drivers
|
| 263 | include/linux/async_tx.h:
|
| 264 | core header file for the async_tx api
|
| 265 | crypto/async_tx/async_tx.c:
|
| 266 | async_tx interface to dmaengine and common code
|
| 267 | crypto/async_tx/async_memcpy.c:
|
| 268 | copy offload
|
| 269 | crypto/async_tx/async_xor.c:
|
| 270 | xor and xor zero sum offload
|