| 1 | |
| 2 |
|
| 3 | ==================
|
| 4 | AF_XDP TX Metadata
|
| 5 | ==================
|
| 6 |
|
| 7 | This document describes how to enable offloads when transmitting packets
|
| 8 | via :doc:`af_xdp`. Refer to :doc:`xdp-rx-metadata` on how to access similar
|
| 9 | metadata on the receive side.
|
| 10 |
|
| 11 | General Design
|
| 12 | ==============
|
| 13 |
|
| 14 | The headroom for the metadata is reserved via ``tx_metadata_len`` and
|
| 15 | ``XDP_UMEM_TX_METADATA_LEN`` flag in ``struct xdp_umem_reg``. The metadata
|
| 16 | length is therefore the same for every socket that shares the same umem.
|
| 17 | The metadata layout is a fixed UAPI, refer to ``struct xsk_tx_metadata`` in
|
| 18 | ``include/uapi/linux/if_xdp.h``. Thus, generally, the ``tx_metadata_len``
|
| 19 | field above should contain ``sizeof(struct xsk_tx_metadata)``.
|
| 20 |
|
| 21 | Note that in the original implementation the ``XDP_UMEM_TX_METADATA_LEN``
|
| 22 | flag was not required. Applications might attempt to create a umem
|
| 23 | with a flag first and if it fails, do another attempt without a flag.
|
| 24 |
|
| 25 | The headroom and the metadata itself should be located right before
|
| 26 | ``xdp_desc->addr`` in the umem frame. Within a frame, the metadata
|
| 27 | layout is as follows::
|
| 28 |
|
| 29 | tx_metadata_len
|
| 30 | / \
|
| 31 | +-----------------+---------+----------------------------+
|
| 32 | | xsk_tx_metadata | padding | payload |
|
| 33 | +-----------------+---------+----------------------------+
|
| 34 | ^
|
| 35 | |
|
| 36 | xdp_desc->addr
|
| 37 |
|
| 38 | An AF_XDP application can request headrooms larger than ``sizeof(struct
|
| 39 | xsk_tx_metadata)``. The kernel will ignore the padding (and will still
|
| 40 | use ``xdp_desc->addr - tx_metadata_len`` to locate
|
| 41 | the ``xsk_tx_metadata``). For the frames that shouldn't carry
|
| 42 | any metadata (i.e., the ones that don't have ``XDP_TX_METADATA`` option),
|
| 43 | the metadata area is ignored by the kernel as well.
|
| 44 |
|
| 45 | The flags field enables the particular offload:
|
| 46 |
|
| 47 | - ``XDP_TXMD_FLAGS_TIMESTAMP``: requests the device to put transmission
|
| 48 | timestamp into ``completion.tx_timestamp`` field of
|
| 49 | ``struct xsk_tx_metadata``.
|
| 50 | - ``XDP_TXMD_FLAGS_CHECKSUM``: requests the device to calculate L4
|
| 51 | checksum. ``request.csum_start`` specifies byte offset of where the
|
| 52 | checksumming should start and ``request.csum_offset`` specifies byte offset
|
| 53 | where the device should store the computed checksum.
|
| 54 | - ``XDP_TXMD_FLAGS_LAUNCH_TIME``: requests the device to schedule the
|
| 55 | packet for transmission at a pre-determined time called launch time. The
|
| 56 | value of launch time is indicated by ``request.launch_time`` field of
|
| 57 | ``struct xsk_tx_metadata``.
|
| 58 |
|
| 59 | Besides the flags above, in order to trigger the offloads, the first
|
| 60 | packet's ``struct xdp_desc`` descriptor should set ``XDP_TX_METADATA``
|
| 61 | bit in the ``options`` field. Also note that in a multi-buffer packet
|
| 62 | only the first chunk should carry the metadata.
|
| 63 |
|
| 64 | Software TX Checksum
|
| 65 | ====================
|
| 66 |
|
| 67 | For development and testing purposes it's possible to pass
|
| 68 | ``XDP_UMEM_TX_SW_CSUM`` flag to ``XDP_UMEM_REG`` UMEM registration call.
|
| 69 | In this case, when running in ``XDP_COPY`` mode, the TX checksum
|
| 70 | is calculated on the CPU. Do not enable this option in production because
|
| 71 | it will negatively affect performance.
|
| 72 |
|
| 73 | Launch Time
|
| 74 | ===========
|
| 75 |
|
| 76 | The value of the requested launch time should be based on the device's PTP
|
| 77 | Hardware Clock (PHC) to ensure accuracy. AF_XDP takes a different data path
|
| 78 | compared to the ETF queuing discipline, which organizes packets and delays
|
| 79 | their transmission. Instead, AF_XDP immediately hands off the packets to
|
| 80 | the device driver without rearranging their order or holding them prior to
|
| 81 | transmission. Since the driver maintains FIFO behavior and does not perform
|
| 82 | packet reordering, a packet with a launch time request will block other
|
| 83 | packets in the same Tx Queue until it is sent. Therefore, it is recommended
|
| 84 | to allocate separate queue for scheduling traffic that is intended for
|
| 85 | future transmission.
|
| 86 |
|
| 87 | In scenarios where the launch time offload feature is disabled, the device
|
| 88 | driver is expected to disregard the launch time request. For correct
|
| 89 | interpretation and meaningful operation, the launch time should never be
|
| 90 | set to a value larger than the farthest programmable time in the future
|
| 91 | (the horizon). Different devices have different hardware limitations on the
|
| 92 | launch time offload feature.
|
| 93 |
|
| 94 | stmmac driver
|
| 95 | -------------
|
| 96 |
|
| 97 | For stmmac, TSO and launch time (TBS) features are mutually exclusive for
|
| 98 | each individual Tx Queue. By default, the driver configures Tx Queue 0 to
|
| 99 | support TSO and the rest of the Tx Queues to support TBS. The launch time
|
| 100 | hardware offload feature can be enabled or disabled by using the tc-etf
|
| 101 | command to call the driver's ndo_setup_tc() callback.
|
| 102 |
|
| 103 | The value of the launch time that is programmed in the Enhanced Normal
|
| 104 | Transmit Descriptors is a 32-bit value, where the most significant 8 bits
|
| 105 | represent the time in seconds and the remaining 24 bits represent the time
|
| 106 | in 256 ns increments. The programmed launch time is compared against the
|
| 107 | PTP time (bits[39:8]) and rolls over after 256 seconds. Therefore, the
|
| 108 | horizon of the launch time for dwmac4 and dwxlgmac2 is 128 seconds in the
|
| 109 | future.
|
| 110 |
|
| 111 | igc driver
|
| 112 | ----------
|
| 113 |
|
| 114 | For igc, all four Tx Queues support the launch time feature. The launch
|
| 115 | time hardware offload feature can be enabled or disabled by using the
|
| 116 | tc-etf command to call the driver's ndo_setup_tc() callback. When entering
|
| 117 | TSN mode, the igc driver will reset the device and create a default Qbv
|
| 118 | schedule with a 1-second cycle time, with all Tx Queues open at all times.
|
| 119 |
|
| 120 | The value of the launch time that is programmed in the Advanced Transmit
|
| 121 | Context Descriptor is a relative offset to the starting time of the Qbv
|
| 122 | transmission window of the queue. The First flag of the descriptor can be
|
| 123 | set to schedule the packet for the next Qbv cycle. Therefore, the horizon
|
| 124 | of the launch time for i225 and i226 is the ending time of the next cycle
|
| 125 | of the Qbv transmission window of the queue. For example, when the Qbv
|
| 126 | cycle time is set to 1 second, the horizon of the launch time ranges
|
| 127 | from 1 second to 2 seconds, depending on where the Qbv cycle is currently
|
| 128 | running.
|
| 129 |
|
| 130 | Querying Device Capabilities
|
| 131 | ============================
|
| 132 |
|
| 133 | Every device exports its offload capabilities via the Netlink netdev family.
|
| 134 | Query the ``xsk-features`` attribute in
|
| 135 | ``Documentation/netlink/specs/netdev.yaml``. Its bits are defined by the
|
| 136 | ``xsk-flags`` enum.
|
| 137 |
|
| 138 | - ``tx-timestamp``: device supports ``XDP_TXMD_FLAGS_TIMESTAMP``
|
| 139 | - ``tx-checksum``: device supports ``XDP_TXMD_FLAGS_CHECKSUM``
|
| 140 | - ``tx-launch-time-fifo``: device supports ``XDP_TXMD_FLAGS_LAUNCH_TIME``
|
| 141 |
|
| 142 | See ``tools/net/ynl/samples/netdev.c`` on how to query this information.
|
| 143 |
|
| 144 | Example
|
| 145 | =======
|
| 146 |
|
| 147 | See ``tools/testing/selftests/bpf/xdp_hw_metadata.c`` for an example
|
| 148 | program that handles TX metadata. Also see https://github.com/fomichev/xskgen
|
| 149 | for a more bare-bones example.
|