| 1 | ******************************** |
| 2 | RDMA Network Block Device (RNBD) |
| 3 | ******************************** |
| 4 | |
| 5 | Introduction |
| 6 | ------------ |
| 7 | |
| 8 | RNBD (RDMA Network Block Device) is a pair of kernel modules |
| 9 | (client and server) that allow for remote access of a block device on |
| 10 | the server over RTRS protocol using the RDMA (InfiniBand, RoCE, iWARP) |
| 11 | transport. After being mapped, the remote block devices can be accessed |
| 12 | on the client side as local block devices. |
| 13 | |
| 14 | I/O is transferred between client and server by the RTRS transport |
| 15 | modules. The administration of RNBD and RTRS modules is done via |
| 16 | sysfs entries. |
| 17 | |
| 18 | Requirements |
| 19 | ------------ |
| 20 | |
| 21 | RTRS kernel modules |
| 22 | |
| 23 | Quick Start |
| 24 | ----------- |
| 25 | |
| 26 | Server side: |
| 27 | # modprobe rnbd_server |
| 28 | |
| 29 | Client side: |
| 30 | # modprobe rnbd_client |
| 31 | # echo "sessname=blya path=ip:10.50.100.66 device_path=/dev/ram0" > \ |
| 32 | /sys/devices/virtual/rnbd-client/ctl/map_device |
| 33 | |
| 34 | Where "sessname=" is a session name, a string to identify the session |
| 35 | on client and on server sides; "path=" is a destination IP address or |
| 36 | a pair of a source and a destination IPs, separated by comma. Multiple |
| 37 | "path=" options can be specified in order to use multipath (see RTRS |
| 38 | description for details); "device_path=" is the block device to be |
| 39 | mapped from the server side. After the session to the server machine is |
| 40 | established, the mapped device will appear on the client side under |
| 41 | /dev/rnbd<N>. |
| 42 | |
| 43 | |
| 44 | RNBD-Server Module Parameters |
| 45 | ============================= |
| 46 | |
| 47 | dev_search_path |
| 48 | --------------- |
| 49 | |
| 50 | When a device is mapped from the client, the server generates the path |
| 51 | to the block device on the server side by concatenating dev_search_path |
| 52 | and the "device_path" that was specified in the map_device operation. |
| 53 | |
| 54 | The default dev_search_path is: "/". |
| 55 | |
| 56 | dev_search_path option can also contain %SESSNAME% in order to provide |
| 57 | different device namespaces for different sessions. See "device_path" |
| 58 | option for details. |
| 59 | |
| 60 | ============================ |
| 61 | Protocol (rnbd/rnbd-proto.h) |
| 62 | ============================ |
| 63 | |
| 64 | 1. Before mapping first device from a given server, client sends an |
| 65 | RNBD_MSG_SESS_INFO to the server. Server responds with |
| 66 | RNBD_MSG_SESS_INFO_RSP. Currently the messages only contain the protocol |
| 67 | version for backward compatibility. |
| 68 | |
| 69 | 2. Client requests to open a device by sending RNBD_MSG_OPEN message. This |
| 70 | contains the path to the device and access mode (read-only or writable). |
| 71 | Server responds to the message with RNBD_MSG_OPEN_RSP. This contains |
| 72 | a 32 bit device id to be used for IOs and device "geometry" related |
| 73 | information: side, max_hw_sectors, etc. |
| 74 | |
| 75 | 3. Client attaches RNBD_MSG_IO to each IO message send to a device. This |
| 76 | message contains device id, provided by server in his rnbd_msg_open_rsp, |
| 77 | sector to be accessed, read-write flags and bi_size. |
| 78 | |
| 79 | 4. Client closes a device by sending RNBD_MSG_CLOSE which contains only the |
| 80 | device id provided by the server. |
| 81 | |
| 82 | ========================================= |
| 83 | Contributors List(in alphabetical order) |
| 84 | ========================================= |
| 85 | Danil Kipnis <[email protected]> |
| 86 | Fabian Holler <[email protected]> |
| 87 | Guoqing Jiang <[email protected]> |
| 88 | Jack Wang <[email protected]> |
| 89 | Kleber Souza <[email protected]> |
| 90 | Lutz Pogrell <[email protected]> |
| 91 | Milind Dumbare <[email protected]> |
| 92 | Roman Penyaev <[email protected]> |
| 93 | Swapnil Ingle <[email protected]> |