| 1 | |
| 2 |
|
| 3 | .. _media_writing_camera_sensor_drivers:
|
| 4 |
|
| 5 | Writing camera sensor drivers
|
| 6 | =============================
|
| 7 |
|
| 8 | This document covers the in-kernel APIs only. For the best practices on
|
| 9 | userspace API implementation in camera sensor drivers, please see
|
| 10 | :ref:`media_using_camera_sensor_drivers`.
|
| 11 |
|
| 12 | CSI-2, parallel and BT.656 buses
|
| 13 | --------------------------------
|
| 14 |
|
| 15 | Please see :ref:`transmitter-receiver`.
|
| 16 |
|
| 17 | Handling clocks
|
| 18 | ---------------
|
| 19 |
|
| 20 | Camera sensors have an internal clock tree including a PLL and a number of
|
| 21 | divisors. The clock tree is generally configured by the driver based on a few
|
| 22 | input parameters that are specific to the hardware: the external clock frequency
|
| 23 | and the link frequency. The two parameters generally are obtained from system
|
| 24 | firmware. **No other frequencies should be used in any circumstances.**
|
| 25 |
|
| 26 | The reason why the clock frequencies are so important is that the clock signals
|
| 27 | come out of the SoC, and in many cases a specific frequency is designed to be
|
| 28 | used in the system. Using another frequency may cause harmful effects
|
| 29 | elsewhere. Therefore only the pre-determined frequencies are configurable by the
|
| 30 | user.
|
| 31 |
|
| 32 | The external clock frequency shall be retrieved by obtaining the external clock
|
| 33 | using the ``devm_v4l2_sensor_clk_get()`` helper function, and then getting its
|
| 34 | frequency with ``clk_get_rate()``. Usage of the helper function guarantees
|
| 35 | correct behaviour regardless of whether the sensor is integrated in a DT-based
|
| 36 | or ACPI-based system.
|
| 37 |
|
| 38 | ACPI
|
| 39 | ~~~~
|
| 40 |
|
| 41 | ACPI-based systems typically don't register the sensor external clock with the
|
| 42 | kernel, but specify the external clock frequency in the ``clock-frequency``
|
| 43 | _DSD property. The ``devm_v4l2_sensor_clk_get()`` helper creates and returns a
|
| 44 | fixed clock set at that rate.
|
| 45 |
|
| 46 | Devicetree
|
| 47 | ~~~~~~~~~~
|
| 48 |
|
| 49 | Devicetree-based systems declare the sensor external clock in the device tree
|
| 50 | and reference it from the sensor node. The preferred way to select the external
|
| 51 | clock frequency is to use the ``assigned-clocks``, ``assigned-clock-parents``
|
| 52 | and ``assigned-clock-rates`` properties in the sensor node to set the clock
|
| 53 | rate. See the `clock device tree bindings
|
| 54 | <https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/clock/clock.yaml>`_
|
| 55 | for more information. The ``devm_v4l2_sensor_clk_get()`` helper retrieves and
|
| 56 | returns that clock.
|
| 57 |
|
| 58 | This approach has the drawback that there's no guarantee that the frequency
|
| 59 | hasn't been modified directly or indirectly by another driver, or supported by
|
| 60 | the board's clock tree to begin with. Changes to the Common Clock Framework API
|
| 61 | are required to ensure reliability.
|
| 62 |
|
| 63 | Power management
|
| 64 | ----------------
|
| 65 |
|
| 66 | Camera sensors are used in conjunction with other devices to form a camera
|
| 67 | pipeline. They must obey the rules listed herein to ensure coherent power
|
| 68 | management over the pipeline.
|
| 69 |
|
| 70 | Camera sensor drivers are responsible for controlling the power state of the
|
| 71 | device they otherwise control as well. They shall use runtime PM to manage
|
| 72 | power states. Runtime PM shall be enabled at probe time and disabled at remove
|
| 73 | time. Drivers should enable runtime PM autosuspend. Also see
|
| 74 | :ref:`async sub-device registration <media-registering-async-subdevs>`.
|
| 75 |
|
| 76 | The runtime PM handlers shall handle clocks, regulators, GPIOs, and other
|
| 77 | system resources required to power the sensor up and down. For drivers that
|
| 78 | don't use any of those resources (such as drivers that support ACPI systems
|
| 79 | only), the runtime PM handlers may be left unimplemented.
|
| 80 |
|
| 81 | In general, the device shall be powered on at least when its registers are
|
| 82 | being accessed and when it is streaming. Drivers should use
|
| 83 | ``pm_runtime_resume_and_get()`` when starting streaming and
|
| 84 | ``pm_runtime_put()`` or ``pm_runtime_put_autosuspend()`` when stopping
|
| 85 | streaming. They may power the device up at probe time (for example to read
|
| 86 | identification registers), but should not keep it powered unconditionally after
|
| 87 | probe.
|
| 88 |
|
| 89 | At system suspend time, the whole camera pipeline must stop streaming, and
|
| 90 | restart when the system is resumed. This requires coordination between the
|
| 91 | camera sensor and the rest of the camera pipeline. Bridge drivers are
|
| 92 | responsible for this coordination, and instruct camera sensors to stop and
|
| 93 | restart streaming by calling the appropriate subdev operations
|
| 94 | (``.enable_streams()`` or ``.disable_streams()``). Camera sensor drivers shall
|
| 95 | therefore **not** keep track of the streaming state to stop streaming in the PM
|
| 96 | suspend handler and restart it in the resume handler. Drivers should in general
|
| 97 | not implement the system PM handlers.
|
| 98 |
|
| 99 | Camera sensor drivers shall **not** implement the subdev ``.s_power()``
|
| 100 | operation, as it is deprecated. While this operation is implemented in some
|
| 101 | existing drivers as they predate the deprecation, new drivers shall use runtime
|
| 102 | PM instead. If you feel you need to begin calling ``.s_power()`` from an ISP or
|
| 103 | a bridge driver, instead add runtime PM support to the sensor driver you are
|
| 104 | using and drop its ``.s_power()`` handler.
|
| 105 |
|
| 106 | Please also see :ref:`examples <media-camera-sensor-examples>`.
|
| 107 |
|
| 108 | Control framework
|
| 109 | ~~~~~~~~~~~~~~~~~
|
| 110 |
|
| 111 | ``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime
|
| 112 | PM ``runtime_resume`` callback, as it has no way to figure out the power state
|
| 113 | of the device. This is because the power state of the device is only changed
|
| 114 | after the power state transition has taken place. The ``s_ctrl`` callback can be
|
| 115 | used to obtain device's power state after the power state transition:
|
| 116 |
|
| 117 | .. c:function:: int pm_runtime_get_if_active(struct device *dev);
|
| 118 |
|
| 119 | The function returns a non-zero value if it succeeded getting the power count or
|
| 120 | runtime PM was disabled, in either of which cases the driver may proceed to
|
| 121 | access the device.
|
| 122 |
|
| 123 | Rotation, orientation and flipping
|
| 124 | ----------------------------------
|
| 125 |
|
| 126 | Use ``v4l2_fwnode_device_parse()`` to obtain rotation and orientation
|
| 127 | information from system firmware and ``v4l2_ctrl_new_fwnode_properties()`` to
|
| 128 | register the appropriate controls.
|
| 129 |
|
| 130 | |
| 131 |
|
| 132 | Example drivers
|
| 133 | ---------------
|
| 134 |
|
| 135 | Features implemented by sensor drivers vary, and depending on the set of
|
| 136 | supported features and other qualities, particular sensor drivers better serve
|
| 137 | the purpose of an example. The following drivers are known to be good examples:
|
| 138 |
|
| 139 | .. flat-table:: Example sensor drivers
|
| 140 | :header-rows: 0
|
| 141 | :widths: 1 1 1 2
|
| 142 |
|
| 143 | * - Driver name
|
| 144 | - File(s)
|
| 145 | - Driver type
|
| 146 | - Example topic
|
| 147 | * - CCS
|
| 148 | - ``drivers/media/i2c/ccs/``
|
| 149 | - Freely configurable
|
| 150 | - Power management (ACPI and DT), UAPI
|
| 151 | * - imx219
|
| 152 | - ``drivers/media/i2c/imx219.c``
|
| 153 | - Register list based
|
| 154 | - Power management (DT), UAPI, mode selection
|
| 155 | * - imx319
|
| 156 | - ``drivers/media/i2c/imx319.c``
|
| 157 | - Register list based
|
| 158 | - Power management (ACPI and DT)
|