linux-phytec-fsl

[media] s5p-mfc: don't close instance after free OUTPUT buffers

User-space applications can use the VIDIOC_REQBUFS ioctl to determine if a
memory mapped, user pointer or DMABUF based I/O is supported by the driver.

So a set of VIDIOC_REQBUFS ioctl calls will be made with count 0 and then
the real VIDIOC_REQBUFS call with count == n. But for count 0, the driver
not only frees the buffer but also closes the MFC instance and s5p_mfc_ctx
state is set to MFCINST_FREE.

The VIDIOC_REQBUFS handler for the output device checks if the s5p_mfc_ctx
state is set to MFCINST_INIT (which happens on an VIDIOC_S_FMT) and fails
otherwise. So after a VIDIOC_REQBUFS(n), future VIDIOC_REQBUFS(n) calls
will fails unless a VIDIOC_S_FMT ioctl calls happens before the reqbufs.

But applications may first set the format and then attempt to determine
the I/O methods supported by the driver (for example Gstramer does it) so
the state won't be set to MFCINST_INIT again and VIDIOC_REQBUFS will fail.

To avoid this issue, only free the buffers on VIDIOC_REQBUFS(0) but don't
close the MFC instance to allow future VIDIOC_REQBUFS(n) calls to succeed.

[javier: Rewrote changelog to explain the problem more detailed]

Signed-off-by: ayaka <ayaka@soulik.info>
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Nicolas Dufresne <nicolas@collabora.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>