media: v4l: async: Don't set notifier's V4L2 device if registering fails

The V4L2 device used to be set when the notifier was registered but this
has been moved to the notifier initialisation. Don't touch the V4L2 device
if registration fails.

Fixes: b8ec754a ("media: v4l: async: Set v4l2_device and subdev in async notifier init")
Cc: <stable@vger.kernel.org> # for 6.6 and later
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>