Revert "ALSA: usb-audio: Refcount multiple accesses on the single clock"

This reverts commit 9830e738 which is
commit c11117b6 upstream.

It breaks the Android kernel abi and can be brought back in the future
in an abi-safe way if it is really needed.

Bug: 161946584
Change-Id: I118d9b1069bcc55b41a2353ecd4e00d42c301cec
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>