cdsprpc,adsprpc: does not use libcdsprpc libadsprpc

Only lib{a,c}dsp_default_listener is using these lib, which can result
in not linkled library when link with -as-needed

Signed-off-by: Julien Massot <jmassot@softbankrobotics.com>
Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>