wifi: iwlwifi: pcie: clean up device removal work

We shouldn't access the device if we don't hold a reference,
and if - after locking - we see that it has no bus, we also
can't do anything, in fact, pci_stop_and_remove_bus_device()
will be a no-op.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Miri Korenblit <miriam.rachel.korenblit@intel.com>
Link: https://msgid.link/20231207044813.6c0879e695f7.I1d3ce75ecad32a4cbf1b9dad61bfb7bc7821fdd9@changeid


Signed-off-by: Johannes Berg <johannes.berg@intel.com>