dm: core: Avoid partially removing devices

At present if device_remove() decides that the device should not actually
be removed, it still calls the uclass pre_remove() method and powers the
device down.

Signed-off-by: Simon Glass <sjg@chromium.org>
diff --git a/test/dm/virtio.c b/test/dm/virtio.c
index ad35598..9a7e658 100644
--- a/test/dm/virtio.c
+++ b/test/dm/virtio.c
@@ -123,7 +123,9 @@
 
 	/* check the device can be successfully removed */
 	dev_or_flags(dev, DM_FLAG_ACTIVATED);
-	ut_assertok(device_remove(bus, DM_REMOVE_ACTIVE_ALL));
+	ut_asserteq(-EKEYREJECTED, device_remove(bus, DM_REMOVE_ACTIVE_ALL));
+
+	ut_asserteq(false, device_active(dev));
 
 	return 0;
 }