This stops a repeat deletion from sending requests all the way to
Kubernetes, and gives a clear error message when a deletion is requested
on an already deleted resource.

Relates to #972