Part of graceful shutdown.

Update #4674

LGTM=adg, josharian
R=adg, josharian, r
CC=golang-codereviews
https://golang.org/cl/69670043