Instead of keeping the desired number of seconds and converting to
time.Duration for every query, convert to time.Duration when
building the config.

Updates #15473

Change-Id: Ib24c050b593b3109011e359f4ed837a3fb45dc65
Reviewed-on: https://go-review.googlesource.com/22548Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>