If an error happens on a connection, server goroutine can call b.Logf
after benchmark finishes.
So join both client and server goroutines.
Update #7718

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/84750047