This change fixes a missing case that a routing address contains an
invalid address family label but it holds a valid length of address
structure.

Also makes test robust.

Fixes #10041.

Change-Id: I2480ba273929e859896697382d1a75b01a116b98
Reviewed-on: https://go-review.googlesource.com/6391Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>