The algtype-based test broke when algtype
got a bit more fine-grained, so replace with
an explicit check for the invalid key types.

R=ken2
CC=golang-dev
https://golang.org/cl/5071041