This check is expensive and adversely impacts startup times for some
servers with several, large RSA keys.

It was nice to have, but it's not really going to stop a targetted
attack and was never designed to – hopefully people's private keys
aren't attacker controlled!

Overall I think the feeling is that people would rather have the CPU
time back.

Fixes #6626.

Change-Id: I0143a58c9f22381116d4ca2a3bbba0d28575f3e5
Reviewed-on: https://go-review.googlesource.com/5641Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>