probablyPrime would return false negatives in some cases.

This code has now been tested against GMP for several million iterations without issues.

Fixes #638.

R=rsc
CC=golang-dev
https://golang.org/cl/252041