Skip to content

G
golang
Commit 37d078ed authored by Russ Cox's avatar Russ Cox
Browse files
Options

math/big: add Baillie-PSW test to (*Int).ProbablyPrime 

After x.ProbablyPrime(n) passes the n Miller-Rabin rounds,
add a Baillie-PSW test before declaring x probably prime.

Although the provable error bounds are unchanged, the empirical
error bounds drop dramatically: there are no known inputs
for which Baillie-PSW gives the wrong answer. For example,
before this CL, big.NewInt(443*1327).ProbablyPrime(1) == true.
Now it is (correctly) false.

The new Baillie-PSW test is two pieces: an added Miller-Rabin
round with base 2, and a so-called extra strong Lucas test.
(See the references listed in prime.go for more details.)
The Lucas test takes about 3.5x as long as the Miller-Rabin round,
which is close to theoretical expectations.

name                              time/op
ProbablyPrime/Lucas             2.91ms ± 2%
ProbablyPrime/MillerRabinBase2   850µs ± 1%
ProbablyPrime/n=0               3.75ms ± 3%

The speed of prime testing for a prime input does get slower:

name                  old time/op  new time/op   delta
ProbablyPrime/n=1    849µs ± 1%   4521µs ± 1%  +432.31%   (p=0.000 n=10+9)
ProbablyPrime/n=5   4.31ms ± 3%   7.87ms ± 1%   +82.70%  (p=0.000 n=10+10)
ProbablyPrime/n=10  8.52ms ± 3%  12.28ms ± 1%   +44.11%  (p=0.000 n=10+10)
ProbablyPrime/n=20  16.9ms ± 2%   21.4ms ± 2%   +26.35%   (p=0.000 n=9+10)

However, because the Baillie-PSW test is only added when the old
ProbablyPrime(n) would return true, testing composites runs at
the same speed as before, except in the case where the result
would have been incorrect and is now correct.

In particular, the most important use of this code is for
generating random primes in crypto/rand. That use spends
essentially all its time testing composites, so it is not
slowed down by the new Baillie-PSW check:

name                  old time/op  new time/op   delta
Prime                104ms ±22%    111ms ±16%      ~     (p=0.165 n=10+10)

Thanks to Serhat Şevki Dinçer for CL 20170, which this CL builds on.

Fixes #13229.

Change-Id: Id26dde9b012c7637c85f2e96355d029b6382812a
Reviewed-on: https://go-review.googlesource.com/30770
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarRobert Griesemer <gri@golang.org>
parent 526b2f85
Hide whitespace changes
Inline Side-by-side
Showing with 390 additions and 57 deletions
src/crypto/rand/util_test.go
View file @ 37d078ed
......@@ -7,7 +7,9 @@ package rand_test
import (
"crypto/rand"
"math/big"
mathrand "math/rand"
"testing"
"time"
)
// https://golang.org/issue/6849.
......@@ -63,3 +65,10 @@ func TestIntNegativeMaxPanics(t *testing.T) {
b := new(big.Int).SetInt64(int64(-1))
testIntPanics(t, b)
}
func BenchmarkPrime(b *testing.B) {
r := mathrand.New(mathrand.NewSource(time.Now().UnixNano()))
for i := 0; i < b.N; i++ {
rand.Prime(r, 1024)
}
}
src/math/big/prime.go
View file @ 37d078ed
......@@ -6,74 +6,84 @@ package big
import "math/rand"
// ProbablyPrime performs n Miller-Rabin tests to check whether x is prime.
// If x is prime, it returns true.
// If x is not prime, it returns false with probability at least 1 - ¼ⁿ.
// ProbablyPrime reports whether x is probably prime,
// applying the Miller-Rabin test with n pseudorandomly chosen bases
// as well as a Baillie-PSW test.
//
// It is not suitable for judging primes that an adversary may have crafted
// to fool this test.
// If x is prime, ProbablyPrime returns true.
// If x is chosen randomly and not prime, ProbablyPrime probably returns false.
// The probability of returning true for a randomly chosen non-prime is at most ¼ⁿ.
//
// ProbablyPrime is 100% accurate for inputs less than 2⁶⁴.
// See Menezes et al., Handbook of Applied Cryptography, 1997, pp. 145-149,
// and FIPS 186-4 Appendix F for further discussion of the error probabilities.
//
// ProbablyPrime is not suitable for judging primes that an adversary may
// have crafted to fool the test.
//
// As of Go 1.8, ProbablyPrime(0) is allowed and applies only a Baillie-PSW test.
// Before Go 1.8, ProbablyPrime applied only the Miller-Rabin tests, and ProbablyPrime(0) panicked.
func (x *Int) ProbablyPrime(n int) bool {
if n <= 0 {
panic("non-positive n for ProbablyPrime")
}
return !x.neg && x.abs.probablyPrime(n)
}
// Note regarding the doc comment above:
// It would be more precise to say that the Baillie-PSW test uses the
// extra strong Lucas test as its Lucas test, but since no one knows
// how to tell any of the Lucas tests apart inside a Baillie-PSW test
// (they all work equally well empirically), that detail need not be
// documented or implicitly guaranteed.
// The comment does avoid saying "the" Baillie-PSW test
// because of this general ambiguity.
// probablyPrime performs n Miller-Rabin tests to check whether x is prime.
// If x is prime, it returns true.
// If x is not prime, it returns false with probability at least 1 - ¼ⁿ.
//
// It is not suitable for judging primes that an adversary may have crafted
// to fool this test.
func (n nat) probablyPrime(reps int) bool {
if len(n) == 0 {
if n < 0 {
panic("negative n for ProbablyPrime")
}
if x.neg || len(x.abs) == 0 {
return false
}
if len(n) == 1 {
if n[0] < 2 {
return false
}
if n[0]%2 == 0 {
return n[0] == 2
}
// primeBitMask records the primes < 64.
const primeBitMask uint64 = 1<<2 | 1<<3 | 1<<5 | 1<<7 |
1<<11 | 1<<13 | 1<<17 | 1<<19 | 1<<23 | 1<<29 | 1<<31 |
1<<37 | 1<<41 | 1<<43 | 1<<47 | 1<<53 | 1<<59 | 1<<61
// We have to exclude these cases because we reject all
// multiples of these numbers below.
switch n[0] {
case 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53:
return true
}
w := x.abs[0]
if len(x.abs) == 1 && w < 64 {
return primeBitMask&(1<<w) != 0
}
if n[0]&1 == 0 {
if w&1 == 0 {
return false // n is even
}
const primesProduct32 = 0xC0CFD797 // Π {p ∈ primes, 2 < p <= 29}
const primesProduct64 = 0xE221F97C30E94E1D // Π {p ∈ primes, 2 < p <= 53}
const primesA = 3 * 5 * 7 * 11 * 13 * 17 * 19 * 23 * 37
const primesB = 29 * 31 * 41 * 43 * 47 * 53
var r Word
var rA, rB uint32
switch _W {
case 32:
r = n.modW(primesProduct32)
rA = uint32(x.abs.modW(primesA))
rB = uint32(x.abs.modW(primesB))
case 64:
r = n.modW(primesProduct64 & _M)
r := x.abs.modW((primesA * primesB) & _M)
rA = uint32(r % primesA)
rB = uint32(r % primesB)
default:
panic("Unknown word size")
panic("math/big: invalid word size")
}
if r%3 == 0 || r%5 == 0 || r%7 == 0 || r%11 == 0 ||
r%13 == 0 || r%17 == 0 || r%19 == 0 || r%23 == 0 || r%29 == 0 {
if rA%3 == 0 || rA%5 == 0 || rA%7 == 0 || rA%11 == 0 || rA%13 == 0 || rA%17 == 0 || rA%19 == 0 || rA%23 == 0 || rA%37 == 0 ||
rB%29 == 0 || rB%31 == 0 || rB%41 == 0 || rB%43 == 0 || rB%47 == 0 || rB%53 == 0 {
return false
}
if _W == 64 && (r%31 == 0 || r%37 == 0 || r%41 == 0 ||
r%43 == 0 || r%47 == 0 || r%53 == 0) {
return false
}
return x.abs.probablyPrimeMillerRabin(n+1, true) && x.abs.probablyPrimeLucas()
}
// probablyPrimeMillerRabin reports whether n passes reps rounds of the
// Miller-Rabin primality test, using pseudo-randomly chosen bases.
// If force2 is true, one of the rounds is forced to use base 2.
// See Handbook of Applied Cryptography, p. 139, Algorithm 4.24.
// The number n is known to be non-zero.
func (n nat) probablyPrimeMillerRabin(reps int, force2 bool) bool {
nm1 := nat(nil).sub(n, natOne)
// determine q, k such that nm1 = q << k
k := nm1.trailingZeroBits()
......@@ -87,8 +97,12 @@ func (n nat) probablyPrime(reps int) bool {
NextRandom:
for i := 0; i < reps; i++ {
x = x.random(rand, nm3, nm3Len)
x = x.add(x, natTwo)
if i == reps-1 && force2 {
x = x.set(natTwo)
} else {
x = x.random(rand, nm3, nm3Len)
x = x.add(x, natTwo)
}
y = y.expNN(x, q, n)
if y.cmp(natOne) == 0 || y.cmp(nm1) == 0 {
continue
......@@ -108,3 +122,199 @@ NextRandom:
return true
}
// probablyPrimeLucas reports whether n passes the "almost extra strong" Lucas probable prime test,
// using Baillie-OEIS parameter selection. This corresponds to "AESLPSP" on Jacobsen's tables (link below).
// The combination of this test and a Miller-Rabin/Fermat test with base 2 gives a Baillie-PSW test.
//
// References:
//
// Baillie and Wagstaff, "Lucas Pseudoprimes", Mathematics of Computation 35(152),
// October 1980, pp. 1391-1417, especially page 1401.
// http://www.ams.org/journals/mcom/1980-35-152/S0025-5718-1980-0583518-6/S0025-5718-1980-0583518-6.pdf
//
// Grantham, "Frobenius Pseudoprimes", Mathematics of Computation 70(234),
// March 2000, pp. 873-891.
// http://www.ams.org/journals/mcom/2001-70-234/S0025-5718-00-01197-2/S0025-5718-00-01197-2.pdf
//
// Baillie, "Extra strong Lucas pseudoprimes", OEIS A217719, https://oeis.org/A217719.
//
// Jacobsen, "Pseudoprime Statistics, Tables, and Data", http://ntheory.org/pseudoprimes.html.
//
// Nicely, "The Baillie-PSW Primality Test", http://www.trnicely.net/misc/bpsw.html.
// (Note that Nicely's definition of the "extra strong" test gives the wrong Jacobi condition,
// as pointed out by Jacobsen.)
//
// Crandall and Pomerance, Prime Numbers: A Computational Perspective, 2nd ed.
// Springer, 2005.
func (n nat) probablyPrimeLucas() bool {
// Discard 0, 1.
if len(n) == 0 || n.cmp(natOne) == 0 {
return false
}
// Two is the only even prime.
// Already checked by caller, but here to allow testing in isolation.
if n[0]&1 == 0 {
return n.cmp(natTwo) == 0
}
// Baillie-OEIS "method C" for choosing D, P, Q,
// as in https://oeis.org/A217719/a217719.txt:
// try increasing P ≥ 3 such that D = P² - 4 (so Q = 1)
// until Jacobi(D, n) = -1.
// The search is expected to succeed for non-square n after just a few trials.
// After more than expected failures, check whether n is square
// (which would cause Jacobi(D, n) = 1 for all D not dividing n).
p := Word(3)
d := nat{1}
t1 := nat(nil) // temp
intD := &Int{abs: d}
intN := &Int{abs: n}
for ; ; p++ {
if p > 10000 {
// This is widely believed to be impossible.
// If we get a report, we'll want the exact number n.
panic("math/big: internal error: cannot find (D/n) = -1 for " + intN.String())
}
d[0] = p*p - 4
j := Jacobi(intD, intN)
if j == -1 {
break
}
if j == 0 {
// d = p²-4 = (p-2)(p+2).
// If (d/n) == 0 then d shares a prime factor with n.
// Since the loop proceeds in increasing p and starts with p-2==1,
// the shared prime factor must be p+2.
// If p+2 == n, then n is prime; otherwise p+2 is a proper factor of n.
return len(n) == 1 && n[0] == p+2
}
if p == 40 {
// We'll never find (d/n) = -1 if n is a square.
// If n is a non-square we expect to find a d in just a few attempts on average.
// After 40 attempts, take a moment to check if n is indeed a square.
t1 = t1.sqrt(n)
t1 = t1.mul(t1, t1)
if t1.cmp(n) == 0 {
return false
}
}
}
// Grantham definition of "extra strong Lucas pseudoprime", after Thm 2.3 on p. 876
// (D, P, Q above have become Δ, b, 1):
//
// Let U_n = U_n(b, 1), V_n = V_n(b, 1), and Δ = b²-4.
// An extra strong Lucas pseudoprime to base b is a composite n = 2^r s + Jacobi(Δ, n),
// where s is odd and gcd(n, 2*Δ) = 1, such that either (i) U_s ≡ 0 mod n and V_s ≡ ±2 mod n,
// or (ii) V_{2^t s} ≡ 0 mod n for some 0 ≤ t < r-1.
//
// We know gcd(n, Δ) = 1 or else we'd have found Jacobi(d, n) == 0 above.
// We know gcd(n, 2) = 1 because n is odd.
//
// Arrange s = (n - Jacobi(Δ, n)) / 2^r = (n+1) / 2^r.
s := nat(nil).add(n, natOne)
r := int(s.trailingZeroBits())
s = s.shr(s, uint(r))
nm2 := nat(nil).sub(n, natTwo) // n-2
// We apply the "almost extra strong" test, which checks the above conditions
// except for U_s ≡ 0 mod n, which allows us to avoid computing any U_k values.
// Jacobsen points out that maybe we should just do the full extra strong test:
// "It is also possible to recover U_n using Crandall and Pomerance equation 3.13:
// U_n = D^-1 (2V_{n+1} - PV_n) allowing us to run the full extra-strong test
// at the cost of a single modular inversion. This computation is easy and fast in GMP,
// so we can get the full extra-strong test at essentially the same performance as the
// almost extra strong test."
// Compute Lucas sequence V_s(b, 1), where:
//
// V(0) = 2
// V(1) = P
// V(k) = P V(k-1) - Q V(k-2).
//
// (Remember that due to method C above, P = b, Q = 1.)
//
// In general V(k) = α^k + β^k, where α and β are roots of x² - Px + Q.
// Crandall and Pomerance (p.147) observe that for 0 ≤ j ≤ k,
//
// V(j+k) = V(j)V(k) - V(k-j).
//
// So in particular, to quickly double the subscript:
//
// V(2k) = V(k)² - 2
// V(2k+1) = V(k) V(k+1) - P
//
// We can therefore start with k=0 and build up to k=s in log₂(s) steps.
natP := nat(nil).setWord(p)
vk := nat(nil).setWord(2)
vk1 := nat(nil).setWord(p)
t2 := nat(nil) // temp
for i := int(s.bitLen()); i >= 0; i-- {
if s.bit(uint(i)) != 0 {
// k' = 2k+1
// V(k') = V(2k+1) = V(k) V(k+1) - P.
t1 = t1.mul(vk, vk1)
t1 = t1.add(t1, n)
t1 = t1.sub(t1, natP)
t2, vk = t2.div(vk, t1, n)
// V(k'+1) = V(2k+2) = V(k+1)² - 2.
t1 = t1.mul(vk1, vk1)
t1 = t1.add(t1, nm2)
t2, vk1 = t2.div(vk1, t1, n)
} else {
// k' = 2k
// V(k'+1) = V(2k+1) = V(k) V(k+1) - P.
t1 = t1.mul(vk, vk1)
t1 = t1.add(t1, n)
t1 = t1.sub(t1, natP)
t2, vk1 = t2.div(vk1, t1, n)
// V(k') = V(2k) = V(k)² - 2
t1 = t1.mul(vk, vk)
t1 = t1.add(t1, nm2)
t2, vk = t2.div(vk, t1, n)
}
}
// Now k=s, so vk = V(s). Check V(s) ≡ ±2 (mod n).
if vk.cmp(natTwo) == 0 || vk.cmp(nm2) == 0 {
// Check U(s) ≡ 0.
// As suggested by Jacobsen, apply Crandall and Pomerance equation 3.13:
//
// U(k) = D⁻¹ (2 V(k+1) - P V(k))
//
// Since we are checking for U(k) == 0 it suffices to check 2 V(k+1) == P V(k) mod n,
// or P V(k) - 2 V(k+1) == 0 mod n.
t1 := t1.mul(vk, natP)
t2 := t2.shl(vk1, 1)
if t1.cmp(t2) < 0 {
t1, t2 = t2, t1
}
t1 = t1.sub(t1, t2)
t3 := vk1 // steal vk1, no longer needed below
vk1 = nil
_ = vk1
t2, t3 = t2.div(t3, t1, n)
if len(t3) == 0 {
return true
}
}
// Check V(2^t s) ≡ 0 mod n for some 0 ≤ t < r-1.
for t := 0; t < r-1; t++ {
if len(vk) == 0 { // vk == 0
return true
}
// Optimization: V(k) = 2 is a fixed point for V(k') = V(k)² - 2,
// so if V(k) = 2, we can stop: we will never find a future V(k) == 0.
if len(vk) == 1 && vk[0] == 2 { // vk == 2
return false
}
// k' = 2k
// V(k') = V(2k) = V(k)² - 2
t1 = t1.mul(vk, vk)
t1 = t1.sub(t1, natTwo)
t2, vk = t2.div(vk, t1, n)
}
return false
}
src/math/big/prime_test.go
View file @ 37d078ed
......@@ -6,7 +6,9 @@ package big
import (
"fmt"
"strings"
"testing"
"unicode"
)
var primes = []string{
......@@ -48,28 +50,96 @@ var composites = []string{
"6084766654921918907427900243509372380954290099172559290432744450051395395951",
"84594350493221918389213352992032324280367711247940675652888030554255915464401",
"82793403787388584738507275144194252681",
// Arnault, "Rabin-Miller Primality Test: Composite Numbers Which Pass It",
// Mathematics of Computation, 64(209) (January 1995), pp. 335-361.
"1195068768795265792518361315725116351898245581", // strong pseudoprime to prime bases 2 through 29
// strong pseudoprime to all prime bases up to 200
`
80383745745363949125707961434194210813883768828755814583748891752229
74273765333652186502336163960045457915042023603208766569966760987284
0439654082329287387918508691668573282677617710293896977394701670823
0428687109997439976544144845341155872450633409279022275296229414984
2306881685404326457534018329786111298960644845216191652872597534901`,
// Extra-strong Lucas pseudoprimes. https://oeis.org/A217719
"989",
"3239",
"5777",
"10877",
"27971",
"29681",
"30739",
"31631",
"39059",
"72389",
"73919",
"75077",
"100127",
"113573",
"125249",
"137549",
"137801",
"153931",
"155819",
"161027",
"162133",
"189419",
"218321",
"231703",
"249331",
"370229",
"429479",
"430127",
"459191",
"473891",
"480689",
"600059",
"621781",
"632249",
"635627",
"3673744903",
"3281593591",
"2385076987",
"2738053141",
"2009621503",
"1502682721",
"255866131",
"117987841",
"587861",
"6368689",
"8725753",
"80579735209",
"105919633",
}
func cutSpace(r rune) rune {
if unicode.IsSpace(r) {
return -1
}
return r
}
func TestProbablyPrime(t *testing.T) {
nreps := 20
if testing.Short() {
nreps = 1
nreps = 3
}
for i, s := range primes {
p, _ := new(Int).SetString(s, 10)
if !p.ProbablyPrime(nreps) {
if !p.ProbablyPrime(nreps) || !p.ProbablyPrime(1) || !p.ProbablyPrime(0) {
t.Errorf("#%d prime found to be non-prime (%s)", i, s)
}
}
for i, s := range composites {
s = strings.Map(cutSpace, s)
c, _ := new(Int).SetString(s, 10)
if c.ProbablyPrime(nreps) {
if c.ProbablyPrime(nreps) || c.ProbablyPrime(1) || c.ProbablyPrime(0) {
t.Errorf("#%d composite found to be prime (%s)", i, s)
}
if testing.Short() {
break
}
}
// check that ProbablyPrime panics if n <= 0
......@@ -77,7 +147,7 @@ func TestProbablyPrime(t *testing.T) {
for _, n := range []int{-1, 0, 1} {
func() {
defer func() {
if n <= 0 && recover() == nil {
if n < 0 && recover() == nil {
t.Fatalf("expected panic from ProbablyPrime(%d)", n)
}
}()
......@@ -90,11 +160,55 @@ func TestProbablyPrime(t *testing.T) {
func BenchmarkProbablyPrime(b *testing.B) {
p, _ := new(Int).SetString("203956878356401977405765866929034577280193993314348263094772646453283062722701277632936616063144088173312372882677123879538709400158306567338328279154499698366071906766440037074217117805690872792848149112022286332144876183376326512083574821647933992961249917319836219304274280243803104015000563790123", 10)
for _, rep := range []int{1, 5, 10, 20} {
b.Run(fmt.Sprintf("Rep=%d", rep), func(b *testing.B) {
for _, n := range []int{0, 1, 5, 10, 20} {
b.Run(fmt.Sprintf("n=%d", n), func(b *testing.B) {
for i := 0; i < b.N; i++ {
p.ProbablyPrime(rep)
p.ProbablyPrime(n)
}
})
}
b.Run("Lucas", func(b *testing.B) {
for i := 0; i < b.N; i++ {
p.abs.probablyPrimeLucas()
}
})
b.Run("MillerRabinBase2", func(b *testing.B) {
for i := 0; i < b.N; i++ {
p.abs.probablyPrimeMillerRabin(1, true)
}
})
}
func TestMillerRabinPseudoprimes(t *testing.T) {
testPseudoprimes(t, "probablyPrimeMillerRabin",
func(n nat) bool { return n.probablyPrimeMillerRabin(1, true) && !n.probablyPrimeLucas() },
// https://oeis.org/A001262
[]int{2047, 3277, 4033, 4681, 8321, 15841, 29341, 42799, 49141, 52633, 65281, 74665, 80581, 85489, 88357, 90751})
}
func TestLucasPseudoprimes(t *testing.T) {
testPseudoprimes(t, "probablyPrimeLucas",
func(n nat) bool { return n.probablyPrimeLucas() && !n.probablyPrimeMillerRabin(1, true) },
// https://oeis.org/A217719
[]int{989, 3239, 5777, 10877, 27971, 29681, 30739, 31631, 39059, 72389, 73919, 75077})
}
func testPseudoprimes(t *testing.T, name string, cond func(nat) bool, want []int) {
n := nat{1}
for i := 3; i < 100000; i += 2 {
n[0] = Word(i)
pseudo := cond(n)
if pseudo && (len(want) == 0 || i != want[0]) {
t.Errorf("%s(%v, base=2) = %v, want false", name, i)
} else if !pseudo && len(want) >= 1 && i == want[0] {
t.Errorf("%s(%v, base=2) = false, want true", name, i)
}
if len(want) > 0 && i == want[0] {
want = want[1:]
}
}
if len(want) > 0 {
t.Fatalf("forgot to test %v", want)
}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment