int(maxMem) is negative on 32 bits.  Need to use
unsigned arithmetic.

TBR=bradfitz
CC=golang-codereviews
https://golang.org/cl/121000045