Avoid unnecessary bitwise-OR operations.

benchmark                      old MB/s     new MB/s  speedup
BenchmarkEncodeToStringBase64  179.02       205.74    1.15x
BenchmarkEncodeToStringBase32  155.86       167.82    1.08x

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/109090043