This also fixes the semantics of some corner cases with the empty
match. TODOs for genericReplacer in the tests are fixed.

benchmark                  old ns/op    new ns/op    delta
BenchmarkGenericNoMatch        71395         3132  -95.61%
BenchmarkGenericMatch1         75610        20280  -73.18%
BenchmarkGenericMatch2        837995        86725  -89.65%

R=nigeltao, rsc
CC=golang-dev
https://golang.org/cl/6492076