Recursive types R containing slices of R's did not terminate despite the
effort in CL 10821.

For recursive types there was a competition between slice expansion by a
factor 'complexSize', and termination with probability '1/complexSize'
which lead to stack overflow as soon as a recursive struct had slices
pointing to its own type.

Fix this by shrinking the size hint as a function of recursion depth.
This has the dual effect of reducing the number of elements generated
per slice and also increasing the probability for termination.

Fixes #11148.

Change-Id: Ib61155b4f2e2de3873d508d63a1f4be759426d67
Reviewed-on: https://go-review.googlesource.com/13830Reviewed-by: Adam Langley <agl@golang.org>