cmd/compile: compute faulting args before writing args to stack

when compiling f(a, b, c), we do something like:
  *(SP+0) = eval(a)
  *(SP+8) = eval(b)
  *(SP+16) = eval(c)
  call f

If one of those evaluations is later determined to unconditionally panic
(say eval(b) in this example), then the call is deadcode eliminated. But
any previous argument write (*(SP+0)=... here) is still around. Becuase
we only compute the size of the outarg area for calls which are still
around at the end of optimization, the space needed for *(SP+0)=v is not
accounted for and thus the outarg area may be too small.

The fix is to make sure that we evaluate any potentially panicing
operation before we write any of the args to the stack. It turns out
that fix is pretty easy, as we already have such a mechanism available
for function args. We just need to extend it to possibly panicing args
as well.

The resulting code (if b and c can panic, but a can't) is:
  tmpb = eval(b)
  *(SP+16) = eval(c)
  *(SP+0) = eval(a)
  *(SP+8) = tmpb
  call f

This change tickled a bug in how we find the arguments for intrinsic
calls, so that latent bug is fixed up as well.

Update #16760.

Change-Id: I0bf5edf370220f82bc036cf2085ecc24f356d166
Reviewed-on: https://go-review.googlesource.com/32551Reviewed-by: Cherry Zhang <cherryyz@google.com>

src/cmd/compile/internal/gc/subr.go

@@ -1182,6 +1182,12 @@ func ullmancalc(n *Node) {
 			ul = UINF
 			goto out
 		}
+	case OINDEX, OSLICE, OSLICEARR, OSLICE3, OSLICE3ARR, OSLICESTR,
+		OIND, ODOTPTR, ODOTTYPE, ODIV:
+		// These ops might panic, make sure they are done
+		// before we start marshaling args for a call. See issue 16760.
+		ul = UINF
+		goto out
 	}
 
 	ul = 1

test/fixedbugs/issue16760.go

+// run
+
+// Copyright 2016 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// Make sure we don't start marshaling (writing to the stack)
+// arguments until those arguments are evaluated and known
+// not to unconditinally panic. If they unconditionally panic,
+// we write some args but never do the call. That messes up
+// the logic which decides how big the argout section needs to be.
+
+package main
+
+type W interface {
+	Write([]byte)
+}
+
+type F func(W)
+
+func foo(f F) {
+	defer func() {
+		if r := recover(); r != nil {
+			usestack(1000)
+		}
+	}()
+	f(nil)
+}
+
+func main() {
+	foo(func(w W) {
+		var x []string
+		w.Write([]byte(x[5]))
+	})
+}
+
+func usestack(n int) {
+	if n == 0 {
+		return
+	}
+	usestack(n - 1)
+}