If the closure pointer gets spilled, we need to spill it with
pointer type to make stack copy and GC happy.

Change-Id: Ic108748e6b9caecd45522141f02c9422567376e3
Reviewed-on: https://go-review.googlesource.com/16363
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Replace REP MOVSB with all the copying techniques used by the
old compiler.  Copy in chunks, DUFFCOPY, etc.

Introduces MOVO opcodes and an Int128 type to move around
16 bytes at a time.

Change-Id: I1e73e68ca1d8b3dd58bb4af2f4c9e5d9bf13a502
Reviewed-on: https://go-review.googlesource.com/16174Reviewed-by: Todd Neal <todd@tneal.org>
Run-TryBot: Keith Randall <khr@golang.org>

Use faulting loads instead of test/jeq to do nil checks.
Fold nil checks into a following load/store if possible.

Makes binaries about 2% smaller.

Change-Id: I54af0f0a93c853f37e34e0ce7e3f01dd2ac87f64
Reviewed-on: https://go-review.googlesource.com/16287Reviewed-by: David Chase <drchase@google.com>

unsafe.Pointer->uintptr, add, then uintptr->unsafe.Pointer.
Do the add directly on the pointer type instead.

Change-Id: I5a3a32691d0a000e16975857974ed9a1039c6d28
Reviewed-on: https://go-review.googlesource.com/16281
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Modified GOSSA{HASH.PKG} environment variable filters to
make it easier to make/run with all SSA for testing.
Disable attempts at SSA for architectures that are not
amd64 (avoid spurious errors/unimplementeds.)

Removed easy out for unimplemented features.

Add convert op for proper liveness in presence of uintptr
to/from unsafe.Pointer conversions.

Tweaked stack sizes to get a pass on windows;
1024 instead 768, was observed to pass at least once.

Change-Id: Ida3800afcda67d529e3b1cf48ca4a3f0fa48b2c5
Reviewed-on: https://go-review.googlesource.com/16201Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>

Calls to NewConfig required an extra parameter that
sometimes could not be nil.

Change-Id: I806dd53c045056a0c2d30d641a20fe27fb790539
Reviewed-on: https://go-review.googlesource.com/16272Reviewed-by: Keith Randall <khr@golang.org>

BP for framepointer experiment
R15 for dynamic linking

Change-Id: I28e48be461d04a4d5c9b013f48fce5c0e58d6a08
Reviewed-on: https://go-review.googlesource.com/16231
Run-TryBot: Todd Neal <todd@tneal.org>
Reviewed-by: Todd Neal <todd@tneal.org>

Register phis are better than stack phis.  If we have
unused registers available, use them for phis.

Change-Id: I3045711c65caa1b6d0be29131b87b57466320cc2
Reviewed-on: https://go-review.googlesource.com/16080
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

It confuses live variable analysis to have a bunch of unreachable
no-ops at the end of a function.  Symptom is:
    gc/plive.go:483 panic: interface conversion: interface {} is nil, not *gc.BasicBlock

I don't see any reason why the old compiler needs these no-ops either.
all.bash passes with the equivalent code removed on master.

Change-Id: Ifcd2c3e139aa16314f08aebc9079b2fb7aa60556
Reviewed-on: https://go-review.googlesource.com/16132Reviewed-by: David Chase <drchase@google.com>

getg reads from memory, so it should really have a
memory arg.  It is critical in functions which call setg
to make sure getg gets ordered correctly with setg.

Change-Id: Ief4875421f741fc49c07b0e1f065ce2535232341
Reviewed-on: https://go-review.googlesource.com/16100
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>

It isn't safe, the place where we're moving the value to
might have a different live memory.  Moving will introduce
two simultaneously live memories.

Change-Id: I07e61a6db8ef285088c530dc2e5d5768d27871ff
Reviewed-on: https://go-review.googlesource.com/16099Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>

It isn't safe in functions that also call setg.

Change-Id: I76a7bf0401b4b6c8a129c245b15a2d6f06080e94
Reviewed-on: https://go-review.googlesource.com/16095Reviewed-by: Todd Neal <todd@tneal.org>

The only major fixup is that duffzero changed from
8-byte writes to 16-byte writes.

Change-Id: I1762b74ce67a8e4b81c11568027cdb3572f7f87c

I cannot find any documentation for this, but these tests no longer run
on the device I have since upgrading to Android L. Presumably it still
works for root, but standard Android programs to not have root access.

Change-Id: I001c8fb5ce22f9ff8d7433f881d0dccbf6ab969d
Reviewed-on: https://go-review.googlesource.com/16056Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>

Rematerialize constants instead of spilling and loading them.
"Constants" includes constant offsets from SP and SB.

Should help somewhat with stack frame sizes.  I'm not sure
exactly how much yet.

Change-Id: I44dbad97aae870cf31cb6e89c92fe4f6a2b9586f
Reviewed-on: https://go-review.googlesource.com/16029
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

These functions are always called together and perform logically
related state resets, so combine them in to just gcResetMarkState.

Fixes #11427.

Change-Id: I06c17ef65f66186494887a767b3993126955b5fe
Reviewed-on: https://go-review.googlesource.com/16041Reviewed-by: Rick Hudson <rlh@golang.org>

Currently gcResetGState is called by func gcscan_m for concurrent GC
and directly by func gc for STW GC. Simplify this by consolidating
these two calls in to one call by func gc above where it splits for
concurrent and STW GC.

As a consequence, gcResetGState and gcResetMarkState are always called
together, so the next commit will consolidate these.

Change-Id: Ib62d404c7b32b28f7d3080d26ecf3966cbc4aca0
Reviewed-on: https://go-review.googlesource.com/16040Reviewed-by: Rick Hudson <rlh@golang.org>

This work queue is no longer used (there are many reads of
work.partial, but the only write is in putpartial, which is never
called).

Fixes #11922.

Change-Id: I08b76c0c02a0867a9cdcb94783e1f7629d44249a
Reviewed-on: https://go-review.googlesource.com/15892Reviewed-by: Rick Hudson <rlh@golang.org>

fmt docs say:

  If the operand is a reflect.Value, the concrete value it
  holds is printed as if it was the operand.

It implies recursive application of this rule, which is not the case.
Clarify the docs.

Change-Id: I019277c7c6439095bab83e5536aa06403638aa51
Reviewed-on: https://go-review.googlesource.com/15952Reviewed-by: Ian Lance Taylor <iant@golang.org>

Also make PIE executables the default build mode, as PIE executables
are required as of Android L.

For #10807

Change-Id: I86b7556b9792105cd2531df1b8f3c8f7a8c5d25c
Reviewed-on: https://go-review.googlesource.com/16055Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>

For #10807

Change-Id: Ied826d06cb622edf6413b6f2cdcc46987ab0b05a
Reviewed-on: https://go-review.googlesource.com/16054Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #12945.

Change-Id: I08b44795fcd7ec59371aea8111f7febead54720b
Reviewed-on: https://go-review.googlesource.com/15900Reviewed-by: Robert Griesemer <gri@golang.org>

Change-Id: Ib424e14cfaab35d37ebdd084d41151928bfd645e
Reviewed-on: https://go-review.googlesource.com/16051Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>

It was generating the wrong error message, always defaulting to "500
Internal Server Error", since the err variable used was always nil.

Fixes #12991

Change-Id: I94b0e516409c131ff3b878bcb91e65f0259ff077
Reviewed-on: https://go-review.googlesource.com/16060Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

It appears this was made possible by commit 89f185fe; before that, g was
not dereferenced above.

Change-Id: I70bc571d924b36351392fd4c13d681e938cfb573
Reviewed-on: https://go-review.googlesource.com/16033Reviewed-by: Andrew Gerrand <adg@golang.org>

Reuse the ArgSymbol for nodes so that the Aux values will be equal for cse.

Change-Id: Iaae80bd19ff2d3f51b6c9049fd860e04baa6f175
Reviewed-on: https://go-review.googlesource.com/15930Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Todd Neal <todd@tneal.org>

from proc1.go to proc.go:
* prepend header comment explaining "Goroutine scheduler"
* insert m0 and g0 var defs after the comment
* append the rest

Updates #12952

Change-Id: I35ee9ae3287675cde0c1b6aeaca0a460393f2354
Reviewed-on: https://go-review.googlesource.com/16024
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The flate library guarantees that the Reader will never read more
bytes than is necessary. This way, the underlying io.Reader will
be left exactly after the last byte of the DEFLATE stream.
Formats like gzip depend on this behavior being true.

As such, inflate conservatively reads the minimum symbol length in
huffSym leading to many individual calls to moreBits. However, if we
take advantage of the fact that every block *must* end with the EOB
symbol, we can choose to read the length of the EOB symbol.
Since the EOB symbol is also the most rare symbol (occuring exactly
once) in a block, we can hypothesize that it is almost as long as
the max symbol length, allowing huffSym to ask for more bits at the
start of every loop. This increases the probabilty that the Huffman
code is decoded on the first iteration of the outer for-loop.

benchmark                              old MB/s     new MB/s     speedup
BenchmarkDecodeDigitsSpeed1e4-4        51.05        54.31        1.06x
BenchmarkDecodeDigitsSpeed1e5-4        58.86        62.24        1.06x
BenchmarkDecodeDigitsSpeed1e6-4        59.63        63.13        1.06x
BenchmarkDecodeDigitsDefault1e4-4      51.94        54.61        1.05x
BenchmarkDecodeDigitsDefault1e5-4      63.70        69.13        1.09x
BenchmarkDecodeDigitsDefault1e6-4      66.08        71.43        1.08x
BenchmarkDecodeDigitsCompress1e4-4     52.25        54.56        1.04x
BenchmarkDecodeDigitsCompress1e5-4     63.34        68.30        1.08x
BenchmarkDecodeDigitsCompress1e6-4     66.84        70.64        1.06x
BenchmarkDecodeTwainSpeed1e4-4         50.74        53.40        1.05x
BenchmarkDecodeTwainSpeed1e5-4         60.77        67.03        1.10x
BenchmarkDecodeTwainSpeed1e6-4         62.08        69.78        1.12x
BenchmarkDecodeTwainDefault1e4-4       53.45        56.40        1.06x
BenchmarkDecodeTwainDefault1e5-4       73.54        79.05        1.07x
BenchmarkDecodeTwainDefault1e6-4       77.68        83.65        1.08x
BenchmarkDecodeTwainCompress1e4-4      53.21        56.15        1.06x
BenchmarkDecodeTwainCompress1e5-4      73.82        77.76        1.05x
BenchmarkDecodeTwainCompress1e6-4      79.23        83.30        1.05x

Change-Id: Ie194925c827988a380b8c2fdd13b13c4faa5d397
Reviewed-on: https://go-review.googlesource.com/15651Reviewed-by: Nigel Tao <nigeltao@golang.org>

* append contents of race1.go to race.go
* delete "Implementation of the race detector API." comment
  from race1.go

Updates #12952

Change-Id: Ibdd9c4dc79a63c3bef69eade9525578063c86c1c
Reviewed-on: https://go-review.googlesource.com/16023
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

PIC code on ppc64le uses R2 as a TOC pointer and when calling a function
through a function pointer must ensure the function pointer is in R12.  These
rules are easy enough to follow unconditionally in our assembly, so do that.

Change-Id: Icfc4e47ae5dfbe15f581cbdd785cdeed6e40bc32
Reviewed-on: https://go-review.googlesource.com/15526Reviewed-by: Ian Lance Taylor <iant@golang.org>

Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). Part 3 of that
is using a #define in the ppc64 assembly to refer to the size of the fixed
part of the stack (finding all these took me about a week!).

Change-Id: I50f22fe1c47af1ec59da1bd7ea8f84a4750df9b7
Reviewed-on: https://go-review.googlesource.com/15525Reviewed-by: Ian Lance Taylor <iant@golang.org>

Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). Part 2a of
preparing for that is to have all bits of arch-independent and ppc64-specific
codegen that need to know call a function to find out.

Change-Id: I55899f73037e92227813c491049a3bd6f30bd41f
Reviewed-on: https://go-review.googlesource.com/15524Reviewed-by: Ian Lance Taylor <iant@golang.org>

Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). So to prepare
for this, make a constant for the fixed part of a stack and use that where
necessary.

Change-Id: I447949f4d725003bb82e7d2cf7991c1bca5aa887
Reviewed-on: https://go-review.googlesource.com/15523Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Replace the confusing game where a frame size of $-8 would suppress the
implicit setting up of a stack frame with a nice explicit flag.

The code to set up the function prologue is still a little confusing but better
than it was.

Change-Id: I1d49278ff42c6bc734ebfb079998b32bc53f8d9a
Reviewed-on: https://go-review.googlesource.com/15670Reviewed-by: Minux Ma <minux@golang.org>

This fixes a TODO in (*WordDecoder).Decode by replacing a call to strings.Split with
simple prefix/suffix checking and some custom slicing.

Benchmark results:

benchmark                    old ns/op     new ns/op     delta
BenchmarkQEncodeWord-8       740           693           -6.35%
BenchmarkQDecodeWord-8       1291          727           -43.69%
BenchmarkQDecodeHeader-8     1194          767           -35.76%

benchmark                    old allocs     new allocs     delta
BenchmarkQEncodeWord-8       1              1              +0.00%
BenchmarkQDecodeWord-8       3              2              -33.33%
BenchmarkQDecodeHeader-8     3              2              -33.33%

benchmark                    old bytes     new bytes     delta
BenchmarkQEncodeWord-8       48            48            +0.00%
BenchmarkQDecodeWord-8       128           48            -62.50%
BenchmarkQDecodeHeader-8     128           48            -62.50%

Change-Id: I2d6844c75ec2e2b79be2e49b7fc4ca320b7e84e5
Reviewed-on: https://go-review.googlesource.com/14432Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

For each type, maintain a list of stack slots used to spill
SSA values to the stack.  Reuse those stack slots for noninterfering
spills.

Lowers frame sizes.  As an example, runtime.mSpan_Sweep goes from
584 bytes to 392 bytes.  heapBitsSetType goes from 576 bytes to 152 bytes.

Change-Id: I0e9afe80c2fd84aff9eb368318685de293c363d0
Reviewed-on: https://go-review.googlesource.com/16022Reviewed-by: David Chase <drchase@google.com>

Changed racewalk/race detector to use FP in a more
sensible way.

Relaxed checks for CONVNOP when race detecting.

Modified tighten to ensure that GetClosurePtr cannot float
out of entry block (turns out this cannot be relaxed, DX is
sometimes stomped by other code accompanying race detection).

Added case for addr(CONVNOP)

Modified addr to take "bounded" flag to suppress nilchecks
where it is set (usually, by race detector).

Cannot leave unimplemented-complainer enabled because it
turns out we are optimistically running SSA on every platform.

Change-Id: Ife021654ee4065b3ffac62326d09b4b317b9f2e0
Reviewed-on: https://go-review.googlesource.com/15710Reviewed-by: Keith Randall <khr@golang.org>

* rename stack1.go -> stack.go
* prepend contents of stack2.go to stack.go

Updates #12952

Change-Id: I60d409af37162a5a7596c678dfebc2cea89564ff
Reviewed-on: https://go-review.googlesource.com/16008Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Go cannot allow lazy PLT resolution when calling between Go functions because
the lazy resolution can use more stack than is available. Lazy resolution is
disabled by passing -z now to the system linker, but unfortunately was only
passed when linking to a Go shared library. That sounds fine, but the shared
library containing the runtime is not linked to any other Go shared library but
calls main.init and main.main via a PLT, and before this fix this did use lazy
resolution. (For some reason this never caused a problem on intel, but it
breaks on ppc64le). Fortunately the fix is very simple: always pass -z now to
the system linker when dynamically linking Go.

Change-Id: I7806d40aac80dcd1e56b95864d1cfeb1c42614e2
Reviewed-on: https://go-review.googlesource.com/15870Reviewed-by: Ian Lance Taylor <iant@golang.org>