Fixes #6833

R=minux.ma, iant
CC=golang-dev
https://golang.org/cl/35790045

R=golang-dev, 0intro, rsc
CC=golang-dev
https://golang.org/cl/39680044

warning: src/cmd/8c/list.c:124 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:134 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:142 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:152 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:156 format mismatch d VLONG, arg 4
warning: src/cmd/8c/list.c:160 format mismatch d VLONG, arg 4
warning: src/cmd/8c/list.c:165 format mismatch d VLONG, arg 4
warning: src/cmd/8c/list.c:167 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:172 format mismatch d VLONG, arg 4
warning: src/cmd/8c/list.c:174 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:178 format mismatch d VLONG, arg 3
warning: src/cmd/8c/list.c:184 format mismatch d VLONG, arg 3

warning: src/cmd/8g/list.c:91 format mismatch d VLONG, arg 4
warning: src/cmd/8g/list.c:100 format mismatch d VLONG, arg 4
warning: src/cmd/8g/list.c:114 format mismatch d VLONG, arg 5
warning: src/cmd/8g/list.c:118 format mismatch d VLONG, arg 5
warning: src/cmd/8g/list.c:122 format mismatch d VLONG, arg 5
warning: src/cmd/8g/list.c:126 format mismatch d VLONG, arg 5
warning: src/cmd/8g/list.c:136 format mismatch d VLONG, arg 4

warning: src/cmd/8l/list.c:107 format mismatch d VLONG, arg 4
warning: src/cmd/8l/list.c:125 format mismatch ux VLONG, arg 4
warning: src/cmd/8l/list.c:128 format mismatch ux VLONG, arg 4
warning: src/cmd/8l/list.c:130 format mismatch d VLONG, arg 4
warning: src/cmd/8l/list.c:134 format mismatch d VLONG, arg 5
warning: src/cmd/8l/list.c:138 format mismatch d VLONG, arg 6
warning: src/cmd/8l/list.c:143 format mismatch d VLONG, arg 5
warning: src/cmd/8l/list.c:148 format mismatch d VLONG, arg 5
warning: src/cmd/8l/list.c:150 format mismatch d VLONG, arg 4
warning: src/cmd/8l/list.c:154 format mismatch d VLONG, arg 4
warning: src/cmd/8l/list.c:158 format mismatch d VLONG, arg 4
warning: src/cmd/8l/obj.c:132 format mismatch ux VLONG, arg 2

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/39710043

Fixes a regression introduced in revision 4cb93e2900d0.

That revision changed runtime·memmove to use SSE MOVOU
instructions for sizes between 17 and 256 bytes. We were
using memmove to save a copy of the note string during
the note handler. The Plan 9 kernel does not allow the
use of floating point in note handlers (which includes
MOVOU since it touches the XMM registers).

Arguably, runtime·memmove should not be using MOVOU when
GO386=387 but that wouldn't help us on amd64. It's very
important that we guard against any future changes so we
use a simple copy loop instead.

This change is extracted from CL 9796043 (since that CL
is still being ironed out).

R=rsc
CC=golang-dev
https://golang.org/cl/34640045

R=golang-dev, minux.ma, iant
CC=golang-dev
https://golang.org/cl/36300045

R=jas, lucio.dere, rsc
CC=golang-dev
https://golang.org/cl/14604047

The funcdata symbol incorrectly named the dead value map the
dead pointer map.  The dead value map identifies all dead
values, including pointers and non-pointers, in a stack frame.
The purpose of this map is to allow the runtime to poison
locations of dead data to catch lost invariants.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/38670043

TBR=iant
CC=golang-dev
https://golang.org/cl/39540043

TBR=iant
CC=golang-dev
https://golang.org/cl/39530043

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/36300044

TBR=iant
CC=golang-dev
https://golang.org/cl/39510043

TBR=iant
CC=golang-dev
https://golang.org/cl/39400044

That option turns off word wrapping of individual
error messages generated by clang. The wrapping
makes the errors harder to read and conflicts with the
idea of a terminal window that can be resized.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/35810043

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/39280043

The new linker will disallow this on arm
(it is already disallowed on amd64 and 386)
in order to be able to lay out each function
separately.

The restriction is only for jumps into the middle
of a function; jumps to the beginning of a function
remain fine.

Prereq for linker cleanup (golang.org/s/go13linker).

R=iant, r, minux.ma
CC=golang-dev
https://golang.org/cl/35800043

Preparation for golang.org/s/go13linker work.

This CL does not build by itself. It depends on 35740044
and 35790044 and will be submitted at the same time.

R=iant
CC=golang-dev
https://golang.org/cl/34590045

Preparation for golang.org/s/go13linker work.

This CL does not build by itself. It depends on 35740044
and 35790044 and will be submitted at the same time.

R=iant
CC=golang-dev
https://golang.org/cl/34580044

Preparation for golang.org/s/go13linker work.

This CL does not build by itself. It depends on 35740044
and 35790044 and will be submitted at the same time.

R=iant
CC=golang-dev
https://golang.org/cl/35830043

There is an enormous amount of code moving around in this CL,
but the code is the same, and it is invoked in the same ways.
This CL is preparation for the new linker structure, not the new
structure itself.

The new library's definition is in include/link.h.

The main change is the use of a Link structure to hold all the
linker-relevant state, replacing the smattering of global variables.
The Link structure should both make it clearer which state must
be carried around and make it possible to parallelize more easily
later.

The main body of the linker has moved into the architecture-independent
cmd/ld directory. That includes the list of known header types, so the
distinction between Hplan9x32 and Hplan9x64 is removed (no other
header type distinguished 32- and 64-bit formats), and code for unused
formats such as ipaq kernels has been deleted.

The code being deleted from 5l, 6l, and 8l reappears in liblink or in ld.
Because multiple files are being merged in the liblink directory,
it is not possible to show the diffs nicely in hg.

The Prog and Addr structures have been unified into an
architecture-independent form and moved to link.h, where they will
be shared by all tools: the assemblers, the compilers, and the linkers.
The unification makes it possible to write architecture-independent
traversal of Prog lists, among other benefits.

The Sym structures cannot be unified: they are too fundamentally
different between the linker and the compilers. Instead, liblink defines
an LSym - a linker Sym - to be used in the Prog and Addr structures,
and the linker now refers exclusively to LSyms. The compilers will
keep using their own syms but will fill out the corresponding LSyms in
the Prog and Addr structures.

Although code from 5l, 6l, and 8l is now in a single library, the
code has been arranged so that only one architecture needs to
be linked into a particular program: 5l will not contain the code
needed for x86 instruction layout, for example.

The object file writing code in liblink/obj.c is from cmd/gc/obj.c.

Preparation for golang.org/s/go13linker work.

This CL does not build by itself. It depends on 35740044
and will be submitted at the same time.

R=iant
CC=golang-dev
https://golang.org/cl/35790044

In addition to adding the library, change the way the anames array is created.
Previously, it was written to src/cmd/6l/enam.c (and similarly for 5l and 8l)
and each of the other tools (6g, 6c, 6a) compiled the 6l/enam.c file in addition
to their own sources.

Now that there is a library shared by all these programs, move the anames
array into that library. To eliminate name conflicts, name the array after
the architecture letter: anames5, anames6, anames8.

First step to linker cleanup (golang.org/s/go13linker).

This CL does not build by itself. It depends on the CLs introducing
liblink and changing commands to use it.

R=iant
CC=golang-dev
https://golang.org/cl/35740044

We are not clearing dead values in the garbage collector so it
is not worth the RSS cost to materialize the data and write it
out to the binary.

R=golang-dev, iant, cshapiro
CC=golang-dev
https://golang.org/cl/38650043

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/38640043

When enabled this new debugging mode will allocate objects on
their own page and never recycle memory addresses.  This is an
essential tool to root cause a broad class of heap corruption.

R=golang-dev, dave, daniel.morsing, dvyukov, rsc, iant, cshapiro
CC=golang-dev
https://golang.org/cl/22060046

R=iant
CC=golang-dev
https://golang.org/cl/38560043

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/38590043

R=golang-dev
CC=golang-dev
https://golang.org/cl/38130043

cmd/5g, cmd/5l, cmd/6g, cmd/6l, cmd/8g, cmd/8l, cmd/gc, runtime: generate pointer maps by liveness analysis

This change allows the garbage collector to examine stack
slots that are determined as live and containing a pointer
value by the garbage collector.  This results in a mean
reduction of 65% in the number of stack slots scanned during
an invocation of "GOGC=1 all.bash".

Unfortunately, this does not yet allow garbage collection to
be precise for the stack slots computed as live.  Pointers
confound the determination of what definitions reach a given
instruction.  In general, this problem is not solvable without
runtime cost but some advanced cooperation from the compiler
might mitigate common cases.

R=golang-dev, rsc, cshapiro
CC=golang-dev
https://golang.org/cl/14430048

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/31900043

Nothing significant so far (right?), but let's lay the keystone.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/36830043

R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/23320044

Output for an allocation and free (sweep) follows

MProf_Malloc(p=0xc2100210a0, size=0x50, type=0x0 <single object>)
        #0 0x46ee15 runtime.mallocgc /usr/local/google/home/cshapiro/go/src/pkg/runtime/malloc.goc:141
        #1 0x47004f runtime.settype_flush /usr/local/google/home/cshapiro/go/src/pkg/runtime/malloc.goc:612
        #2 0x45f92c gc /usr/local/google/home/cshapiro/go/src/pkg/runtime/mgc0.c:2071
        #3 0x45f89e mgc /usr/local/google/home/cshapiro/go/src/pkg/runtime/mgc0.c:2050
        #4 0x45258b runtime.mcall /usr/local/google/home/cshapiro/go/src/pkg/runtime/asm_amd64.s:179

MProf_Free(p=0xc2100210a0, size=0x50)
        #0 0x46ee15 runtime.mallocgc /usr/local/google/home/cshapiro/go/src/pkg/runtime/malloc.goc:141
        #1 0x47004f runtime.settype_flush /usr/local/google/home/cshapiro/go/src/pkg/runtime/malloc.goc:612
        #2 0x45f92c gc /usr/local/google/home/cshapiro/go/src/pkg/runtime/mgc0.c:2071
        #3 0x45f89e mgc /usr/local/google/home/cshapiro/go/src/pkg/runtime/mgc0.c:2050
        #4 0x45258b runtime.mcall /usr/local/google/home/cshapiro/go/src/pkg/runtime/asm_amd64.s:179

R=golang-dev, dvyukov, rsc, cshapiro
CC=golang-dev
https://golang.org/cl/21990045

Shaves 1% off of binary size.

update #6853

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/35940047

This change reduces the cost of the stack scanning by frames.
It moves the stack scanning from the serial root enumeration
phase to the parallel tracing phase.  The output that follows
are timings for the issue 6482 benchmark

Baseline

BenchmarkGoroutineSelect	      50	 108027405 ns/op
BenchmarkGoroutineBlocking	      50	  89573332 ns/op
BenchmarkGoroutineForRange	      20	  95614116 ns/op
BenchmarkGoroutineIdle		      20	 122809512 ns/op

Stack scan by frames, non-parallel

BenchmarkGoroutineSelect	      20	 297138929 ns/op
BenchmarkGoroutineBlocking	      20	 301137599 ns/op
BenchmarkGoroutineForRange	      10	 312499469 ns/op
BenchmarkGoroutineIdle		      10	 209428876 ns/op

Stack scan by frames, parallel

BenchmarkGoroutineSelect	      20	 183938431 ns/op
BenchmarkGoroutineBlocking	      20	 170109999 ns/op
BenchmarkGoroutineForRange	      20	 179628882 ns/op
BenchmarkGoroutineIdle		      20	 157541498 ns/op

The remaining performance disparity is due to inefficiencies
in gentraceback and its callees.  The effect was isolated by
using a parallel stack scan where scanstack was modified to do
a conservative scan of the stack segments without gentraceback
followed by a call of gentrackback with a no-op callback.

The output that follows are the top-10 most frequent tops of
stacks as determined by the Linux perf record facility.

Baseline

+  25.19%  gc.test  gc.test            [.] runtime.xchg
+  19.00%  gc.test  gc.test            [.] scanblock
+   8.53%  gc.test  gc.test            [.] scanstack
+   8.46%  gc.test  gc.test            [.] flushptrbuf
+   5.08%  gc.test  gc.test            [.] procresize
+   3.57%  gc.test  gc.test            [.] runtime.chanrecv
+   2.94%  gc.test  gc.test            [.] dequeue
+   2.74%  gc.test  gc.test            [.] addroots
+   2.25%  gc.test  gc.test            [.] runtime.ready
+   1.33%  gc.test  gc.test            [.] runtime.cas64

Gentraceback

+  18.12%  gc.test  gc.test             [.] runtime.xchg
+  14.68%  gc.test  gc.test             [.] scanblock
+   8.20%  gc.test  gc.test             [.] runtime.gentraceback
+   7.38%  gc.test  gc.test             [.] flushptrbuf
+   6.84%  gc.test  gc.test             [.] scanstack
+   5.92%  gc.test  gc.test             [.] runtime.findfunc
+   3.62%  gc.test  gc.test             [.] procresize
+   3.15%  gc.test  gc.test             [.] readvarint
+   1.92%  gc.test  gc.test             [.] addroots
+   1.87%  gc.test  gc.test             [.] runtime.chanrecv

R=golang-dev, dvyukov, rsc
CC=golang-dev
https://golang.org/cl/17410043

Pass as a slice of strings instead.  For 2-5 strings, implement
dedicated routines so no slices are needed.

static call counts in the go binary:
 2 strings: 342 occurrences
 3 strings:  98
 4 strings:  30
 5 strings:  13
6+ strings:  14

Why?  C varags, bad for stack scanning and copying.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/36380043

Now that the map implementation is reading the keys and values from
arbitrary memory (instead of from stack slots), it needs to tell the
race detector when it does so.

Fixes #6875.

R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/36360043

Update #6876.

R=dave, bradfitz
CC=golang-dev
https://golang.org/cl/36370043

zero object allocated, so we still need to allocate a new
zero area every time.

Fixes #6876.

R=golang-dev
CC=golang-dev
https://golang.org/cl/36320043

and methodValueCall directly.  Instead, we inline their behavior
inside of reflect.call.

This change is required because otherwise we have a situation where
reflect.callXX calls makeFuncStub, neither of which knows the
layout of the args passed between them.  That's bad for
precise gc & stack copying.

Fixes #6619.

R=golang-dev, dvyukov, rsc, iant, khr
CC=golang-dev
https://golang.org/cl/26970044

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/28860043

This change is part of the plan to get rid of all vararg C calls
which are a pain for getting exact stack scanning.

We allocate a chunk of zero memory to return a pointer to when a
map access doesn't find the key.  This is simpler than returning nil
and fixing things up in the caller.  Linker magic allocates a single
zero memory area that is shared by all (non-reflect-generated) map
types.

Passing things by reference gets rid of some copies, so it speeds
up code with big keys/values.

benchmark             old ns/op    new ns/op    delta
BenchmarkBigKeyMap           34           31   -8.48%
BenchmarkBigValMap           37           30  -18.62%
BenchmarkSmallKeyMap         26           23  -11.28%

R=golang-dev, dvyukov, khr, rsc
CC=golang-dev
https://golang.org/cl/14794043