The new select tests currently fail (the race is not detected).

R=khr
CC=golang-codereviews
https://golang.org/cl/54220043

Helps reduce contention on sched.goidgen.

benchmark                               old ns/op    new ns/op    delta
BenchmarkCreateGoroutines-16                  259          237   -8.49%
BenchmarkCreateGoroutinesParallel-16          127           43  -66.06%

R=golang-codereviews, dave, bradfitz, khr
CC=golang-codereviews, rsc
https://golang.org/cl/46970043

- do not lose profiling signals when we have no mcache (possible for syscalls/cgo)
- do not lose any profiling signals on windows
- fix profiling of cgo programs on windows (they had no m->thread setup)
- properly setup tls in cgo programs on windows
- check _beginthread return value

Fixes #6417.
Fixes #6986.

R=alex.brainman, rsc
CC=golang-codereviews
https://golang.org/cl/44820047

R=rsc, r, 0intro
CC=golang-codereviews
https://golang.org/cl/36060047

Fixes #6295.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/20850043

Fixes #6572.

LGTM=rsc, daniel.morsing, rsc
R=golang-codereviews, bradfitz, minux.ma, iant, rsc, gobot, daniel.morsing
CC=golang-codereviews
https://golang.org/cl/14516055

In particular: setsockopt, getsockopt, bind, connect.

There are probably more.

All platforms cross-compile with make.bash, and all.bash still
pases on linux/amd64.

Update #7169

R=rsc
CC=golang-codereviews
https://golang.org/cl/55410043

Fixes #7152.

R=iant, rsc, gobot
CC=golang-codereviews
https://golang.org/cl/54790044

Now that liblink is compiled into the compilers and assemblers,
it must not refer to the "linkmode", since that is not known until
link time. This CL makes the ARM support no longer use linkmode,
which fixes a bug with cgo binaries that contain their own TLS
variables.

The x86 code must also remove linkmode; that is issue 7164.

Fixes #6992.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/55160043

Now it should always fail on ARM.
(The fix is on its way too.)

R=iant, r, dave
CC=golang-codereviews
https://golang.org/cl/55140043

Update #7059

R=rsc
CC=golang-codereviews
https://golang.org/cl/55370043

Fixes #7059

R=golang-codereviews, iant, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/53470043

R=golang-codereviews, iant, khr, dvyukov
CC=golang-codereviews
https://golang.org/cl/54160043

Sorry, "hg status" hides .6 files by default.
We should probably fix that.

TBR=iant
CC=golang-codereviews
https://golang.org/cl/55290043

R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/55140044

R=iant
CC=golang-codereviews
https://golang.org/cl/53820043

The AKILL stuff is not ready yet and
was not meant to be in the last CL.

R=iant
CC=golang-codereviews
https://golang.org/cl/55170043

The escape analysis works by tracing assignment paths from
variables that start with pointer type, or addresses of variables
(addresses are always pointers).  It does allow non-pointers
in the path, so that in this code it sees x's value escape into y:

        var x *[10]int
        y := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(x))+32))

It must allow uintptr in order to see through this kind of
"pointer arithmetic".

It also traces such values if they end up as uintptrs passed to
functions. This used to be important because packages like
encoding/gob passed around uintptrs holding real pointers.

The introduction of precise collection of stacks has forced
code to be more honest about which declared stack variables
hold pointers and which do not. In particular, the garbage
collector no longer sees pointers stored in uintptr variables.
Because of this, packages like encoding/gob have been fixed.

There is not much point in the escape analysis accepting
uintptrs as holding pointers at call boundaries if the garbage
collector does not.

Excluding uintptr-valued arguments brings the escape
analysis in line with the garbage collector and has the
useful side effect of making arguments to syscall.Syscall
not appear to escape.

That is, this CL should yield the same benefits as
CL 45930043 (rolled back in CL 53870043), but it does
so by making uintptrs less special, not more.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53940043

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53930043

R=golang-codereviews, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/53950043

Many calls to symgrow pass a vlong value.  Change the function
to not implicitly truncate, and to instead give an error if
the value is too large.

R=golang-codereviews, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/54010043

Currently we collect (add) all roots into a global array in a single-threaded GC phase.
This hinders parallelism.
With this change we just kick off parallel for for number_of_goroutines+5 iterations.
Then parallel for callback decides whether it needs to scan stack of a goroutine
scan data segment, scan finalizers, etc. This eliminates the single-threaded phase entirely.
This requires to store all goroutines in an array instead of a linked list
(to allow direct indexing).
This CL also removes DebugScan functionality. It is broken because it uses
unbounded stack, so it can not run on g0. When it was working, I've found
it helpless for debugging issues because the two algorithms are too different now.
This change would require updating the DebugScan, so it's simpler to just delete it.

With 8 threads this change reduces GC pause by ~6%, while keeping cputime roughly the same.

garbage-8
allocated                 2987886      2989221      +0.04%
allocs                      62885        62887      +0.00%
cputime                  21286000     21272000      -0.07%
gc-pause-one             26633247     24885421      -6.56%
gc-pause-total             873570       811264      -7.13%
rss                     242089984    242515968      +0.18%
sys-gc                   13934336     13869056      -0.47%
sys-heap                205062144    205062144      +0.00%
sys-other                12628288     12628288      +0.00%
sys-stack                11534336     11927552      +3.41%
sys-total               243159104    243487040      +0.13%
time                      2809477      2740795      -2.44%

R=golang-codereviews, rsc
CC=cshapiro, golang-codereviews, khr
https://golang.org/cl/46860043

It's entirely outdated today.

R=golang-codereviews, bradfitz, gobot, r
CC=golang-codereviews
https://golang.org/cl/43500045

Instead of a per-goroutine stack of defers for all sizes,
introduce per-P defer pool for argument sizes 8, 24, 40, 56, 72 bytes.

For a program that starts 1e6 goroutines and then joins then:
old: rss=6.6g virtmem=10.2g time=4.85s
new: rss=4.5g virtmem= 8.2g time=3.48s

R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/42750044

R=golang-codereviews, dvyukov, khr
CC=golang-codereviews
https://golang.org/cl/54060043

Currently for 2-word blocks we set the flag to clear the flag. Makes no sense.
In particular on 32-bits we call memclr always.

R=golang-codereviews, dave, iant
CC=golang-codereviews, khr, rsc
https://golang.org/cl/41170044

The deadlock is between span->specialLock and proflock:

goroutine 11 [running]:
runtime.MProf_Free(0x7fa272d26508, 0xc210054180, 0xc0)
        src/pkg/runtime/mprof.goc:220 +0x27
runtime.freespecial(0x7fa272d1e088, 0xc210054180, 0xc0)
        src/pkg/runtime/mheap.c:691 +0x6a
runtime.freeallspecials(0x7fa272d1af50, 0xc210054180, 0xc0)
        src/pkg/runtime/mheap.c:717 +0xb5
runtime.free(0xc210054180)
        src/pkg/runtime/malloc.goc:190 +0xfd
selectgo(0x7fa272a5ef58)
        src/pkg/runtime/chan.c:1136 +0x2d8
runtime.selectgo(0xc210054180)
        src/pkg/runtime/chan.c:840 +0x12
runtime_test.func·058()
        src/pkg/runtime/proc_test.go:146 +0xb4
runtime.goexit()
        src/pkg/runtime/proc.c:1405
created by runtime_test.TestTimerFairness
        src/pkg/runtime/proc_test.go:152 +0xd1

goroutine 12 [running]:
addspecial(0xc2100540c0, 0x7fa272d1e0a0)
        src/pkg/runtime/mheap.c:569 +0x88
runtime.setprofilebucket(0xc2100540c0, 0x7fa272d26508)
        src/pkg/runtime/mheap.c:668 +0x73
runtime.MProf_Malloc(0xc2100540c0, 0xc0, 0x0)
        src/pkg/runtime/mprof.goc:212 +0x16b
runtime.mallocgc(0xc0, 0x0, 0xc200000000)
        src/pkg/runtime/malloc.goc:142 +0x239
runtime.mal(0xbc)
        src/pkg/runtime/malloc.goc:703 +0x38
newselect(0x2, 0x7fa272a5cf60)
        src/pkg/runtime/chan.c:632 +0x53
runtime.newselect(0xc200000002, 0xc21005f000)
        src/pkg/runtime/chan.c:615 +0x28
runtime_test.func·058()
        src/pkg/runtime/proc_test.go:146 +0x37
runtime.goexit()
        src/pkg/runtime/proc.c:1405
created by runtime_test.TestTimerFairness
        src/pkg/runtime/proc_test.go:152 +0xd1

Fixes #7099.

R=golang-codereviews, khr
CC=golang-codereviews
https://golang.org/cl/53120043

The test prints an excessive \n when /dev/null is not present.

R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/54890043

Fixes #7157.

R=alex.brainman, bradfitz
CC=golang-codereviews
https://golang.org/cl/54880043

What was happenning is as follows:
Each writer goroutine always triggers GC during its scheduling quntum.
After GC goroutines are shuffled so that the timer goroutine is always second in the queue.
This repeats infinitely, causing timer goroutine starvation.
Fixes #7126.

R=golang-codereviews, shanemhansen, khr, khr
CC=golang-codereviews
https://golang.org/cl/53080043

It's pretty distracting to use expvar with the output of both
the top-level map and map values jumping around randomly.

Also fixes a potential race where multiple clients trying to
increment a map int or float key at the same time could lose
updates.

R=golang-codereviews, couchmoney
CC=golang-codereviews
https://golang.org/cl/54320043

Use testing.AllocsPerRun now that it exists, instead of doing it by hand.

Fixes #6076

R=golang-codereviews, alex.brainman
CC=golang-codereviews
https://golang.org/cl/53810043

R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/54290043

R=r
CC=golang-codereviews
https://golang.org/cl/54040043

Fixes bug 7145

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53970043

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/54000043

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53690043

Vararg C calls present a problem for the GC because the
argument types are not derivable from the signature.  Remove
them by passing pointers to channel elements instead of the
channel elements directly.

R=golang-codereviews, gobot, rsc, dvyukov
CC=golang-codereviews
https://golang.org/cl/53430043

Fixes #7138.

R=r, bradfitz, dave
CC=dvyukov, golang-codereviews
https://golang.org/cl/53910043

The compiler change is an ugly hack.
We can do better.

««« original CL description
syscall: mark arguments to Syscall as noescape
Heap arguments to "async" syscalls will break when/if we have moving GC anyway.
With this change is must not break until moving GC, because a user must
reference the object in Go to preserve liveness. Otherwise the code is broken already.
Reduces number of leaked params from 125 to 36 on linux.

R=golang-codereviews, mikioh.mikioh, bradfitz
CC=cshapiro, golang-codereviews, khr, rsc
https://golang.org/cl/45930043
»»»

R=golang-codereviews, r
CC=bradfitz, dvyukov, golang-codereviews
https://golang.org/cl/53870043