This change include updates to the probeIPv4Stack
and probeIPv6Stack to ensure that one or both
protocols are supported by ip(3).
The addition of fdMutex to netFD fixes the
TestTCPConcurrentAccept failures.
Additional changes add support for keepalive.

R=golang-codereviews, 0intro
CC=golang-codereviews, rsc
https://golang.org/cl/49920048

Fixes #6546.

LGTM=dave, bradfitz, r
R=r, dave, bradfitz
CC=golang-codereviews
https://golang.org/cl/55780043

warning: /usr/go/src/cmd/gc/obj.c:23 format mismatch -10d VLONG, arg 9
warning: /usr/go/src/cmd/gc/plive.c:1680 set and not used: printed
warning: /usr/go/src/cmd/ld/lib.c:332 non-interruptable temporary
warning: /usr/go/src/cmd/ld/lib.c:338 non-interruptable temporary

R=rsc
CC=golang-codereviews
https://golang.org/cl/53820046

R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/55720044

Linux Flock_t, Unix FcntlFlock, and various BSD Termios.

R=golang-codereviews, dsymonds
CC=golang-codereviews
https://golang.org/cl/55460043

Unbreaks the build.

R=golang-codereviews, dsymonds
CC=golang-codereviews
https://golang.org/cl/53820045

R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/55720043

Doesn't really matter for the most part, since the runtime-integrated
network poller uses its own kevent implementation, but for people using
the syscall directly, we should use an unsafe.Pointer for the precise GC
to retain the pointer arguments.

Also push down unsafe.Pointer a bit further in exec_linux.go, not
that there are any GC preemption points in the middle and sys
is still live anyway.

R=golang-codereviews, dvyukov
CC=golang-codereviews, iant
https://golang.org/cl/55520043

Place && at the end of line.
Offset expression continuation.

R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/55380044

Introduces two-phase goroutine parking mechanism -- prepare to park, commit park.
This mechanism does not require backing mutex to protect wait predicate.
Use it in netpoll. See comment in netpoll.goc for details.
This slightly reduces contention between reader, writer and read/write io notifications;
and just eliminates a bunch of mutex operations from hotpaths, thus making then faster.

benchmark                             old ns/op    new ns/op    delta
BenchmarkTCP4ConcurrentReadWrite           2109         1945   -7.78%
BenchmarkTCP4ConcurrentReadWrite-2         1162         1113   -4.22%
BenchmarkTCP4ConcurrentReadWrite-4          798          755   -5.39%
BenchmarkTCP4ConcurrentReadWrite-8          803          748   -6.85%
BenchmarkTCP4Persistent                    9411         9240   -1.82%
BenchmarkTCP4Persistent-2                  5888         5813   -1.27%
BenchmarkTCP4Persistent-4                  4016         3968   -1.20%
BenchmarkTCP4Persistent-8                  3943         3857   -2.18%

R=golang-codereviews, mikioh.mikioh, gobot, iant, rsc
CC=golang-codereviews, khr
https://golang.org/cl/45700043

The new select tests currently fail (the race is not detected).

R=khr
CC=golang-codereviews
https://golang.org/cl/54220043

Helps reduce contention on sched.goidgen.

benchmark                               old ns/op    new ns/op    delta
BenchmarkCreateGoroutines-16                  259          237   -8.49%
BenchmarkCreateGoroutinesParallel-16          127           43  -66.06%

R=golang-codereviews, dave, bradfitz, khr
CC=golang-codereviews, rsc
https://golang.org/cl/46970043

- do not lose profiling signals when we have no mcache (possible for syscalls/cgo)
- do not lose any profiling signals on windows
- fix profiling of cgo programs on windows (they had no m->thread setup)
- properly setup tls in cgo programs on windows
- check _beginthread return value

Fixes #6417.
Fixes #6986.

R=alex.brainman, rsc
CC=golang-codereviews
https://golang.org/cl/44820047

R=rsc, r, 0intro
CC=golang-codereviews
https://golang.org/cl/36060047

Fixes #6295.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/20850043

Fixes #6572.

LGTM=rsc, daniel.morsing, rsc
R=golang-codereviews, bradfitz, minux.ma, iant, rsc, gobot, daniel.morsing
CC=golang-codereviews
https://golang.org/cl/14516055

In particular: setsockopt, getsockopt, bind, connect.

There are probably more.

All platforms cross-compile with make.bash, and all.bash still
pases on linux/amd64.

Update #7169

R=rsc
CC=golang-codereviews
https://golang.org/cl/55410043

Fixes #7152.

R=iant, rsc, gobot
CC=golang-codereviews
https://golang.org/cl/54790044

Now that liblink is compiled into the compilers and assemblers,
it must not refer to the "linkmode", since that is not known until
link time. This CL makes the ARM support no longer use linkmode,
which fixes a bug with cgo binaries that contain their own TLS
variables.

The x86 code must also remove linkmode; that is issue 7164.

Fixes #6992.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/55160043

Now it should always fail on ARM.
(The fix is on its way too.)

R=iant, r, dave
CC=golang-codereviews
https://golang.org/cl/55140043

Update #7059

R=rsc
CC=golang-codereviews
https://golang.org/cl/55370043

Fixes #7059

R=golang-codereviews, iant, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/53470043

R=golang-codereviews, iant, khr, dvyukov
CC=golang-codereviews
https://golang.org/cl/54160043

Sorry, "hg status" hides .6 files by default.
We should probably fix that.

TBR=iant
CC=golang-codereviews
https://golang.org/cl/55290043

R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/55140044

R=iant
CC=golang-codereviews
https://golang.org/cl/53820043

The AKILL stuff is not ready yet and
was not meant to be in the last CL.

R=iant
CC=golang-codereviews
https://golang.org/cl/55170043

The escape analysis works by tracing assignment paths from
variables that start with pointer type, or addresses of variables
(addresses are always pointers).  It does allow non-pointers
in the path, so that in this code it sees x's value escape into y:

        var x *[10]int
        y := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(x))+32))

It must allow uintptr in order to see through this kind of
"pointer arithmetic".

It also traces such values if they end up as uintptrs passed to
functions. This used to be important because packages like
encoding/gob passed around uintptrs holding real pointers.

The introduction of precise collection of stacks has forced
code to be more honest about which declared stack variables
hold pointers and which do not. In particular, the garbage
collector no longer sees pointers stored in uintptr variables.
Because of this, packages like encoding/gob have been fixed.

There is not much point in the escape analysis accepting
uintptrs as holding pointers at call boundaries if the garbage
collector does not.

Excluding uintptr-valued arguments brings the escape
analysis in line with the garbage collector and has the
useful side effect of making arguments to syscall.Syscall
not appear to escape.

That is, this CL should yield the same benefits as
CL 45930043 (rolled back in CL 53870043), but it does
so by making uintptrs less special, not more.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53940043

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53930043

R=golang-codereviews, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/53950043

Many calls to symgrow pass a vlong value.  Change the function
to not implicitly truncate, and to instead give an error if
the value is too large.

R=golang-codereviews, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/54010043

Currently we collect (add) all roots into a global array in a single-threaded GC phase.
This hinders parallelism.
With this change we just kick off parallel for for number_of_goroutines+5 iterations.
Then parallel for callback decides whether it needs to scan stack of a goroutine
scan data segment, scan finalizers, etc. This eliminates the single-threaded phase entirely.
This requires to store all goroutines in an array instead of a linked list
(to allow direct indexing).
This CL also removes DebugScan functionality. It is broken because it uses
unbounded stack, so it can not run on g0. When it was working, I've found
it helpless for debugging issues because the two algorithms are too different now.
This change would require updating the DebugScan, so it's simpler to just delete it.

With 8 threads this change reduces GC pause by ~6%, while keeping cputime roughly the same.

garbage-8
allocated                 2987886      2989221      +0.04%
allocs                      62885        62887      +0.00%
cputime                  21286000     21272000      -0.07%
gc-pause-one             26633247     24885421      -6.56%
gc-pause-total             873570       811264      -7.13%
rss                     242089984    242515968      +0.18%
sys-gc                   13934336     13869056      -0.47%
sys-heap                205062144    205062144      +0.00%
sys-other                12628288     12628288      +0.00%
sys-stack                11534336     11927552      +3.41%
sys-total               243159104    243487040      +0.13%
time                      2809477      2740795      -2.44%

R=golang-codereviews, rsc
CC=cshapiro, golang-codereviews, khr
https://golang.org/cl/46860043

It's entirely outdated today.

R=golang-codereviews, bradfitz, gobot, r
CC=golang-codereviews
https://golang.org/cl/43500045

Instead of a per-goroutine stack of defers for all sizes,
introduce per-P defer pool for argument sizes 8, 24, 40, 56, 72 bytes.

For a program that starts 1e6 goroutines and then joins then:
old: rss=6.6g virtmem=10.2g time=4.85s
new: rss=4.5g virtmem= 8.2g time=3.48s

R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/42750044

R=golang-codereviews, dvyukov, khr
CC=golang-codereviews
https://golang.org/cl/54060043

Currently for 2-word blocks we set the flag to clear the flag. Makes no sense.
In particular on 32-bits we call memclr always.

R=golang-codereviews, dave, iant
CC=golang-codereviews, khr, rsc
https://golang.org/cl/41170044

The deadlock is between span->specialLock and proflock:

goroutine 11 [running]:
runtime.MProf_Free(0x7fa272d26508, 0xc210054180, 0xc0)
        src/pkg/runtime/mprof.goc:220 +0x27
runtime.freespecial(0x7fa272d1e088, 0xc210054180, 0xc0)
        src/pkg/runtime/mheap.c:691 +0x6a
runtime.freeallspecials(0x7fa272d1af50, 0xc210054180, 0xc0)
        src/pkg/runtime/mheap.c:717 +0xb5
runtime.free(0xc210054180)
        src/pkg/runtime/malloc.goc:190 +0xfd
selectgo(0x7fa272a5ef58)
        src/pkg/runtime/chan.c:1136 +0x2d8
runtime.selectgo(0xc210054180)
        src/pkg/runtime/chan.c:840 +0x12
runtime_test.func·058()
        src/pkg/runtime/proc_test.go:146 +0xb4
runtime.goexit()
        src/pkg/runtime/proc.c:1405
created by runtime_test.TestTimerFairness
        src/pkg/runtime/proc_test.go:152 +0xd1

goroutine 12 [running]:
addspecial(0xc2100540c0, 0x7fa272d1e0a0)
        src/pkg/runtime/mheap.c:569 +0x88
runtime.setprofilebucket(0xc2100540c0, 0x7fa272d26508)
        src/pkg/runtime/mheap.c:668 +0x73
runtime.MProf_Malloc(0xc2100540c0, 0xc0, 0x0)
        src/pkg/runtime/mprof.goc:212 +0x16b
runtime.mallocgc(0xc0, 0x0, 0xc200000000)
        src/pkg/runtime/malloc.goc:142 +0x239
runtime.mal(0xbc)
        src/pkg/runtime/malloc.goc:703 +0x38
newselect(0x2, 0x7fa272a5cf60)
        src/pkg/runtime/chan.c:632 +0x53
runtime.newselect(0xc200000002, 0xc21005f000)
        src/pkg/runtime/chan.c:615 +0x28
runtime_test.func·058()
        src/pkg/runtime/proc_test.go:146 +0x37
runtime.goexit()
        src/pkg/runtime/proc.c:1405
created by runtime_test.TestTimerFairness
        src/pkg/runtime/proc_test.go:152 +0xd1

Fixes #7099.

R=golang-codereviews, khr
CC=golang-codereviews
https://golang.org/cl/53120043

The test prints an excessive \n when /dev/null is not present.

R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/54890043

Fixes #7157.

R=alex.brainman, bradfitz
CC=golang-codereviews
https://golang.org/cl/54880043

What was happenning is as follows:
Each writer goroutine always triggers GC during its scheduling quntum.
After GC goroutines are shuffled so that the timer goroutine is always second in the queue.
This repeats infinitely, causing timer goroutine starvation.
Fixes #7126.

R=golang-codereviews, shanemhansen, khr, khr
CC=golang-codereviews
https://golang.org/cl/53080043