This is joint work with Daniel Morsing.

In order for the register allocator to alias two variables, they must have the same width, stack offset, and etype. Code generation was altering a variable's etype in a few places. This prevented the variable from being moved to a register, which in turn prevented peephole optimization. This failure to alias was very common, with almost 23,000 instances just running make.bash.

This phenomenon was not visible in the register allocation debug output because the variables that failed to alias had the same name. The debugging-only change to bits.c fixes this by printing the variable number with its name.

This CL fixes the source of all etype mismatches for 6g, all but one case for 8g, and depressingly few cases for 5g. (I believe that extending CL 6819083 to 5g is a prerequisite.) Fixing the remaining cases in 8g and 5g is work for the future.

The etype mismatch fixes are:

* [gc] Slicing changed the type of the base pointer into a uintptr in order to perform arithmetic on it. Instead, support addition directly on pointers.

* [*g] OSPTR was giving type uintptr to slice base pointers; undo that. This arose, for example, while compiling copy(dst, src).

* [8g] 64 bit float conversion was assigning int64 type during codegen, overwriting the existing uint64 type.

Note that some etype mismatches are appropriate, such as a struct with a single field or an array with a single element.

With these fixes, the number of registerizations that occur while running make.bash for 6g increases ~10%. Hello world binary size shrinks ~1.5%. Running all benchmarks in the standard library show performance improvements ranging from nominal to substantive (>10%); a full comparison using 6g on my laptop is available at https://gist.github.com/josharian/8f9b5beb46667c272064. The microbenchmarks must be taken with a grain of salt; see issue 7920. The few benchmarks that show real regressions are likely due to issue 7920. I manually examined the generated code for the top few regressions and none had any assembly output changes. The few benchmarks that show extraordinary improvements are likely also due to issue 7920.

Performance results from 8g appear similar to 6g.

5g shows no performance improvements. This is not surprising, given the discussion above.

Update #7316

LGTM=rsc
R=rsc, daniel.morsing, bradfitz
CC=dave, golang-codereviews
https://golang.org/cl/91850043

The runtime was detecting the cycle already,
but we can give a better error without even
building the binary.

Fixes #7789.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/96290043

This change requires using SWIG version 3.0 or later.  Earlier
versions of SWIG do not generate the pragmas required to use
the external linker.

Fixes #7155.
Fixes #7156.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/97120046

Fixes #7931.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/100390044

Prevent idle transport on race condition.

Fixes #7847

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/96230044

Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/100390045

Before we used line 1 of the first source file.
This should be clearer.

Fixes #4388.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/92250044

<enter reason for undo>

««« original CL description
net: make use of SO_LINGER_SEC on darwin

Fixes #7971.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/92210044
»»»

TBR=iant
R=golang-codereviews
CC=golang-codereviews
https://golang.org/cl/96220049

Fixes #7971.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/92210044

Fixes #7915.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/96210044

If it's not used (such as on other systems or if softfloat
is disabled) the linker will discard it.

The alternative is to teach cmd/go that every binary
depends on math implicitly on arm. I started down that
path but it's too scary. If we're going to get dependencies
right we should get dependencies right.

Fixes #6994.

LGTM=bradfitz, dave
R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/95290043

Most code is copy from addr2line change 01dd67e5827f

Update #7406
Fixes #7937

LGTM=iant
R=golang-codereviews, iant, 0intro
CC=golang-codereviews
https://golang.org/cl/95090044

LGTM=robert.hencke, iant
R=golang-codereviews, robert.hencke, iant
CC=golang-codereviews
https://golang.org/cl/96230043

LGTM=bradfitz
R=bradfitz, 0intro
CC=golang-codereviews
https://golang.org/cl/95280043

Fixes #6897.

LGTM=bradfitz
R=golang-codereviews, bradfitz, r, rsc
CC=golang-codereviews
https://golang.org/cl/91230045

LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/99180043

This is a trivial change to make use of an existing `nl` byte slice
containing a single '\n' character. It's already declared and
used in another place in this file, so it might as well be used
in the other location instead of
a new slice literal. There should be no change in behavior,
aside from potentially less allocations.

This is my first CL, so I wanted to use a simple, hopefully non-controversial,
minor improvement to get more comfortable with golang contribution process.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/97280043

LGTM=minux.ma
R=cespare, rsc, minux.ma
CC=golang-codereviews
https://golang.org/cl/96210043

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/91250043

<enter reason for undo>

««« original CL description
runtime/race: fix the link for the race detector.

LGTM=bradfitz
R=golang-dev, bradfitz
CC=golang-codereviews
https://golang.org/cl/100330043

»»»

TBR=minux
R=minux.ma
CC=golang-codereviews
https://golang.org/cl/96200044

LGTM=bradfitz
R=golang-dev, bradfitz
CC=golang-codereviews
https://golang.org/cl/100330043

Fixes #7943

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/98170043

Fixes #7816.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/98160043

LGTM=mikioh.mikioh
R=golang-codereviews, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/94240045

1. fix executable extension (a.out -> a.exe).
2. fix pthread build error on mingw
3. if depends lib messing, skip the test

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/100210043

Add a test for the current behaviour.

Fixes #7482.

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews
https://golang.org/cl/95160043

list has been adding them one at a time haphazardly
(race and tags were there and documented; compiler
was there and undocumented).

clean -i needs -compiler in order to clean the
installed targets for alternate compilers.

Fixes #7302.

While we're here, tweak the language in the 'go get' docs
about build flags.

Fixes #7807.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/99130043

Fixes #7851.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/93200043

If you write:

        var x = 3

then the compiler arranges for x to be initialized in the linker
with an actual 3 from the data segment, rather than putting
x in the bss and emitting init-time "x = 3" assignment code.

If you write:

        var y = x
        var x = 3

then the compiler is clever and treats this the same as if
the code said 'y = 3': they both end up in the data segment
with no init-time assignments.

If you write

        var y = x
        var x int

then the compiler was treating this the same as if the
code said 'x = 0', making both x and y zero and avoiding
any init-time assignment.

This copying optimization to avoid init-time assignment of y
is incorrect if 'var x int' doesn't mean 'x = 0' but instead means
'x is initialized in C or assembly code'. The program ends up
with 'y = 0' instead of 'y = the value specified for x in that other code'.

Disable the propagation if there is no initializer for x.

This comes up in some uses of cgo, because cgo generates
Go globals that are initialized in accompanying C files.

Fixes #7665.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/93200044

Fixes #7928.

LGTM=bradfitz
R=golang-codereviews
CC=agl, bradfitz, golang-codereviews
https://golang.org/cl/91320043

If the ... element type contained no pointers,
then the escape analysis did not track the ... itself.
This manifested in an escaping ...byte being treated
as non-escaping.

Fixes #7934.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/100310043

The register allocator decides which variables should be placed into registers by charging for each load/store and crediting for each use, and then selecting an allocation with minimal cost. NOPs will be eliminated, however, so using a variable in a NOP should not generate credit.

Issue 7867 arises from attempted registerization of multi-word variables because they are used in NOPs. By not crediting for that use, they will no longer be considered for registerization.

This fix could theoretically lead to better register allocation, but NOPs are rare relative to other instructions.

Fixes #7867.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/94810044

Fixes #6844.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/97840043

Fixes #7901.

LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/90930044

TestDialFailPDLeak was created for testing runtime-integrated netwrok
poller stuff and used during Go 1.2 development cycle. Unfortunately
it's still flakey because it depends on MemStats of runtime, not
pollcache directly, and MemStats accounts and revises its own stats
occasionally.

For now the codepaths related to runtime-intergrated network poller
are pretty stable, so removing this test case never suffers us.

Fixes #6553.

LGTM=josharian, iant
R=iant, josharian
CC=golang-codereviews
https://golang.org/cl/98080043

LGTM=rsc
R=rsc, khr
CC=golang-codereviews
https://golang.org/cl/97010043

There was confusion in the behavior of json.Indent; This change
attempts to clarify the behavior by providing a bit more verbiage
to the documentation as well as provide an example function.

Fixes #7821.

LGTM=robert.hencke, adg
R=golang-codereviews, minux.ma, bradfitz, aram, robert.hencke, r, adg
CC=golang-codereviews
https://golang.org/cl/97840044

Fixes #7936.

LGTM=alex.brainman, bradfitz, iant
R=golang-codereviews, alex.brainman, bradfitz, iant
CC=golang-codereviews
https://golang.org/cl/100060043

Update #7188
Fixes #7894.

LGTM=alex.brainman, bradfitz, r
R=golang-codereviews, alex.brainman, mikioh.mikioh, gobot, r, bradfitz
CC=golang-codereviews
https://golang.org/cl/95870043

benchmark                 old ns/op     new ns/op     delta
BenchmarkCopyFat512       1307          329           -74.83%
BenchmarkCopyFat256       666           169           -74.62%
BenchmarkCopyFat1024      2617          671           -74.36%
BenchmarkCopyFat128       343           89.0          -74.05%
BenchmarkCopyFat64        182           48.9          -73.13%
BenchmarkCopyFat32        103           28.8          -72.04%
BenchmarkClearFat128      102           46.6          -54.31%
BenchmarkClearFat512      344           167           -51.45%
BenchmarkClearFat64       50.5          26.5          -47.52%
BenchmarkClearFat256      147           87.2          -40.68%
BenchmarkClearFat32       22.7          16.4          -27.75%
BenchmarkClearFat1024     511           662           +29.55%

Fixes #7624

LGTM=rsc
R=golang-codereviews, khr, bradfitz, josharian, dave, rsc
CC=golang-codereviews
https://golang.org/cl/92760044