Struct fields can be suppressed in JSON serialization by "-" tags, but
that doesn't preclude generation of "-" object keys.
Document and verify the mechanism for doing so.

Change-Id: I7f60e1759cfee15cb7b2447cd35fab91c5b004e6
Reviewed-on: https://go-review.googlesource.com/21204
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Deletes the following s390x instructions:

 - ADDME
 - ADDZE
 - SUBME
 - SUBZE

They appear to be emulated PPC instructions left over from the
porting process and I don't think they will ever be useful.

Change-Id: I9b1ba78019dbd1218d0c8f8ea2903878802d1990
Reviewed-on: https://go-review.googlesource.com/30538
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

    Use AVX if available on 4th generation of Intel(TM) Core(TM) processors.

    (collected on E5 2609v3 @1.9GHz)
    name                        old speed      new speed       delta
    Memmove/1-6                  158MB/s ± 0%    172MB/s ± 0%    +9.09% (p=0.000 n=16+16)
    Memmove/2-6                  316MB/s ± 0%    345MB/s ± 0%    +9.09% (p=0.000 n=18+16)
    Memmove/3-6                  517MB/s ± 0%    517MB/s ± 0%      ~ (p=0.445 n=16+16)
    Memmove/4-6                  687MB/s ± 1%    690MB/s ± 0%    +0.35% (p=0.000 n=20+17)
    Memmove/5-6                  729MB/s ± 0%    729MB/s ± 0%    +0.01% (p=0.000 n=16+18)
    Memmove/6-6                  875MB/s ± 0%    875MB/s ± 0%    +0.01% (p=0.000 n=18+18)
    Memmove/7-6                 1.02GB/s ± 0%   1.02GB/s ± 1%      ~ (p=0.139 n=19+20)
    Memmove/8-6                 1.26GB/s ± 0%   1.26GB/s ± 0%    +0.00% (p=0.000 n=18+18)
    Memmove/9-6                 1.42GB/s ± 0%   1.42GB/s ± 0%    +0.00% (p=0.000 n=17+18)
    Memmove/10-6                1.58GB/s ± 0%   1.58GB/s ± 0%    +0.00% (p=0.000 n=19+19)
    Memmove/11-6                1.74GB/s ± 0%   1.74GB/s ± 0%    +0.00% (p=0.001 n=18+17)
    Memmove/12-6                1.90GB/s ± 0%   1.90GB/s ± 0%    +0.00% (p=0.000 n=19+19)
    Memmove/13-6                2.05GB/s ± 0%   2.05GB/s ± 0%    +0.00% (p=0.000 n=18+19)
    Memmove/14-6                2.21GB/s ± 0%   2.21GB/s ± 0%    +0.00% (p=0.000 n=16+20)
    Memmove/15-6                2.37GB/s ± 0%   2.37GB/s ± 0%    +0.00% (p=0.004 n=19+20)
    Memmove/16-6                2.53GB/s ± 0%   2.53GB/s ± 0%    +0.00% (p=0.000 n=16+16)
    Memmove/32-6                4.67GB/s ± 0%   4.67GB/s ± 0%    +0.00% (p=0.000 n=17+17)
    Memmove/64-6                8.67GB/s ± 0%   8.64GB/s ± 0%    -0.33% (p=0.000 n=18+17)
    Memmove/128-6               12.6GB/s ± 0%   11.6GB/s ± 0%    -8.05% (p=0.000 n=16+19)
    Memmove/256-6               16.3GB/s ± 0%   16.6GB/s ± 0%    +1.66% (p=0.000 n=20+18)
    Memmove/512-6               21.5GB/s ± 0%   24.4GB/s ± 0%   +13.35% (p=0.000 n=18+17)
    Memmove/1024-6              24.7GB/s ± 0%   33.7GB/s ± 0%   +36.12% (p=0.000 n=18+18)
    Memmove/2048-6              27.3GB/s ± 0%   43.3GB/s ± 0%   +58.77% (p=0.000 n=19+17)
    Memmove/4096-6              37.5GB/s ± 0%   50.5GB/s ± 0%   +34.56% (p=0.000 n=19+19)
    MemmoveUnalignedDst/1-6      135MB/s ± 0%    146MB/s ± 0%    +7.69% (p=0.000 n=16+14)
    MemmoveUnalignedDst/2-6      271MB/s ± 0%    292MB/s ± 0%    +7.69% (p=0.000 n=18+18)
    MemmoveUnalignedDst/3-6      438MB/s ± 0%    438MB/s ± 0%      ~ (p=0.352 n=16+19)
    MemmoveUnalignedDst/4-6      584MB/s ± 0%    584MB/s ± 0%      ~ (p=0.876 n=17+17)
    MemmoveUnalignedDst/5-6      631MB/s ± 1%    632MB/s ± 0%    +0.25% (p=0.000 n=20+17)
    MemmoveUnalignedDst/6-6      759MB/s ± 0%    759MB/s ± 0%    +0.00% (p=0.000 n=19+16)
    MemmoveUnalignedDst/7-6      885MB/s ± 0%    883MB/s ± 1%      ~ (p=0.647 n=18+20)
    MemmoveUnalignedDst/8-6     1.08GB/s ± 0%   1.08GB/s ± 0%    +0.00% (p=0.035 n=19+18)
    MemmoveUnalignedDst/9-6     1.22GB/s ± 0%   1.22GB/s ± 0%      ~ (p=0.251 n=18+17)
    MemmoveUnalignedDst/10-6    1.35GB/s ± 0%   1.35GB/s ± 0%      ~ (p=0.327 n=17+18)
    MemmoveUnalignedDst/11-6    1.49GB/s ± 0%   1.49GB/s ± 0%      ~ (p=0.531 n=18+19)
    MemmoveUnalignedDst/12-6    1.63GB/s ± 0%   1.63GB/s ± 0%      ~ (p=0.886 n=19+18)
    MemmoveUnalignedDst/13-6    1.76GB/s ± 0%   1.76GB/s ± 1%    -0.24% (p=0.006 n=18+20)
    MemmoveUnalignedDst/14-6    1.90GB/s ± 0%   1.90GB/s ± 0%      ~ (p=0.818 n=20+19)
    MemmoveUnalignedDst/15-6    2.03GB/s ± 0%   2.03GB/s ± 0%      ~ (p=0.294 n=17+16)
    MemmoveUnalignedDst/16-6    2.17GB/s ± 0%   2.17GB/s ± 0%      ~ (p=0.602 n=16+18)
    MemmoveUnalignedDst/32-6    4.05GB/s ± 0%   4.05GB/s ± 0%    +0.00% (p=0.010 n=18+17)
    MemmoveUnalignedDst/64-6    7.59GB/s ± 0%   7.59GB/s ± 0%    +0.00% (p=0.022 n=18+16)
    MemmoveUnalignedDst/128-6   11.1GB/s ± 0%   11.4GB/s ± 0%    +2.79% (p=0.000 n=18+17)
    MemmoveUnalignedDst/256-6   16.4GB/s ± 0%   16.7GB/s ± 0%    +1.59% (p=0.000 n=20+17)
    MemmoveUnalignedDst/512-6   15.7GB/s ± 0%   21.3GB/s ± 0%   +35.87% (p=0.000 n=18+20)
    MemmoveUnalignedDst/1024-6  16.0GB/s ±20%   31.5GB/s ± 0%   +96.93% (p=0.000 n=20+14)
    MemmoveUnalignedDst/2048-6  19.6GB/s ± 0%   42.1GB/s ± 0%  +115.16% (p=0.000 n=17+18)
    MemmoveUnalignedDst/4096-6  6.41GB/s ± 0%  33.18GB/s ± 0%  +417.56% (p=0.000 n=17+18)
    MemmoveUnalignedSrc/1-6      171MB/s ± 0%    166MB/s ± 0%    -3.33% (p=0.000 n=19+16)
    MemmoveUnalignedSrc/2-6      343MB/s ± 0%    342MB/s ± 1%    -0.41% (p=0.000 n=17+20)
    MemmoveUnalignedSrc/3-6      508MB/s ± 0%    493MB/s ± 1%    -2.90% (p=0.000 n=17+17)
    MemmoveUnalignedSrc/4-6      677MB/s ± 0%    660MB/s ± 2%    -2.55% (p=0.000 n=17+20)
    MemmoveUnalignedSrc/5-6      790MB/s ± 0%    790MB/s ± 0%      ~ (p=0.139 n=17+17)
    MemmoveUnalignedSrc/6-6      948MB/s ± 0%    946MB/s ± 1%      ~ (p=0.330 n=17+19)
    MemmoveUnalignedSrc/7-6     1.11GB/s ± 0%   1.11GB/s ± 0%    -0.05% (p=0.026 n=17+17)
    MemmoveUnalignedSrc/8-6     1.38GB/s ± 0%   1.38GB/s ± 0%      ~ (p=0.091 n=18+16)
    MemmoveUnalignedSrc/9-6     1.42GB/s ± 0%   1.40GB/s ± 1%    -1.04% (p=0.000 n=19+20)
    MemmoveUnalignedSrc/10-6    1.58GB/s ± 0%   1.56GB/s ± 1%    -1.15% (p=0.000 n=18+19)
    MemmoveUnalignedSrc/11-6    1.73GB/s ± 0%   1.71GB/s ± 1%    -1.30% (p=0.000 n=20+20)
    MemmoveUnalignedSrc/12-6    1.89GB/s ± 0%   1.87GB/s ± 1%    -1.18% (p=0.000 n=17+20)
    MemmoveUnalignedSrc/13-6    2.05GB/s ± 0%   2.02GB/s ± 1%    -1.18% (p=0.000 n=17+20)
    MemmoveUnalignedSrc/14-6    2.21GB/s ± 0%   2.18GB/s ± 1%    -1.14% (p=0.000 n=17+20)
    MemmoveUnalignedSrc/15-6    2.36GB/s ± 0%   2.34GB/s ± 1%    -1.04% (p=0.000 n=17+20)
    MemmoveUnalignedSrc/16-6    2.52GB/s ± 0%   2.49GB/s ± 1%    -1.26% (p=0.000 n=19+20)
    MemmoveUnalignedSrc/32-6    4.82GB/s ± 0%   4.61GB/s ± 0%    -4.40% (p=0.000 n=19+20)
    MemmoveUnalignedSrc/64-6    5.03GB/s ± 4%   7.97GB/s ± 0%   +58.55% (p=0.000 n=20+16)
    MemmoveUnalignedSrc/128-6   11.1GB/s ± 0%   11.2GB/s ± 0%    +0.52% (p=0.000 n=17+18)
    MemmoveUnalignedSrc/256-6   16.5GB/s ± 0%   16.4GB/s ± 0%    -0.10% (p=0.000 n=20+18)
    MemmoveUnalignedSrc/512-6   21.0GB/s ± 0%   22.1GB/s ± 0%    +5.48% (p=0.000 n=14+17)
    MemmoveUnalignedSrc/1024-6  24.9GB/s ± 0%   31.9GB/s ± 0%   +28.20% (p=0.000 n=19+20)
    MemmoveUnalignedSrc/2048-6  23.3GB/s ± 0%   33.8GB/s ± 0%   +45.22% (p=0.000 n=17+19)
    MemmoveUnalignedSrc/4096-6  37.3GB/s ± 0%   42.7GB/s ± 0%   +14.30% (p=0.000 n=17+17)

Change-Id: Id66aa3e499ccfb117cb99d623ef326b50d057b64
Reviewed-on: https://go-review.googlesource.com/29590
Run-TryBot: Denis Nagorny <denis.nagorny@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Adds the following instructions and uses them in the SSA backend:

 - ANDW
 - ORW
 - XORW

The instruction encodings for 32-bit operations are typically shorter,
particularly when an immediate is used. For example, XORW $-1, R1
only requires one instruction, whereas XOR requires two.

Also removes some unused instructions (that were emulated):

 - ANDN
 - NAND
 - ORN
 - NOR

Change-Id: Iff2a16f52004ba498720034e354be9771b10cac4
Reviewed-on: https://go-review.googlesource.com/30291Reviewed-by: Cherry Zhang <cherryyz@google.com>

Better to just rematerialize them when needed instead of
cross-register spilling or other techniques for keeping them in
registers.

This helps for amd64 code that does 1 << x. It is better to do
  loop:
    MOVQ $1, AX  // materialize arg to SLLQ
    SLLQ CX, AX
    ...
    goto loop
than to do
  MOVQ $1, AX    // materialize outsize of loop
  loop:
    MOVQ AX, DX  // save value that's about to be clobbered
    SLLQ CX, AX
    MOVQ DX, AX  // move it back to the correct register
    goto loop

Update #16092

Change-Id: If7ac290208f513061ebb0736e8a79dcb0ba338c0
Reviewed-on: https://go-review.googlesource.com/30471
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

The CgoExternalThreadSIGPROF test starts a thread at constructor time
that does a busy loop. That can throw off some other tests. So only
build that code if testprogcgo is built with the tag threadprof, and
adjust the tests that use that code to pass that build tag.

This revealed that the CgoPprofThread test was not testing what it
should have, as it never actually started the cpuHog thread. It was
passing because of the busy loop thread. Fix it to start the thread as
intended.

Change-Id: I087a9e4fc734a86be16a287456441afac5676beb
Reviewed-on: https://go-review.googlesource.com/30362
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Currently if you declare a type overwriting a predeclared type
and export methods on it they will be exposed in godoc, even
though the type itself is not exported. This corrects that
by making all methods on these types hidden, since that's
the expected output.

Fixes #9860

Change-Id: I14037bdcef1b4bbefcf299a143bac8bf363718e0
Reviewed-on: https://go-review.googlesource.com/20610Reviewed-by: Russ Cox <rsc@golang.org>

It is pretty confusing when there are Go files ignored for mismatching
build tags and similar and we output "no buildable Go files" without
giving any other information about some Go files have been ignored.

Fixes #17008.

Change-Id: I1766ee86a9a7a72f6694deae3f73b47bfc9d0be5
Reviewed-on: https://go-review.googlesource.com/29113
Run-TryBot: Jaana Burcu Dogan <jbd@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Suppose you have already downloaded "foo.bar/baz", where the repo
is for all of foo.bar/, and you then "go get -u foo.bar/...".
The command-line wildcard expands to foo.bar/baz,
and go get updates the foo.bar/ repo.
Suppose that the repo update brought in foo.bar/quux,
though, which depends on other.site/bar.
Download does not consider foo.bar/quux, since it's
only looking at foo.bar/baz, so it didn't download other.site/bar.
After the download, we call importPaths(args) to decide what to install.
That call was reevaluating the original wildcard with the new repo
and matching foo.bar/quux, which was missing its dependency
other.site/bar, causing a build failure.

The fix in this CL is to remember the pre-download expansion
of the argument list and pass it to the installer. Then only the things
we tried to download get installed.

The case where foo.bar/ is not even checked out yet continues to work,
because in that case we leave the wildcard in place, and download
reevaluates it during the download.

The fix in this CL may not be the right long-term fix, but it is at least a fix.
It may be that download should be passed all the original wildcards
so that it can reexpand them as new code is downloaded, ideally reaching
a fixed point. That can be left for another day.

In short:

- The problem is that the "install" half of "go get" was trying to install
  more than the "download" half was properly downloading.
- The fix in this CL is to install just what was downloaded (install less).
- It may be that a future CL should instead download what will be installed (download more).

Fixes #14450.

Change-Id: Ia1984761d24439549b7cff322bc0dbc262c1a653
Reviewed-on: https://go-review.googlesource.com/19892
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

This updates runtime/memclr_ppc64x.s to improve performance,
by unrolling loops for larger clears.

Fixes #17348

benchmark                    old MB/s     new MB/s     speedup
BenchmarkMemclr/5-80         199.71       406.63       2.04x
BenchmarkMemclr/16-80        693.66       1817.41      2.62x
BenchmarkMemclr/64-80        2309.35      5793.34      2.51x
BenchmarkMemclr/256-80       5428.18      14765.81     2.72x
BenchmarkMemclr/4096-80      8611.65      27191.94     3.16x
BenchmarkMemclr/65536-80     8736.69      28604.23     3.27x
BenchmarkMemclr/1M-80        9304.94      27600.09     2.97x
BenchmarkMemclr/4M-80        8705.66      27589.64     3.17x
BenchmarkMemclr/8M-80        8575.74      23631.04     2.76x
BenchmarkMemclr/16M-80       8443.10      19240.68     2.28x
BenchmarkMemclr/64M-80       8390.40      9493.04      1.13x
BenchmarkGoMemclr/5-80       263.05       630.37       2.40x
BenchmarkGoMemclr/16-80      904.33       1148.49      1.27x
BenchmarkGoMemclr/64-80      2830.20      8756.70      3.09x
BenchmarkGoMemclr/256-80     6064.59      20299.46     3.35x

Change-Id: Ic76c9183c8b4129ba3df512ca8b0fe6bd424e088
Reviewed-on: https://go-review.googlesource.com/30373
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Munday <munday@ca.ibm.com>
Reviewed-by: David Chase <drchase@google.com>

RHEL 7 introduces a new tool, update-ca-trust(8), which places the
certificate bundle in a new location. Add this path to the list of
locations that are searched for the certificate bundle.

Fixes #15749

Change-Id: Idc97f885ee48ef085f1eb4dacbd1c2cf55f94ff5
Reviewed-on: https://go-review.googlesource.com/30375Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #13485.

Change-Id: I11dd15c7d14fc19d42a3b48427a4cc1208b18e6a
Reviewed-on: https://go-review.googlesource.com/30456Reviewed-by: Matthew Dempsky <mdempsky@google.com>

This reverts commit 59320c39.

Reasons:
This CL was causing failures on a large regression test that we run
within Google. The issues arises from two bugs in the CL:
* The CL dropped support for ';' as a delimiter (see https://golang.org/issue/2210)
* The handling of an empty string caused an empty record to be added when
no record was added (see https://golang.org/cl/30454 for my attempted fix)

The logic being added is essentially a variation of url.ParseQuery,
but altered to accept an io.Reader instead of a string.
Since it is duplicated (but modified) logic, there needs to be good
tests to ensure that it's implementation doesn't drift in functionality
from url.ParseQuery. Fixing the above issues and adding the associated
regression tests leads to >100 lines of codes.
For a 4% reduction in CPU time, I think this complexity and duplicated
logic is not worth the effort.

As such, I am abandoning my efforts to fix the existing issues and
believe that reverting CL/20301 is the better course of action.

Updates #14655

Change-Id: Ibb5be0a5b48a16c46337e213b79467fcafee69df
Reviewed-on: https://go-review.googlesource.com/30470
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Updates #16360

Change-Id: I0e0afe7a89f2ebcb3e5bbc345f77a605d3afc398
Reviewed-on: https://go-review.googlesource.com/30103Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The UnmarshalTypeError has two new fields Struct and Field,
used when constructing the error message.

Fixes #6716.

Change-Id: I67da171480a9491960b3ae81893770644180f848
Reviewed-on: https://go-review.googlesource.com/18692
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #17224.

Some systems have more than just "lo" in a fresh network namespace, due
to IPv6. Instead of testing for exactly 3 lines of output (implying 1
interface), just test to make sure that the unshare call resulted in
fewer interfaces than before. This should still verify that unshare did
something.

Change-Id: Iaf84c2b0e673fc207059d62e2f4dd7583a753419
Reviewed-on: https://go-review.googlesource.com/30372Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Jessica Frazelle <me@jessfraz.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change float32/float64 formatting to use non-exponential form
for a slightly wider range, to more closely match ES6 JSON.stringify
and other JSON generators.

Most notably:

	1e20 now formats as 100000000000000000000 (previously 1e+20)
	1e-6 now formats as 0.000001 (previously 1e-06)
	1e-7 now formats as 1e-7 (previously 1e-07)

This also brings the int64 and float64 formatting in line with each other,
for all shared representable values. For example both int64(1234567)
and float64(1234567) now format as "1234567", where before the
float64 formatted as "1.234567e+06".

The only variation now compared to ES6 JSON.stringify is that
Go continues to encode negative zero as "-0", not "0", so that
the value continues to be preserved during JSON round trips.

Fixes #6384.
Fixes #14135.

Change-Id: Ib0e0e009cd9181d75edc0424a28fe776bcc5bbf8
Reviewed-on: https://go-review.googlesource.com/30371Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

These are conditional branches that takes a register instead of
flags as control value.

Reduce binary size by 0.7%, text size by 2.4% (cmd/go as an
exmaple).

Change-Id: I0020cfde745f9eab680b8b949ad28c87fe183afd
Reviewed-on: https://go-review.googlesource.com/30030Reviewed-by: David Chase <drchase@google.com>

In the function prologue, we emit a jump to the beginning of
the function immediately after calling morestack. And in the
runtime stack growing code, it decodes and emulates that jump.
This emulation was necessary before we had per-PC SP deltas,
since the traceback code assumed that the frame size was fixed
for the whole function, except on the first instruction where
it was 0. Since we now have per-PC SP deltas and PCDATA, we
can correctly record that the frame size is 0. This makes the
emulation unnecessary.

This may be helpful for registerized calling convention, where
there may be unspills of arguments after calling morestack. It
also simplifies the runtime.

Change-Id: I7ebee31eaee81795445b33f521ab6a79624c4ceb
Reviewed-on: https://go-review.googlesource.com/30138Reviewed-by: David Chase <drchase@google.com>

Windows doesn't remove an open file.

Fixes #17345

Change-Id: I283930c7d6eb3bc09ad208191afefe989804ce32
Reviewed-on: https://go-review.googlesource.com/30430
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This change updates PPC64.rules to recognize constant shift
counts and generate more efficient code sequences in those cases.

Fixes #17336

Change-Id: I8a7b812408d7a68388df41e42bad045dd214be17
Reviewed-on: https://go-review.googlesource.com/30310
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

complex64 and complex128 are treated like [2]float32 and [2]float64,
so it makes sense to align them the same way.

Change-Id: Ic614bcdcc91b080aeb1ad1fed6fc15ba5a2971f8
Reviewed-on: https://go-review.googlesource.com/19800
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

To refine a set of possibly equivalent values, the old CSE algorithm
picked one value, compared it against all the others, and made two sets
out of the results (the values that match the picked value and the
values that didn't).  Unfortunately, this leads to O(n^2) behavior. The
picked value ends up being equal to no other values, we make size 1 and
size n-1 sets, and then recurse on the size n-1 set.

Instead, sort the set by the equivalence classes of its arguments.  Then
we just look for spots in the sorted list where the equivalence classes
of the arguments change.  This lets us do a multi-way split for O(n lg
n) time.

This change makes cmpDepth unnecessary.

The refinement portion used to call the type comparator.  That is
unnecessary as the type was already part of the initial partition.

Lowers time of 16361 from 8 sec to 3 sec.
Lowers time of 15112 from 282 sec to 20 sec. That's kind of unfair, as
CL 30257 changed it from 21 sec to 282 sec. But that CL fixed other bad
compile times (issue #17127) by large factors, so net still a big win.

Fixes #15112
Fixes #16361

Change-Id: I351ce111bae446608968c6d48710eeb6a3d8e527
Reviewed-on: https://go-review.googlesource.com/30354Reviewed-by: Todd Neal <todd@tneal.org>

This commit changes parseRecord to allocate a single string per record,
instead of per field, by using indexes into the raw record.

Benchstat (done with f69991c1)

name                          old time/op    new time/op    delta
Read-8                          3.17µs ± 0%    2.78µs ± 1%  -12.35%  (p=0.016 n=4+5)
ReadWithFieldsPerRecord-8       3.18µs ± 1%    2.79µs ± 1%  -12.23%  (p=0.008 n=5+5)
ReadWithoutFieldsPerRecord-8    4.59µs ± 0%    2.77µs ± 0%  -39.58%  (p=0.016 n=4+5)
ReadLargeFields-8               57.0µs ± 0%    55.7µs ± 0%   -2.18%  (p=0.008 n=5+5)

name                          old alloc/op   new alloc/op   delta
Read-8                            660B ± 0%      664B ± 0%   +0.61%  (p=0.008 n=5+5)
ReadWithFieldsPerRecord-8         660B ± 0%      664B ± 0%   +0.61%  (p=0.008 n=5+5)
ReadWithoutFieldsPerRecord-8    1.14kB ± 0%    0.66kB ± 0%  -41.75%  (p=0.008 n=5+5)
ReadLargeFields-8               3.86kB ± 0%    3.94kB ± 0%   +1.86%  (p=0.008 n=5+5)

name                          old allocs/op  new allocs/op  delta
Read-8                            30.0 ± 0%      18.0 ± 0%  -40.00%  (p=0.008 n=5+5)
ReadWithFieldsPerRecord-8         30.0 ± 0%      18.0 ± 0%  -40.00%  (p=0.008 n=5+5)
ReadWithoutFieldsPerRecord-8      50.0 ± 0%      18.0 ± 0%  -64.00%  (p=0.008 n=5+5)
ReadLargeFields-8                 66.0 ± 0%      24.0 ± 0%  -63.64%  (p=0.008 n=5+5)

For a simple application that I wrote, which reads in a CSV file (via
ReadAll) and outputs the number of rows read (15857625 rows), this change
reduces the total time on my notebook from ~58 seconds to ~48 seconds.

This reduces time and allocations (bytes) each by ~6% for a real world
CSV file at work (~230000 rows, 13 colums).

Updates #16791

Change-Id: Ia07177c94624e55cdd3064a7d2751fb69322d3e4
Reviewed-on: https://go-review.googlesource.com/24723Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

plive.go (except for printeffects and livenessprintblock) and
reflect.go changes were prepared mechanically with gofmt -r.

Passes toolstash.

name       old alloc/op    new alloc/op    delta
Template      44.3MB ± 0%     44.3MB ± 0%    ~           (p=0.367 n=30+30)
Unicode       37.4MB ± 0%     37.4MB ± 0%    ~           (p=0.665 n=30+30)
GoTypes        125MB ± 0%      125MB ± 0%    ~           (p=0.067 n=30+30)
Compiler       515MB ± 0%      515MB ± 0%    ~           (p=0.542 n=30+28)

name       old allocs/op   new allocs/op   delta
Template        434k ± 0%       434k ± 0%    ~           (p=0.076 n=30+29)
Unicode         367k ± 0%       367k ± 0%    ~           (p=0.716 n=29+30)
GoTypes        1.24M ± 0%      1.24M ± 0%    ~           (p=0.428 n=29+29)
Compiler       4.47M ± 0%      4.47M ± 0%    ~           (p=0.225 n=28+30)

Change-Id: Ibaf0668567b3f69fba06aa03b7997c8fb152113a
Reviewed-on: https://go-review.googlesource.com/30356
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Always close the file regardless of whether the copy succeeds or fails.
Pass along the close error if the copy succeeds

Fixes #16296

Change-Id: Ib394655b91d25750f029f17b3846d985f673fb50
Reviewed-on: https://go-review.googlesource.com/30410Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

It already implemented the Timeout method,
but implementing the full net.Error is more convenient.

Fixes #14238 (again).

Change-Id: Ia87f897f0f35bcb49865e2355964049227951ca6
Reviewed-on: https://go-review.googlesource.com/30370
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Also adds two tests: one to exercise the counter incrementing code
and one which checks the output of the optimized implementation
against that of the generic implementation for large/unaligned data
sizes.

Uses the KIMD instruction for GHASH and the KMCTR instruction for AES
in counter mode.

AESGCMSeal1K  75.0MB/s ± 2%  1008.7MB/s ± 1%  +1245.71%  (p=0.000 n=10+10)
AESGCMOpen1K  75.3MB/s ± 1%  1006.0MB/s ± 1%  +1235.59%   (p=0.000 n=10+9)
AESGCMSeal8K  78.5MB/s ± 1%  1748.4MB/s ± 1%  +2127.34%   (p=0.000 n=9+10)
AESGCMOpen8K  78.5MB/s ± 0%  1752.7MB/s ± 0%  +2134.07%   (p=0.000 n=10+9)

Change-Id: I88dbcfcb5988104bfd290ae15a60a2721c1338be
Reviewed-on: https://go-review.googlesource.com/30361Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Should fix the clang builder.

Change-Id: I3ee34581b6a7ec902420de72a8a08a2426997782
Reviewed-on: https://go-review.googlesource.com/30363
Run-TryBot: Michael Munday <munday@ca.ibm.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I9783d8023d453a72c4605a308064bef98168bcb8
Reviewed-on: https://go-review.googlesource.com/30360Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #17006

Change-Id: I3c2060ca5384a4b9782a7d804305d2cf4388dd5a
Reviewed-on: https://go-review.googlesource.com/29014
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

Calling cgocallback from a signal handler can fail when using the race
detector. Calling cgocallback will lead to a call to newextram which
will call oneNewExtraM which will call racegostart. The racegostart
function will set up some race detector data structures, and doing that
will sometimes call the C memory allocator. If we are running the signal
handler from a signal that interrupted the C memory allocator, we will
crash or hang.

Instead, change the signal handler code to call needm and dropm. The
needm function will grab allocated m and g structures and initialize the
g to use the current stack--the signal stack. That is all we need to
safely call code that allocates memory and checks whether it needs to
split the stack. This may temporarily leave us with no m available to
run a cgo callback, but that is OK in this case since the code we call
will quickly either crash or call dropm to return the m.

Implementing this required changing some of the setSignalstackSP
functions to avoid a write barrier. These functions never need a write
barrier but in some cases generated one anyhow because on some systems
the ss_sp field is a pointer.

Change-Id: I3893f47c3a66278f85eab7f94c1ab11d4f3be133
Reviewed-on: https://go-review.googlesource.com/30218
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

Change-Id: I56de359a5ccdc0a10925cd372fa86534353c6ca0
Reviewed-on: https://go-review.googlesource.com/30358
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Benchmarks broken off from https://golang.org/cl/24723 and modified to
allocate less in the places we're not trying to measure.

Updates #16791

Change-Id: I508e4cfeac60322d56f1d71ff1912f6a6f183a63
Reviewed-on: https://go-review.googlesource.com/30357
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

All the prefixes of the testGIF produce errors today,
but they differ wildly in which errors: some are io.EOF,
others are io.ErrUnexpectedEOF, and others are gif-specific.
Make them all gif-specific to explain context, and make
any complaining about EOF be sure to mention the EOF
is unexpected.

Fixes #11390.

Change-Id: I742c39c88591649276268327ea314e68d1de1845
Reviewed-on: https://go-review.googlesource.com/17493
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #11441.

Test tables generated by

	package main

	import (
		"bytes"
		"fmt"
		"log"
		"os/exec"
		"strconv"
		"strings"
	)

	var inputs = []float64{
		0.5,
		1.5,
		2.5,
		3.5,
		-0.5,
		-1.5,
		-2.5,
		-3.5,
		0.1,
		0.01,
		1e-8,
		1e-16,
		1e-3,
		1e-16,
		1e-308,
		5.6e-309,
		5.5e-309,
		1e-309,
		1e-323,
		5e-324,
		-0.1,
		-0.01,
		-1e-8,
		-1e-16,
		-1e-3,
		-1e-16,
		-1e-308,
		-5.6e-309,
		-5.5e-309,
		-1e-300 / 1e9,
		-1e-300 / 1e23,
		-5e-300 / 1e24,
		-0.9999999999999999,
		-1.0000000000000002,
		-1.9999999999999998,
		-2.0000000000000004,
		-100.00000000000001,
		-99.999999999999986,
		17,
		171,
		171.6,
		171.624,
		171.625,
		172,
		2000,
		-100.5,
		-160.5,
		-170.5,
		-171.5,
		-176.5,
		-177.5,
		-178.5,
		-179.5,
		-201.0001,
		-202.9999,
		-1000.5,
		-1000000000.3,
		-4503599627370495.5,
		-63.349078729022985,
		-127.45117632943295,
	}

	func main() {
		var buf bytes.Buffer
		for _, v := range inputs {
			fmt.Fprintf(&buf, "gamma(%.1000g)\n", v)
		}
		cmd := exec.Command("gp", "-q")
		cmd.Stdin = &buf
		out, err := cmd.CombinedOutput()
		if err != nil {
			log.Fatalf("gp: %v", err)
		}
		f := strings.Split(string(out), "\n")
		if len(f) > 0 && f[len(f)-1] == "" {
			f = f[:len(f)-1]
		}
		if len(f) != len(inputs) {
			log.Fatalf("gp: wrong output count\n%s\n", out)
		}
		for i, g := range f {
			gf, err := strconv.ParseFloat(strings.Replace(g, " E", "e", -1), 64)
			if err != nil {
				if strings.Contains(err.Error(), "value out of range") {
					if strings.HasPrefix(g, "-") {
						fmt.Printf("\t{%g, Inf(-1)},\n", inputs[i])
					} else {
						fmt.Printf("\t{%g, Inf(1)},\n", inputs[i])
					}
					continue
				}
				log.Fatal(err)
			}
			if gf == 0 && strings.HasPrefix(g, "-") {
				fmt.Printf("\t{%g, Copysign(0, -1)},\n", inputs[i])
				continue
			}
			fmt.Printf("\t{%g, %g},\n", inputs[i], gf)
		}
	}

Change-Id: Ie98c7751d92b8ffb40e8313f5ea10df0890e2feb
Reviewed-on: https://go-review.googlesource.com/30146
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Quentin Smith <quentin@golang.org>

The 387 implementation is less accurate and slower.

name     old time/op  new time/op  delta
Exp-8    29.7ns ± 2%  24.0ns ± 2%  -19.08%  (p=0.000 n=10+10)

This makes Gamma more accurate too.

Change-Id: Iad33b9cce0b087ccbce3e08ba7a6d285c4999d02
Reviewed-on: https://go-review.googlesource.com/30230
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Quentin Smith <quentin@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The documentation for doc says:
> Doc prints the documentation comments associated with the item identified by its
> arguments (a package, const, func, type, var, or method) followed by a one-line
> summary of each of the first-level items "under" that item (package-level
> declarations for a package, methods for a type, etc.).

Certain variables (and constants, functions, and types) have value specifications
that are multiple lines long. Prior to this change, doc would print out all of the
lines necessary to display the value. This is inconsistent with the documented
behavior, which guarantees a one-line summary for all first-level items.
We fix this here by writing a general oneLineNode method that always returns
a one-line summary (guaranteed!) of any input node.

Packages like image/color/palette and unicode now become much
more readable since large slices are now a single line.

$ go doc image/color/palette
<<<
// Before:
var Plan9 = []color.Color{
	color.RGBA{0x00, 0x00, 0x00, 0xff},
	color.RGBA{0x00, 0x00, 0x44, 0xff},
	color.RGBA{0x00, 0x00, 0x88, 0xff},
	... // Hundreds of more lines!
}
var WebSafe = []color.Color{
	color.RGBA{0x00, 0x00, 0x00, 0xff},
	color.RGBA{0x00, 0x00, 0x33, 0xff},
	color.RGBA{0x00, 0x00, 0x66, 0xff},
	... // Hundreds of more lines!
}

// After:
var Plan9 = []color.Color{ ... }
var WebSafe = []color.Color{ ... }
>>>

In order to test this, I ran `go doc` and `go doc -u` on all of the
standard library packages and diff'd the output with and without the
change to ensure that all differences were intended.

Fixes #13072

Change-Id: Ida10b7796b7e4e174a929b55c60813a9eb7158fe
Reviewed-on: https://go-review.googlesource.com/25420
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

Change-Id: I3f13488605ab62eba5d3c59d5e9df1bcf69dd571
Reviewed-on: https://go-review.googlesource.com/30355Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

For #16339

Change-Id: I8927f40e0fd166795f41c784ad92449743f73af5
Reviewed-on: https://go-review.googlesource.com/30213Reviewed-by: Matthew Dempsky <mdempsky@google.com>