This adds a best-effort pass to remove stack barriers immediately
after the end of mark termination. This isn't necessary for the Go
runtime, but should help external tools that perform stack walks but
aren't aware of Go's stack barriers such as GDB, perf, and VTune.
(Though clearly they'll still have trouble unwinding stacks during
mark.)

Change-Id: I66600fae1f03ee36b5459d2b00dcc376269af18e
Reviewed-on: https://go-review.googlesource.com/20668Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently we remove stack barriers during STW mark termination, which
has a non-trivial per-goroutine cost and means that we have to touch
even clean stacks during mark termination. However, there's no problem
with leaving them in during the sweep phase. They just have to be out
by the time we install new stack barriers immediately prior to
scanning the stack such as during the mark phase of the next GC cycle
or during mark termination in a STW GC.

Hence, move the gcRemoveStackBarriers from STW mark termination to
just before we install new stack barriers during concurrent mark. This
removes the cost from STW. Furthermore, this combined with concurrent
stack shrinking means that the mark termination scan of a clean stack
is a complete no-op, which will make it possible to skip clean stacks
entirely during mark termination.

This has the downside that it will mess up anything outside of Go that
tries to walk Go stacks all the time instead of just some of the time.
This includes tools like GDB, perf, and VTune. We'll improve the
situation shortly.

Change-Id: Ia40baad8f8c16aeefac05425e00b0cf478137097
Reviewed-on: https://go-review.googlesource.com/20667Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently we enqueue span root mark jobs during both concurrent mark
and mark termination, but we make the job a no-op during mark
termination.

This is silly. Instead of queueing them up just to not do them, don't
queue them up in the first place.

Change-Id: Ie1d36de884abfb17dd0db6f0449a2b7c997affab
Reviewed-on: https://go-review.googlesource.com/20666Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently we free cached stacks of dead Gs during STW stack root
marking. We do this during STW because there's no way to take
ownership of a particular dead G, so attempting to free a dead G's
stack during concurrent stack root marking could race with reusing
that G.

However, we can do this concurrently if we take a completely different
approach. One way to prevent reuse of a dead G is to remove it from
the free G list. Hence, this adds a new fixed root marking task that
simply removes all Gs from the list of dead Gs with cached stacks,
frees their stacks, and then adds them to the list of dead Gs without
cached stacks.

This is also a necessary step toward rescanning only dirty stacks,
since it eliminates another task from STW stack marking.

Change-Id: Iefbad03078b284a2e7bf30fba397da4ca87fe095
Reviewed-on: https://go-review.googlesource.com/20665Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently all free Gs are added to one list. Split this into two
lists: one for free Gs with cached stacks and one for Gs without
cached stacks.

This lets us preferentially allocate Gs that already have a stack, but
more importantly, it sets us up to free cached G stacks concurrently.

Change-Id: Idbe486f708997e1c9d166662995283f02d1eeb3c
Reviewed-on: https://go-review.googlesource.com/20664Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Let's define the line number of a multiline rule as the line
number on which the -> appears.  This helps make the rule
cover analysis look a bit nicer.

Change-Id: I4ac4c09f2240285976590ecfd416bc4c05e78946
Reviewed-on: https://go-review.googlesource.com/22473Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

Instead of eagerly creating strings like "literal 2.01" for every
lexed number in case we need to mention it in an error message, defer
this work to (*parser).syntax_error.

name      old allocs/op  new allocs/op  delta
Template      482k ± 0%      482k ± 0%  -0.12%   (p=0.000 n=9+10)
GoTypes      1.35M ± 0%     1.35M ± 0%  -0.04%  (p=0.015 n=10+10)
Compiler     5.45M ± 0%     5.44M ± 0%  -0.12%    (p=0.000 n=9+8)

Change-Id: I333b3c80e583864914412fb38f8c0b7f1d8c8821
Reviewed-on: https://go-review.googlesource.com/22480
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This simplifies comparison of object files across different builds
by ensuring that the strings in the zcgo.go always appear in the
same order.

Change-Id: I3639ea4fd10e0d645b838d1bbb03cd33deca340e
Reviewed-on: https://go-review.googlesource.com/22478Reviewed-by: Matthew Dempsky <mdempsky@google.com>

This change significantly speeds up case-insensitive regexp matching.

benchmark                      old ns/op      new ns/op      delta
BenchmarkMatchEasy0i_32-8      2690           1473           -45.24%
BenchmarkMatchEasy0i_1K-8      80404          42269          -47.43%
BenchmarkMatchEasy0i_32K-8     3272187        2076118        -36.55%
BenchmarkMatchEasy0i_1M-8      104805990      66503805       -36.55%
BenchmarkMatchEasy0i_32M-8     3360192200     2126121600     -36.73%

benchmark                      old MB/s     new MB/s     speedup
BenchmarkMatchEasy0i_32-8      11.90        21.72        1.83x
BenchmarkMatchEasy0i_1K-8      12.74        24.23        1.90x
BenchmarkMatchEasy0i_32K-8     10.01        15.78        1.58x
BenchmarkMatchEasy0i_1M-8      10.00        15.77        1.58x
BenchmarkMatchEasy0i_32M-8     9.99         15.78        1.58x

Issue #13288

Change-Id: I94af7bb29e75d60b4f6ee760124867ab271b9642
Reviewed-on: https://go-review.googlesource.com/16943Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

This version of the file name honors the -trimprefix flag,
which strips off variable parts like $WORK or $PWD.
The TestCgoConsistentResults test now passes.

Change-Id: If93980b054f9b13582dd314f9d082c26eaac4f41
Reviewed-on: https://go-review.googlesource.com/22444Reviewed-by: Robert Griesemer <gri@golang.org>

Change-Id: I0f016fa000f949d27847d645b4cdebe68a8abf20
Reviewed-on: https://go-review.googlesource.com/22474
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #15443

Change-Id: Ia3593104fc1a4255926ae5675c25390604b44b7b
Reviewed-on: https://go-review.googlesource.com/22453
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Also adds TestGdbBacktrace to the runtime package.

Dwarf modifications written by Bryan Chan (@bryanpkc) who is also
at IBM and covered by the same CLA.

Fixes #14628

Change-Id: I106a1f704c3745a31f29cdadb0032e3905829850
Reviewed-on: https://go-review.googlesource.com/20193Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The comment says 'DΟ NΟT SUBMIT', and that text being in a file can cause
automated errors or warnings when trying to check the Go sources into other
source control systems.

(We reject that string in CL commit messages, which I've avoided here
by changing the O's to Ο's above.)

Change-Id: I6cdd57a8612ded5208f05a8bd6b137f44424a030
Reviewed-on: https://go-review.googlesource.com/22434
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Make sure ops have the right number of args, set
aux and auxint only if allowed, etc.

Normalize error reporting format.

Change-Id: Ie545fcc5990c8c7d62d40d9a0a55885f941eb645
Reviewed-on: https://go-review.googlesource.com/22320Reviewed-by: David Chase <drchase@google.com>

Renames block to blockGeneric so that it can be called when the
assembly feature check fails. This means making block a var on
platforms without an assembly implementation (similar to the sha1
package).

Also adds a test to check that the fallback path works correctly
when the feature check fails.

name        old speed      new speed       delta
Hash8Bytes  7.13MB/s ± 2%  19.89MB/s ± 1%  +178.82%   (p=0.000 n=9+10)
Hash1K       121MB/s ± 1%    661MB/s ± 1%  +444.54%   (p=0.000 n=10+9)
Hash8K       137MB/s ± 0%    918MB/s ± 1%  +569.29%  (p=0.000 n=10+10)

Change-Id: Id65dd6e943f14eeffe39a904dc88065fc6a60179
Reviewed-on: https://go-review.googlesource.com/22402Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #15434.

Change-Id: Ia88b740df5418a6d3af1c29a03756f4234f388b0
Reviewed-on: https://go-review.googlesource.com/22428Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The linker was incorrectly decoding type name lengths, causing
typelinks to be sorted out of order and in cases where the name was
the exact right length, linker panics.

Added a test to the reflect package that causes TestTypelinksSorted
to fail before this CL. It's not the exact failure seen in #15448
but it has the same cause: decodetype_name calculating the wrong
length.

The equivalent decoders in reflect/type.go and runtime/type.go
have the parenthesis in the right place.

Fixes #15448

Change-Id: I33257633d812b7d2091393cb9d6cc8a73e0138c8
Reviewed-on: https://go-review.googlesource.com/22403Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Here, "fix" means "replace".  The new dominator computation
is the "simple" algorithm from Lengauer and Tarjan's TOPLAS
paper, with minimal changes.

Also included is a test that tweaks the fixed error.

Change-Id: I0abdf53d5d64df1e67e4e62f55e88957045cd63b
Reviewed-on: https://go-review.googlesource.com/22401
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Use PCMPESTRI instruction if available.

Index-4              21.1ns ± 0%  21.1ns ± 0%     ~     (all samples are equal)
IndexHard1-4          395µs ± 0%   105µs ± 0%  -73.53%        (p=0.000 n=19+20)
IndexHard2-4          300µs ± 0%   147µs ± 0%  -51.11%        (p=0.000 n=19+20)
IndexHard3-4          665µs ± 0%   665µs ± 0%     ~           (p=0.942 n=16+19)

Change-Id: I4f66794164740a2b939eb1c78934e2390b489064
Reviewed-on: https://go-review.googlesource.com/22337
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This is not necessary for reproduceability but it removes
differences due to imported package order between compiles
using textual vs binary export format. The packages list
tends to be very short, so it's ok doing it always for now.

Guarded with a documented (const) flag so it's trivial to
disable and remove eventually.

Also, use the same flag now to enforce parameter numbering.

Change-Id: Ie05d2490df770239696ecbecc07532ed62ccd5c0
Reviewed-on: https://go-review.googlesource.com/22445
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

The code sequence for large-offset floating-point stores
includes adding the base pointer to r11.  Make sure we
can interpret that instruction correctly.

Fixes build.

Fixes #15440

Change-Id: I7fe5a4a57e08682967052bf77c54e0ec47fcb53e
Reviewed-on: https://go-review.googlesource.com/22440Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>

The numbering is only required for parameters of functions/methods
with exported inlineable bodies. For now, always export parameter names
with internal numbering to minimize the diffs between assembly code
dumps of code compiled with the textual vs the binary format.

To be disabled again once the new export format is default.

Change-Id: I6d14c564e734cc5596c7e995d8851e06d5a35013
Reviewed-on: https://go-review.googlesource.com/22441Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Fixes #15439.

Change-Id: I5a32384c46e20f8db6968e5a9e854c45ab262fe4
Reviewed-on: https://go-review.googlesource.com/22429Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Zero the entire buffer so we don't need to
lower its capacity upon return.  This lets callers
do some appending without allocation.

Zeroing is cheap, the byte buffer requires only
4 extra instructions.

Fixes #14235

Change-Id: I970d7badcef047dafac75ac17130030181f18fe2
Reviewed-on: https://go-review.googlesource.com/22424
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Introduce OrB, EqB, NeqB, AndB to handle bool operations.

Change-Id: I53e4d5125a8090d5eeb4576db619103f19fff58d
Reviewed-on: https://go-review.googlesource.com/22412Reviewed-by: Keith Randall <khr@golang.org>

Note that the spec already makes that point with a comment in the very first
example for struct field tags. This change is simply stating this explicitly
in the actual spec prose.

- gccgo and go/types already follow this rule
- the current reflect package API doesn't distinguish between absent tags
  and empty tags (i.e., there is no discoverable difference)

Fixes #15412.

Change-Id: I92f9c283064137b4c8651630cee0343720717a02
Reviewed-on: https://go-review.googlesource.com/22391Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

The underlying issues have been fixed.
All the individual fixes have their own tests,
but it's still useful to have a plain source test.

Fixes #15084

Change-Id: I06c485a7d0716201bd57d1f3be53668dddd7ec14
Reviewed-on: https://go-review.googlesource.com/22426
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

As a nice side-effect, this allows us to
unify several code paths.

The terminology (low, high, max, simple slice expr,
full slice expr) is taken from the spec and
the examples in the spec.

This is a trial run. The plan, probably for Go 1.8,
is to change slice expressions to use Node.List
instead of OKEY, and to do some similar
tree structure changes for other ops.

Passes toolstash -cmp. No performance change.
all.bash passes with GO_GCFLAGS=-newexport.

Updates #15350

Change-Id: Ic1efdc36e79cdb95ae1636e9817a3ac8f83ab1ac
Reviewed-on: https://go-review.googlesource.com/22425Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Update #15410

Change-Id: Iad3f2639aa7a67b11efc35a629e1893f7d87b957
Reviewed-on: https://go-review.googlesource.com/22430
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Updates #15345

Change-Id: Iae35d3e378cbc8157ba1ff91e4971ed4515a5e5c
Reviewed-on: https://go-review.googlesource.com/22394Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

* Make budget an int32 to avoid needless conversions.
* Introduce some temporary variables to reduce repetition.
* If ... args are present, they will be the last argument
  to the function. No need to scan all arguments.

Passes toolstash -cmp.

Change-Id: I55203609f5d2f25a4e238cd48c63214651120cfc
Reviewed-on: https://go-review.googlesource.com/22421
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Generated with eg:

func before(n gc.Nodes) int { return len(n.Slice()) }
func after(n gc.Nodes) int  { return n.Len() }

Change-Id: Ifdf01915e60069166afe96aa7b1d08720bf62fc5
Reviewed-on: https://go-review.googlesource.com/22420
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This is a follow-up to CLs 19769 and 19770.

Change-Id: Ia9b71055613b80df4ce62b34fcc4f479f04f72fd
Reviewed-on: https://go-review.googlesource.com/22399
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

Passes toolstash -cmp.

Change-Id: I42c962cc5a3ffec2969f223cf238c2fdadbf5857
Reviewed-on: https://go-review.googlesource.com/22381
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Passes toolstash -cmp.

Change-Id: I915e76374fd64aa2597e6fa47e4fa95ca00ca643
Reviewed-on: https://go-review.googlesource.com/22380
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

Note that this is only safe because
the compiler generates multiple distinct
gc.Types. If we switch to having canonical
gc.Types, then this will need to be updated
to handle the case in which the user uses both
map[T]S and also map[[8]T]S. In that case,
the runtime needs algs for [8]T, but this could
mark the sole [8]T type as Noalg. This is a general
problem with having a single bool to represent
whether alg generation is needed for a type.

Cuts 5k off cmd/go and 22k off golang.org/x/tools/cmd/godoc,
approx 0.04% and 0.12% respectively.

For #6853 and #9930

Change-Id: I30a15ec72ecb62e2aa053260a7f0f75015fc0ade
Reviewed-on: https://go-review.googlesource.com/19769Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Note that this is only safe because
the compiler generates multiple distinct
gc.Types. If we switch to having canonical
gc.Types, then this will need to be updated
to handle the case in which the user uses both
map[[n]T]S and also calls a function f(...T) with n arguments.
In that case, the runtime needs algs for [n]T, but this could
mark the sole [n]T type as Noalg. This is a general
problem with having a single bool to represent
whether alg generation is needed for a type.

Cuts 17k off cmd/go and 13k off golang.org/x/tools/cmd/godoc,
approx 0.14% and 0.07% respectively.

For #6853 and #9930

Change-Id: Iccb6b9fd88ade5497d7090528a903816d340bf0a
Reviewed-on: https://go-review.googlesource.com/19770Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

  func f(x, y, z *int) {
    a := []*int{x,y,z}
    ...
  }

We used to use:
  var tmp [3]*int
  a := tmp[:]
  a[0] = x
  a[1] = y
  a[2] = z

Now we do:
  var tmp [3]*int
  tmp[0] = x
  tmp[1] = y
  tmp[2] = z
  a := tmp[:]

Doesn't sound like a big deal, but the compiler has trouble
eliminating write barriers when using the former method because it
doesn't know that the slice points to the stack.  In the latter
method, the compiler knows the array is on the stack and as a result
doesn't emit any write barriers.

This turns out to be extremely common when building ... args, like
for calls fmt.Printf.

Makes go binaries ~1% smaller.

Doesn't have a measurable effect on the go1 fmt benchmarks,
unfortunately.

Fixes #14263
Update #6853

Change-Id: I9074a2788ec9e561a75f3b71c119b69f304d6ba2
Reviewed-on: https://go-review.googlesource.com/22395
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

Arm arch uses coarse-grained kernel timer as cputicks.
As the result sort.Sort smashes trace entirely. Use sort.Stable instead.

Change-Id: Idfa017a86a489be58cf239f7fe56d7f4b66b52a9
Reviewed-on: https://go-review.googlesource.com/22317
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>