This change modifies the behavior of span allocations to no longer
prefer the free treap over the scavenged treap.

While there is an additional cost to allocating out of the scavenged
treap, the current behavior of preferring the unscavenged spans can
lead to unbounded growth of a program's virtual memory footprint.

In small programs (low # of Ps, low resident set size, low allocation
rate) this behavior isn't really apparent and is difficult to
reproduce.

However, in relatively large, long-running programs we see this
unbounded growth in free spans, and an unbounded amount of heap
growths.

It still remains unclear how this policy change actually ends up
increasing the number of heap growths over time, but switching the
policy back to best-fit does indeed solve the problem.

Change-Id: Ibb88d24f9ef6766baaa7f12b411974cc03341e7b
Reviewed-on: https://go-review.googlesource.com/c/148979
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

This change adds the treapIter type which provides an iterator
abstraction for walking over an mTreap. In particular, the mTreap type
now has iter() and rev() for iterating both forwards (smallest to
largest) and backwards (largest to smallest). It also has an erase()
method for erasing elements at the iterator's current position.

For #28479.

While the expectation is that this change will slow down Go programs,
the impact on Go1 and Garbage is negligible.

Go1:     https://perf.golang.org/search?q=upload:20181214.6
Garbage: https://perf.golang.org/search?q=upload:20181214.11

Change-Id: I60dbebbbe73cbbe7b78d45d2093cec12cc0bc649
Reviewed-on: https://go-review.googlesource.com/c/151537
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>

This commit move data addresses to 0x200000000 for XCOFF executables.
.data and .bss must always be position-independent on AIX. This
modification allows to detect more easily if they aren't, as segfault
will be triggered.

Change-Id: Ied7a5b72b9f4ff9f870a1626cf07c48110635e62
Reviewed-on: https://go-review.googlesource.com/c/151040
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

When a println arg contains a call to an inlineable function
that itself contains a switch, that switch statement will be
walked twice, once by the walkexprlist formerly in the
OPRINT/OPRINTN case, then by walkexprlistcheap in walkprint.

Remove the first walkexprlist, it is not necessary.
walkexprlist =
		s[i] = walkexpr(s[i], init)
walkexprlistcheap = {
		s[i] = cheapexpr(n, init)
		s[i] = walkexpr(s[i], init)
}

Seems like this might be possible in other places, i.e.,
calls to inlineable switch-containing functions.

See also #25776.
Fixes #29220.

Change-Id: I3781e86aad6688711597b8bee9bc7ebd3af93601
Reviewed-on: https://go-review.googlesource.com/c/154497
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

sysUsed on Windows cares about the result from the VirtualAlloc syscall
returning exactly the address that was passed to it. However,
VirtualAlloc aligns the address its given to the kernel's allocation
granularity, so the returned address may not be the same.

Note that this wasn't an issue in the past because we only sysUsed
regions owned by spans, and spans are always a multiple of 8K, which
is a multiple of the allocation granularity on most Windows machines.

Change-Id: I3f5ccd63c6bbbd8b7995945ecedee17573b31667
Reviewed-on: https://go-review.googlesource.com/c/153677
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>

Change-Id: I2a5613377a38815fb8746c5bfb07ccbbc2e6dd0b
Reviewed-on: https://go-review.googlesource.com/c/153829Reviewed-by: Adam Langley <agl@golang.org>

Change-Id: If65518c76a865c03266be76b1c21c76e1c8b4763
Reviewed-on: https://go-review.googlesource.com/c/153828Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This captures the stack trace where mark completion observed that each
P had no work, and then dumps this if that P later discovers more
work. Hopefully this will help bound where the work was created.

For #27993.

Change-Id: I4f29202880d22c433482dc1463fb50ab693b6de6
Reviewed-on: https://go-review.googlesource.com/c/154599
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

Many of the crashes observed in #27993 involve committing the new
_defer object at the end of newdefer. It would be helpful to know if
the _defer was just allocated or was retrieved from the defer pool. In
order to indicate this in the traceback, this CL duplicates the tail
of newdefer so that the PC/line number will tell us whether d is new
or not.

For #27993.

Change-Id: Icd3e23dbcf00461877bb082b6f18df701149a607
Reviewed-on: https://go-review.googlesource.com/c/154598
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

Currently we only know the slot address and the value being written in
the throwOnGCWork crash tracebacks, and we have to infer the old value
from what's dumped by gcWork.checkPut. Sometimes these old values
don't make sense, like when we see a write of a nil pointer to a
freshly-allocated object, yet we observe marking a value (where did
that pointer come from?).

This CL adds the old value of the slot and the first two pointers in
the buffer to the traceback.

For #27993.

Change-Id: Ib70eead1afb9c06e8099e520172c3a2acaa45f80
Reviewed-on: https://go-review.googlesource.com/c/154597
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

This commit updates the new symbol addressing made for aix/ppc64 according
to feedbacks given in CL 151039.

Change-Id: Ic4eb9943dc520d65f7d084adf8fa9a2530f4d3f9
Reviewed-on: https://go-review.googlesource.com/c/151302
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: Ifb16fd28105efd05cebbd615b52e45330b77cede
Reviewed-on: https://go-review.googlesource.com/c/154600Reviewed-by: Ian Lance Taylor <iant@golang.org>

A prior optimization (https://golang.org/cl/106175) removed the
generation of unnecessary method expression wrappers, but also
eliminated the generation of the wrapper for error.Error which
was still required.

Special-case error type in the optimization.

Fixes #29304.

Change-Id: I54c8afc88a2c6d1906afa2d09c68a0a3f3e2f1e3
Reviewed-on: https://go-review.googlesource.com/c/154578Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Currently we reset the write barrier buffer before processing the
pointers in it. As a result, if there were any write barriers in the
code that processes the buffer, it would corrupt the write barrier
buffer and cause us to mark objects without later scanning them.

As far as I can tell, this shouldn't be happening, but rather than
relying on hope (and incomplete static analysis), this CL changes
wbBufFlush1 to poison the write barrier buffer while processing it,
and only reset it once it's done.

Updates #27993. (Unlike many of the other changes for this issue,
there's no need to roll back this CL. It's a good change in its own
right.)

Change-Id: I6d2d9f1b69b89438438b9ee624f3fff9f009e29d
Reviewed-on: https://go-review.googlesource.com/c/154537
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

This commit fixes error messages displayed on aix/ppc64 with external
linking.

Change-Id: I5311d36f30394be717827891e070db249482814a
Reviewed-on: https://go-review.googlesource.com/c/151041Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

It appears calling GetFileInformationByHandleEx with
FILE_ATTRIBUTE_TAG_INFO fails on FAT file system. FAT does not
support symlinks, so assume there are no symlnks when
GetFileInformationByHandleEx returns ERROR_INVALID_PARAMETER.

Fixes #29214

Change-Id: If2d9f3288bd99637681ab5fd4e4581c77b578a69
Reviewed-on: https://go-review.googlesource.com/c/154377
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

The OpenBSD armv7 port has working VFPv3 these days - re-enable the VFP
detection code so that GOARM=7 is used by default on openbsd/arm.

Change-Id: I0271d81c048d2d55becd2803c19e5f1542076357
Reviewed-on: https://go-review.googlesource.com/c/154378Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>

This fixes a few different issues that led to hangs and general
flakiness in the TestDebugCall* tests.

1. This fixes missing wake-ups in two error paths of the SIGTRAP
   signal handler. If the goroutine was in an unknown state, or if
   there was an unknown debug call status, we currently don't wake the
   injection coordinator. These are terminal states, so this resulted
   in a hang.

2. This adds a retry if the target goroutine is in a transient state
   that prevents us from injecting a call. The most common failure
   mode here is that the target goroutine is in _Grunnable, but this
   was previously masked because it deadlocked the test.

3. Related to 2, this switches the "ready" signal from the target
   goroutine from a blocking channel send to a non-blocking channel
   send. This makes it much less likely that we'll catch this
   goroutine while it's in the runtime performing that send.

4. This increases GOMAXPROCS from 2 to 8 during these tests. With the
   current setting of 2, we can have at most the non-preemptible
   goroutine we're injecting a call in to and the goroutine that's
   trying to make it exit. If anything else comes along, it can
   deadlock. One particular case I observed was in TestDebugCallGC,
   where runtime.GC() returns before the forEachP that prepares
   sweeping on all goroutines has finished. When this happens, the
   forEachP blocks on the non-preemptible loop, which means we now
   have at least three goroutines that need to run.

Fixes #25519.

Updates #29124.

Change-Id: I7bc41dc0b865b7d0bb379cb654f9a1218bc37428
Reviewed-on: https://go-review.googlesource.com/c/154112
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

Fixes #29271

Change-Id: Iff6a16c659ad6ec1b4d9559fcbcd40196086c60e
Reviewed-on: https://go-review.googlesource.com/c/154380
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Similar to to macOS' CF* types and JNI's jobject and derived types,
the EGLDisplay type is declared as a pointer but can contain
non-pointers (see #27054).
Fix it the same way: map EGLDisplay to uintptr in Go.

Fixes #27054

RELNOTE=yes

Change-Id: I6136f8f8162687c5493b30ed324e29efe55a8fd7
Reviewed-on: https://go-review.googlesource.com/c/154417
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Updates #26148

Change-Id: I704efafca39e4397caf2db0146d83d309c761dd1
Reviewed-on: https://go-review.googlesource.com/c/154357Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

CL 146297 ignored archive members with short names that don't have
the .o suffix, however, it also ignored .syso files as well.
This change restores the original .syso behavior and adds a test.

As the test is basically following a shell script, we make use of
the existing cmd/go/testdata/script framework. To support running
C compiler in the script, we added a `cc` command, which runs the
C compiler along with correct platform specific arguments.

Fixes #29253.

Change-Id: If8520151c4d6a74ab9fe84d34bff9a4480688815
Reviewed-on: https://go-review.googlesource.com/c/154109
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>

cmd/go/main.go sets GOCACHE explicitly, so if we don't save some
metadata about how DefaultDir arrived at its answer we will be unable
to reconstruct it later.

Fixes #29243

Change-Id: Ic8bb859ab045a29c91f6a4527e65aedabf874d53
Reviewed-on: https://go-review.googlesource.com/c/154309
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>

Change-Id: Ic098bd69fa9e3f7b2ed6c451a7a266167c0cde94
Reviewed-on: https://go-review.googlesource.com/c/154302Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

Change-Id: Id71aad4cf6149e0ba15f7fec0b74517827c37866
Reviewed-on: https://go-review.googlesource.com/c/154303Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

We shouldn't pollute people's flags with this debugging flag that was
never really meant to be public. It's certainly not documented.

So keep it for now, but don't register it unless it looks like it's in
use (by looking at os.Args). Kinda gross, but less gross than before.

Fixes #28619

Change-Id: I47498948a26a71ff36f9658a6d9dac73fd0a3016
Reviewed-on: https://go-review.googlesource.com/c/154217
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>

Updates to x/net git rev 891ebc4b82d6e74f468c533b06f983c7be918a96 for:

   http2/hpack: track the beginning of a header block
   https://go-review.googlesource.com/c/153978

Updates golang/go#29187

Change-Id: Ie2568b3f8d6aaa3f097a4ac25d3acdc794f5ff5c
Reviewed-on: https://go-review.googlesource.com/c/154118
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matt Layher <mdlayher@gmail.com>

Previously, RepoRootForImportPath trimmed certain "..." wildcards from
package patterns (even though its name suggests that the argument must
be an actual import path). It trimmed at the first path element that
was literally "..." (although wildcards in general may appear within a
larger path element), and relied on a subsequent check in
RepoRootForImportPath to catch confusing resolutions.

However, that causes 'go get' with wildcard patterns in fresh paths to
fail as of CL 154101: a wildcard pattern is not a valid import path,
and fails the path check. (The existing Test{Vendor,Go}Get* packages
in go_test.go and vendor_test.go catch the failure, but they are all
skipped when the "-short" flag is set — including in all.bash — and we
had forgotten to run them separately.)

We now trim the path before any element that contains a wildcard, and
perform the path check (and repo resolution) on only that prefix. It
is possible that the expanded path after fetching the repo will be
invalid, but a repository can contain directories that are not valid
import paths in general anyway.

Fixes #29241

Change-Id: I70fb2f7fc6603b7d339fd6c02e8cdeacfc93fc4b
Reviewed-on: https://go-review.googlesource.com/c/154108Reviewed-by: Russ Cox <rsc@golang.org>

Replaces //go:linkname by assembly functions for syscall
functions on aix/ppc64.
Since the new runtime internal ABI, this was triggering an error if
syscall.Syscall6 was called by others packages like x/sys/unix.
This commit should remove every future occurences of this problem.

Fixes #28769

Change-Id: I6a4bf77472ee1e974bdb76b27e74275e568f5a76
Reviewed-on: https://go-review.googlesource.com/c/153997
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

Instead of testing len(slice)+numNewElements > cap(slice) use
uint(len(slice)+numNewElements) > uint(cap(slice)) to test
if a slice needs to be grown in an append operation.

This prevents a possible overflow when len(slice) is near the maximum
int value and the addition of a constant number of new elements
makes it overflow and wrap around to a negative number which is
smaller than the capacity of the slice.

Appending a slice to a slice with append(s1, s2...) already used
a uint comparison to test slice capacity and therefore was not
vulnerable to the same overflow issue.

Fixes: #29190

Change-Id: I41733895838b4f80a44f827bf900ce931d8be5ca
Reviewed-on: https://go-review.googlesource.com/c/154037
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Some of the builders cannot infer user and email from the builder hostname.

Change-Id: I27e5d011fa1471f27763b6b7fa1bf59e418b925c
Reviewed-on: https://team-review.git.corp.google.com/c/376739Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154107Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I3fe44887a84586d73be01df78a9cbb002c1fc9c5
Reviewed-on: https://team-review.git.corp.google.com/c/376465Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154106Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

That number grows quadratically with the number of intermediate
certificates in certain pathological cases (for example if they all have
the same Subject) leading to a CPU DoS. Set a fixed budget that should
fit all real world chains, given we only look at intermediates provided
by the peer.

The algorithm can be improved, but that's left for follow-up CLs:

    * the cache logic should be reviewed for correctness, as it seems to
      override the entire chain with the cached one
    * the equality check should compare Subject and public key, not the
      whole certificate
    * certificates with the right SKID but the wrong Subject should not
      be considered, and in particular should not take priority over
      certificates with the right Subject

Fixes #29233

Change-Id: Ib257c12cd5563df7723f9c81231d82b882854213
Reviewed-on: https://team-review.git.corp.google.com/c/370475Reviewed-by: Andrew Bonventre <andybons@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154105Reviewed-by: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>

This fixes a regression of #18660 with the new path checks.

Updates #29230

Change-Id: I2dd9adab999e7f810e0e746ad8b75ea9622f56e7
Reviewed-on: https://team-review.git.corp.google.com/c/370578Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154104Reviewed-by: Bryan C. Mills <bcmills@google.com>

This should be a no-op, but produces deterministic (and more correct)
behavior if we have accidentally failed to sanitize one of the inputs.

Updates #29231

Change-Id: I1271d0ffd01a691ec8c84906c4e02d9e2be19c72
Reviewed-on: https://team-review.git.corp.google.com/c/370575Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154103Reviewed-by: Bryan C. Mills <bcmills@google.com>

Updates #29230

Change-Id: Ia32d8ec1fc0c4e242f50d8871c0ef3ce315f3c65
Reviewed-on: https://team-review.git.corp.google.com/c/370571Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154102Reviewed-by: Bryan C. Mills <bcmills@google.com>

On some platforms, directories beginning with dot are treated as
hidden files, and filenames containing unusual characters can be
confusing for users to manipulate (and delete).

Fixes #29230
Fixes #29231

Change-Id: Ic6f97f577d8fafa83ef62438095a5c7ae022881a
Reviewed-on: https://team-review.git.corp.google.com/c/368507Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-on: https://go-review.googlesource.com/c/154101Reviewed-by: Bryan C. Mills <bcmills@google.com>

To fix #5043, we added logic to allow balanced pairs of parenthesis
so that we could match URLs like:
	http://example.com/some_resource(foo)

Howewer, such logic breaks when parsing something like the following:
	art by [https://example.com/person][Person Name]].
such that the following is considered the link:
	https://example.com/person][Person

Since the logic added in #5043 was just a heuristic, we adjust
the heuristic that in addition to requiring balanced pairs,
the first parenthesis must be an opening one.

For further robustness, we apply this heuristic to
parenthesis, braces, and brackets.

Fixes #22285

Change-Id: I23b728a644e35ce3995b05a79129cad2c1e3b1ce
Reviewed-on: https://go-review.googlesource.com/c/94876
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Fixes #26532

Change-Id: Ic086c90503c7b24982f947c828c7ccf016ddbf69
Reviewed-on: https://go-review.googlesource.com/c/154120
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #26053

Change-Id: Ic2052b1d0d4e0826a217a520c83d7bb0995ea72a
GitHub-Last-Rev: 5a3eea3dd22b0a194591ce2294b88cb340df1e8d
GitHub-Pull-Request: golang/go#26054
Reviewed-on: https://go-review.googlesource.com/c/120835Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>