Change-Id: Id1e30d70d6891ef12110f8e7832b94eeac9e2fa9
Reviewed-on: https://go-review.googlesource.com/13250Reviewed-by: Russ Cox <rsc@golang.org>

88e945fd introduced a non-speculative double check of the heap trigger
before actually starting a concurrent GC. This was necessary to fix a
race for heap-triggered GC, but broke sysmon-triggered periodic GC,
since the heap check will of course fail for periodically triggered
GC.

Fix this by telling startGC whether or not this GC was triggered by
heap size or a timer and only doing the heap size double check for GCs
triggered by heap size.

Fixes #12026.

Change-Id: I7c3f6ec364545c36d619f2b4b3bf3b758e3bcbd6
Reviewed-on: https://go-review.googlesource.com/13168Reviewed-by: Russ Cox <rsc@golang.org>

First step towards cleaning up the operator section - no language
changes. Specifically:

- Grouped arithmetic operations by types (integer, floating-point,
  string), with corresponding h4 headings.

- Changed Operator precedence title from h3 to h4.

- Moved Integer Overflow section after integer operations and changed
  its title from h3 to h4.

This puts things that belong together closer. No heading id's were
lost (in case of references from outside the spec).

Change-Id: I6b349ba8d86a6ae29b596beb297cc45c81e69399
Reviewed-on: https://go-review.googlesource.com/13143Reviewed-by: Rob Pike <r@golang.org>

Inconsistency identified by Anmol Sethi (anmol@aubble.com).

Fixes #10341.

Change-Id: I1a1f5b22aad29b56280f81026feaa37a61b3e0a9
Reviewed-on: https://go-review.googlesource.com/13132Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #9837.

Change-Id: Ia513c7e5db221eee8e3ab0affa6d3688d2099fd9
Reviewed-on: https://go-review.googlesource.com/13130Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #10514.

Change-Id: Iae95a304d3ebb1ed82567aa234e05dc434db984f
Reviewed-on: https://go-review.googlesource.com/13098Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Missed in CL 13074.

Change-Id: Ic0600341abbc423cd8d7b2201bf50e3b0bf398a7
Reviewed-on: https://go-review.googlesource.com/13167Reviewed-by: Ian Lance Taylor <iant@golang.org>

Now that it works we need to turn it back on.

Fixes #10119.

Change-Id: I9c62d3026f7bb62c49a601ad73f33bf655372915
Reviewed-on: https://go-review.googlesource.com/13162Reviewed-by: Ian Lance Taylor <iant@golang.org>

It is just far too slow.
I have a CL for Go 1.6 that makes many of these into internal tests.
That will improve the coverage.

It does not matter much, because basically none of the go command
tests are architecture dependent, so the other builders will catch
any problems.

Fixes freebsd-arm builder.

Change-Id: I8b2f6ac2cc1e7657019f7731c6662dc43e20bfb5
Reviewed-on: https://go-review.googlesource.com/13166Reviewed-by: Ian Lance Taylor <iant@golang.org>

This works after golang.org/cl/13120 is running on the
coordinator (maybe it already is).

Change-Id: I4053d8e2f32fafd47b927203a6f66d5858e23376
Reviewed-on: https://go-review.googlesource.com/13165Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I4e8c20284255e0e17b6fb72475d2d37f49994788
Reviewed-on: https://go-review.googlesource.com/13113Reviewed-by: Rob Pike <r@golang.org>

Tracing functionality was moved from runtime/pprof to runtime/trace.

Change-Id: I694e0f209d043c7ffecb113f1825175bf963dde3
Reviewed-on: https://go-review.googlesource.com/13074Reviewed-by: Rob Pike <r@golang.org>

This is what is causing freebsd/arm to crash mysteriously when using cgo.
The bug was introduced in golang.org/cl/4030, which moved this code out
of rt0_go and into its own function. The ARM ABI says that calls must
be made with the stack pointer at an 8-byte boundary, but only FreeBSD
seems to crash when this is violated.

Fixes #10119.

Change-Id: Ibdbe76b2c7b80943ab66b8abbb38b47acb70b1e5
Reviewed-on: https://go-review.googlesource.com/13161Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>

This change allows the download page to redirect the user to
/doc/install?download=filename so the user can see installation
instructions specific to the file they are downloading.

This change also expands the "Test your Go installation" section
to instruct the user to create a workspace, hopefully leading
to less confusion down the line.

It also changes the front page download link to go directly
to the downloads page, which will in turn take them to the
installation instructions (the original destination).

This is related to this change to the tools repo:
https://golang.org/cl/13180

Change-Id: I658327bdb93ad228fb1846e389b281b15da91b1d
Reviewed-on: https://go-review.googlesource.com/13151Reviewed-by: Chris Broadfoot <cbro@golang.org>

Fixes #11995

Change-Id: I9e2901d77ebde705f59822e7d4a8163cbacffcd7
Reviewed-on: https://go-review.googlesource.com/13150Reviewed-by: Rob Pike <r@golang.org>

Fixes #12017.

Change-Id: I3dfcf9d0b62cae02eca1973383f0aad286a6ef4d
Reviewed-on: https://go-review.googlesource.com/13136Reviewed-by: Keith Randall <khr@golang.org>

Change-Id: I66f7937b22bb6e05c3f2f0f2a057151020ad9699
Reviewed-on: https://go-review.googlesource.com/13049Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

When commit 510fd135 enabled assists during the scan phase, it failed
to also update the code in the GC controller that computed the assist
CPU utilization and adjusted the trigger based on it. Fix that code so
it uses the start of the scan phase as the wall-clock time when
assists were enabled rather than the start of the mark phase.

Change-Id: I05013734b4448c3e2c730dc7b0b5ee28c86ed8cf
Reviewed-on: https://go-review.googlesource.com/13048Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

At the start of a GC cycle, the garbage collector computes the assist
ratio based on the total scannable heap size. This was intended to be
conservative; after all, this assumes the entire heap may be reachable
and hence needs to be scanned. But it only assumes that the *current*
entire heap may be reachable. It fails to account for heap allocated
during the GC cycle. If the trigger ratio is very low (near zero), and
most of the heap is reachable when GC starts (which is likely if the
trigger ratio is near zero), then it's possible for the mutator to
create new, reachable heap fast enough that the assists won't keep up
based on the assist ratio computed at the beginning of the cycle. As a
result, the heap can grow beyond the heap goal (by hundreds of megs in
stress tests like in issue #11911).

We already have some vestigial logic for dealing with situations like
this; it just doesn't run often enough. Currently, every 10 ms during
the GC cycle, the GC revises the assist ratio. This was put in before
we switched to a conservative assist ratio (when we really were using
estimates of scannable heap), and it turns out to be exactly what we
need now. However, every 10 ms is far too infrequent for a rapidly
allocating mutator.

This commit reuses this logic, but replaces the 10 ms timer with
revising the assist ratio every time the heap is locked, which
coincides precisely with when the statistics used to compute the
assist ratio are updated.

Fixes #11911.

Change-Id: I377b231ab064946228378fa10422a46d1b50f4c5
Reviewed-on: https://go-review.googlesource.com/13047Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This was useful in debugging the mutator assist behavior for #11911,
and it fits with the other gcpacertrace output.

Change-Id: I1e25590bb4098223a160de796578bd11086309c7
Reviewed-on: https://go-review.googlesource.com/13046Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Proportional concurrent sweep is currently based on a ratio of spans
to be swept per bytes of object allocation. However, proportional
sweeping is performed during span allocation, not object allocation,
in order to minimize contention and overhead. Since objects are
allocated from spans after those spans are allocated, the system tends
to operate in debt, which means when the next GC cycle starts, there
is often sweep debt remaining, so GC has to finish the sweep, which
delays the start of the cycle and delays enabling mutator assists.

For example, it's quite likely that many Ps will simultaneously refill
their span caches immediately after a GC cycle (because GC flushes the
span caches), but at this point, there has been very little object
allocation since the end of GC, so very little sweeping is done. The
Ps then allocate objects from these cached spans, which drives up the
bytes of object allocation, but since these allocations are coming
from cached spans, nothing considers whether more sweeping has to
happen. If the sweep ratio is high enough (which can happen if the
next GC trigger is very close to the retained heap size), this can
easily represent a sweep debt of thousands of pages.

Fix this by making proportional sweep proportional to the number of
bytes of spans allocated, rather than the number of bytes of objects
allocated. Prior to allocating a span, both the small object path and
the large object path ensure credit for allocating that span, so the
system operates in the black, rather than in the red.

Combined with the previous commit, this should eliminate all sweeping
from GC start up. On the stress test in issue #11911, this reduces the
time spent sweeping during GC (and delaying start up) by several
orders of magnitude:

                mean    99%ile     max
    pre fix      1 ms    11 ms   144 ms
    post fix   270 ns   735 ns   916 ns

Updates #11911.

Change-Id: I89223712883954c9d6ec2a7a51ecb97172097df3
Reviewed-on: https://go-review.googlesource.com/13044Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Currently it's possible for the next_gc heap size trigger computed for
the next GC cycle to be less than the current allocated heap size.
This means the next cycle will start immediately, which means there's
no time to perform the concurrent sweep between GC cycles. This places
responsibility for finishing the sweep on GC itself, which delays GC
start-up and hence delays mutator assist.

Fix this by ensuring that next_gc is always at least a little higher
than the allocated heap size, so we won't trigger the next cycle
instantly.

Updates #11911.

Change-Id: I74f0b887bf187518d5fedffc7989817cbcf30592
Reviewed-on: https://go-review.googlesource.com/13043Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Currently there are two sensitive periods during which a mutator can
allocate past the heap goal but mutator assists can't be enabled: 1)
at the beginning of GC between when the heap first passes the heap
trigger and sweep termination and 2) at the end of GC between mark
termination and when the background GC goroutine parks. During these
periods there's no back-pressure or safety net, so a rapidly
allocating mutator can allocate past the heap goal. This is
exacerbated if there are many goroutines because the GC coordinator is
scheduled as any other goroutine, so if it gets preempted during one
of these periods, it may stay preempted for a long period (10s or 100s
of milliseconds).

Normally the mutator does scan work to create back-pressure against
allocation, but there is no scan work during these periods. Hence, as
a fall back, if a mutator would assist but can't yet, simply yield the
CPU. This delays the mutator somewhat, but more importantly gives more
CPU time to the GC coordinator for it to complete the transition.

This is obviously a workaround. Issue #11970 suggests a far better but
far more invasive way to fix this.

Updates #11911. (This very nearly fixes the issue, but about once
every 15 minutes I get a GC cycle where the assists are enabled but
don't do enough work.)

Change-Id: I9768b79e3778abd3e06d306596c3bd77f65bf3f1
Reviewed-on: https://go-review.googlesource.com/13026Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Currently allocation checks the GC trigger speculatively during
allocation and then triggers the GC without rechecking. As a result,
it's possible for G 1 and G 2 to detect the trigger simultaneously,
both enter startGC, G 1 actually starts GC while G 2 gets preempted
until after the whole GC cycle, then G 2 immediately starts another GC
cycle even though the heap is now well under the trigger.

Fix this by re-checking the GC trigger non-speculatively just before
actually kicking off a new GC cycle.

This contributes to #11911 because when this happens, we definitely
don't finish the background sweep before starting the next GC cycle,
which can significantly delay the start of concurrent scan.

Change-Id: I560ab79ba5684ba435084410a9765d28f5745976
Reviewed-on: https://go-review.googlesource.com/13025Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Change-Id: Ifac10621fece766f3a0e8551e98d1f8d7072852f
Reviewed-on: https://go-review.googlesource.com/13068Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

I accidentally submitted https://golang.org/cl/13080 too early.

Update #11955.

Change-Id: I1a5a6860bb46bc4bc6fd278f8a867d2dd9e411e1
Reviewed-on: https://go-review.googlesource.com/13096Reviewed-by: Andrew Gerrand <adg@golang.org>

Do not assume that if stat shows multiple links that we should mark the
file as a hardlink in the tar format.  If the hardlink link was not
referenced, this caused a link to "/".  On an overlay file system, all
files have multiple links.

The caller must keep the inode references and set TypeLink, Size = 0,
and LinkName themselves.

Change-Id: I873b8a235bc8f8fbb271db74ee54232da36ca013
Reviewed-on: https://go-review.googlesource.com/13045Reviewed-by: Ian Lance Taylor <iant@golang.org>

The buildmode docs mention exported functions, but don't say anything
about how to export them.  Mention the cgo tool to make this somewhat
clearer.

Fixes #11955.

Change-Id: Ie5420445daa87f5aceec6ad743465d5d32d0a786
Reviewed-on: https://go-review.googlesource.com/13080Reviewed-by: Russ Cox <rsc@golang.org>

For niceness, when go/exact was moved from x/tools, it
was renamed go/constant.

For simplicity, when go/types was moved from x/tools, its
imports of (now) go/constant were done with a rename:

    import exact "go/constant"

This kept the code just as it was before and avoided the issue
of what to call the internal constant called, um, constant.

But not all was hidden, as the text of some fields of structs and
the like leaked the old name, so things like "exact.Value" appeared
in type definitions and function signatures in the documentation.
This is unacceptable.

Fix the documentation issue by fixing the code. Rename the constant
constant constant_, and remove the renaming import.

This should go into 1.5. It's mostly a mechanical change, is
internal to the package, and fixes the documentation. It contains
no semantic changes except to fix a benchmark that was broken
in the original transition.

Fixes #11949.

Change-Id: Ieb94b6558535b504180b1378f19e8f5a96f92d3c
Reviewed-on: https://go-review.googlesource.com/13051Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: Ia5d41b66006682084fcbfac3da020946ea3dd116
Reviewed-on: https://go-review.googlesource.com/13093Reviewed-by: Andrew Gerrand <adg@golang.org>

Change-Id: Ia21501df23a91c065d9f2acc6f043019a1419b22
Reviewed-on: https://go-review.googlesource.com/13092Reviewed-by: Andrew Gerrand <adg@golang.org>

I walked through the steps for a contribution but ended up
with an error when doing "git mail" because I didn't have a
signed agreement.

Added a section to check for or create one through Gerrit right
after the user has created the account and logged in.

Moved some info from copyright section to the new section.

Change-Id: I79bbd3e18fc3a742fa59a242085da14be9e19ba0
Reviewed-on: https://go-review.googlesource.com/13062Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>

Change-Id: I4dc75065038a9cfd06f61c0deca1c86c70713d3a
Reviewed-on: https://go-review.googlesource.com/13091Reviewed-by: Andrew Gerrand <adg@golang.org>

This was confusing when I was trying to fix go build -o.
Perhaps due to that fix, this can now be simplified from
three functions to one.

Change-Id: I878a6d243b14132a631e7c62a3bb6d101bc243ea
Reviewed-on: https://go-review.googlesource.com/13027Reviewed-by: Ian Lance Taylor <iant@golang.org>

Quoting the new docs:

«
If the arguments to build are a list of .go files, build treats
them as a list of source files specifying a single package.

When compiling a single main package, build writes
the resulting executable to an output file named after
the first source file ('go build ed.go rx.go' writes 'ed' or 'ed.exe')
or the source code directory ('go build unix/sam' writes 'sam' or 'sam.exe').
The '.exe' suffix is added when writing a Windows executable.

When compiling multiple packages or a single non-main package,
build compiles the packages but discards the resulting object,
serving only as a check that the packages can be built.

The -o flag, only allowed when compiling a single package,
forces build to write the resulting executable or object
to the named output file, instead of the default behavior described
in the last two paragraphs.
»

There is a change in behavior here, namely that 'go build -o x.a x.go'
where x.go is not a command (not package main) did not write any
output files (back to at least Go 1.2) but now writes x.a.
This seems more reasonable than trying to explain that -o is
sometimes silently ignored.

Otherwise the behavior is unchanged.

The lines being deleted in goFilesPackage look like they are
setting up 'go build x.o' to write 'x.a', but they were overridden
by the p.target = "" in runBuild. Again back to at least Go 1.2,
'go build x.go' for a non-main package has never produced
output. It seems better to keep it that way than to change it,
both for historical consistency and for consistency with
'go build strings' and 'go build std'.

All of this behavior is now tested.

Fixes #10865.

Change-Id: Iccdf21f366fbc8b5ae600a1e50dfe7fc3bff8b1c
Reviewed-on: https://go-review.googlesource.com/13024Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dave Day <djd@golang.org>

It was failing with multiple goroutines a few out of every thousand
runs (with errRequestCanceled) because it was using the same
*http.Request for all 5 RoundTrips, but the RoundTrips' goroutines
(notably the readLoop method) were all still running, sharing that
same pointer. Because the response has no body (which is what
TestZeroLengthPostAndResponse tests), the readLoop was marking the
connection as reusable early (before the caller read until the body's
EOF), but the Transport code was clearing the Request's cancelation
func *AFTER* the caller had already received it from RoundTrip. This
let the test continue looping and do the next request with the same
pointer, fetch a connection, and then between getConn and roundTrip
have an invariant violated: the Request's cancelation func was nil,
tripping this check:

        if !pc.t.replaceReqCanceler(req.Request, pc.cancelRequest) {
                pc.t.putIdleConn(pc)
                return nil, errRequestCanceled
        }

The solution is to clear the request cancelation func in the readLoop
goroutine in the no-body case before it's returned to the caller.

This now passes reliably:

$ go test -race -run=TestZeroLengthPostAndResponse -count=3000

I think we've only seen this recently because we now randomize scheduling
of goroutines in race mode (https://golang.org/cl/11795). This race
has existed for a long time but the window was hard to hit.

Change-Id: Idb91c582919f85aef5b9e5ef23706f1ba9126e9a
Reviewed-on: https://go-review.googlesource.com/13070
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Introduced in https://go-review.googlesource.com/12865 (git rev c2db5f4c).

This fix doesn't add any new lock acquistions: it just moves the
existing one taken by the unreadDataSize method and moves it out
wider.

It became flaky at rev c2db5f4c, but now reliably passes again:
$ go test -v -race -run=TestTransportAndServerSharedBodyRace -count=100 net/http

Fixes #11985

Change-Id: I6956d62839fd7c37e2f7441b1d425793f4a0db30
Reviewed-on: https://go-review.googlesource.com/12909
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Updates #11990.

Change-Id: I6c58923a1b5a3805acfb6e333e3c9e87f4edf4ba
Reviewed-on: https://go-review.googlesource.com/13050Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #11973.

Change-Id: Icffa3213246663982b7cc795982e0923e272f405
Reviewed-on: https://go-review.googlesource.com/12919Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I992cb1afeef498353d529238e508fa438d6c069c
Reviewed-on: https://go-review.googlesource.com/12912Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>