This is based on the crash dump provided by Alan
and on mental experiments:

sweep 0 74
fatal error: gc: unswept span
runtime stack:
runtime.throw(0x9df60d)
markroot(0xc208002000, 0x3)
runtime.parfordo(0xc208002000)
runtime.gchelper()

I think that when we moved all stacks into heap,
we introduced a bunch of bad data races. This was later
worsened by parallel stack shrinking.

Observation 1: exitsyscall can allocate a stack from heap at any time (including during STW).
Observation 2: parallel stack shrinking can (surprisingly) grow heap during marking.
Consider that we steadily grow stacks of a number of goroutines from 8K to 16K.
And during GC they all can be shrunk back to 8K. Shrinking will allocate lots of 8K
stacks, and we do not necessary have that many in heap at this moment. So shrinking
can grow heap as well.

Consequence: any access to mheap.allspans in GC (and otherwise) must take heap lock.
This is not true in several places.

Fix this by protecting accesses to mheap.allspans and introducing allspans cache for marking,
similar to what we use for sweeping.

LGTM=rsc
R=golang-codereviews, rsc
CC=adonovan, golang-codereviews, khr, rlh
https://golang.org/cl/126510043

Cache unrolled GC bitmask for types up to 64/32K on 64/32-bit systems,
this corresponds to up to 4K cached bitmask.
Perf builders say that 2% of time is spent in unrollgcproginplace_m/unrollgcprog1
on http benchmark:
http://goperfd.appspot.com/log/f42045f45bf61a0da53b724a7c8567824a0ad6c9

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews, khr, rlh
https://golang.org/cl/122680043

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews, khr
https://golang.org/cl/132090043

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews, khr
https://golang.org/cl/132100043

The low-level implementation of divide on ARM assumes that
it can panic with an error created by newErrorCString without
allocating. If we make interface data words require pointer values,
the current definition would require an allocation when stored
in an interface. Changing the definition to use unsafe.Pointer
instead of uintptr avoids the allocation. This change is okay
because the field really is a pointer (to a C string in rodata).

Update #8405.

This should make CL 133830043 safe to try again.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=dave, golang-codereviews, r
https://golang.org/cl/133820043

This change broke divmod.go on all arm platforms.

««« original CL description
cmd/gc: change interface representation: only pointers in data word

Note that there are various cleanups that can be made if we keep
this change, but I do not want to start making changes that
depend on this one until the 1.4 cycle closes.

Fixes #8405.

LGTM=r
R=golang-codereviews, adg, r, bradfitz
CC=golang-codereviews, iant
https://golang.org/cl/130240043
»»»

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/133810043

This is a very dumb translation to keep the code as close to the original C as possible.

LGTM=rsc
R=khr, minux, rsc, josharian
CC=golang-codereviews
https://golang.org/cl/126490043

Note that there are various cleanups that can be made if we keep
this change, but I do not want to start making changes that
depend on this one until the 1.4 cycle closes.

Fixes #8405.

LGTM=r
R=golang-codereviews, adg, r, bradfitz
CC=golang-codereviews, iant
https://golang.org/cl/130240043

LGTM=bradfitz
R=daniel.morsing, bradfitz
CC=golang-codereviews
https://golang.org/cl/130500044

This makes newproc invisible to the GC. This is a pretty simple change since parts of newproc already depends on being run on the M stack.

LGTM=dvyukov
R=golang-codereviews, dvyukov
CC=golang-codereviews, khr
https://golang.org/cl/129520043

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/126540043

Note is required for timers and heap scavenger.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews, khr, rlh
https://golang.org/cl/128620043

The current code is correct, but vet does not understand it:
asm_amd64.s:963: [amd64] invalid MOVL of ret+0(FP); int64 is 8-byte value
asm_amd64.s:964: [amd64] invalid offset ret+4(FP); expected ret+0(FP)

LGTM=minux
R=golang-codereviews, minux
CC=golang-codereviews
https://golang.org/cl/125200044

Fix issue by always appending newline after user input, before
the closing curly bracket. The adjust func is modified to remove
this new newline.

Add test case (it fails before CL, passes after).

Fixes #8411.

LGTM=gri
R=golang-codereviews, bradfitz, josharian, gri
CC=golang-codereviews
https://golang.org/cl/124700043

LGTM=khr
R=khr, dvyukov, dave
CC=golang-codereviews, rsc
https://golang.org/cl/124300044

Fixes #8503.

Thanks to no.smile.face for the original report.

LGTM=bradfitz, r, ruiu
R=bradfitz, ruiu, r
CC=golang-codereviews
https://golang.org/cl/132730043

Add gofmt.go and gofmt_test.go as they are part of the test data set.

See CL 130440043.

LGTM=dvyukov
R=golang-codereviews, dvyukov
CC=golang-codereviews
https://golang.org/cl/132820044

Timer callback does not accept now as first arg anymore.

LGTM=dave
R=golang-codereviews, dave
CC=golang-codereviews
https://golang.org/cl/126530043

Fixes #8145.

LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/126440043

1) Interpret a comment of the form

//gofmt <flags>

in test files to drive the respective
gofmt command. Eliminates the need to
enumerate all test files in the test
harness.

2) Added -update flag to make it easier
to update test cases.

LGTM=josharian
R=golang-codereviews, josharian
CC=golang-codereviews
https://golang.org/cl/130440043

Fixes #8543.

LGTM=dvyukov, r
R=rsc, r
CC=dvyukov, golang-codereviews
https://golang.org/cl/123580043

Update #8527

Fixes, src/cmd/6l/../ld/pcln.c:93:18: runtime error: left shift of negative value -2

LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/127440043

Generated by a+c.

R=gobot
CC=golang-codereviews
https://golang.org/cl/130430043

This files were added accidentally and are
not required for running the tests (they
are produced by failing tests for easier
debugging).

LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/131030044

Fixes compilation of runtime on Solaris where the inner struct
was not called "_4_".

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/129460043

Australian timezones abbreviation for standard and daylight saving time were recently
changed from EST for both to AEST and AEDT in the icann tz database (see changelog
on www.iana.org/time-zones).

A test in the time package was written to check that the ParseInLocation function
understand that Feb EST and Aug EST are different time zones, even though they are
both called EST. This is no longer the case, and the Date function now returns
AEST or AEDT for australian tz on every Linux system with an up to date tz database
(and this makes the test fail).

Since I wasn't able to find another country that 1) uses daylight saving and 2) has
the same abbreviation for both on tzdata, I changed the test to make sure that
ParseInLocation does not get confused when it parses, in different locations, two
dates with the same abbreviation (this was suggested in the mailing list).

Fixes #8547.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/130920043

Generated by a+c.

R=gobot
CC=golang-codereviews
https://golang.org/cl/125660043

LGTM=khr
R=khr
CC=dvyukov, golang-codereviews
https://golang.org/cl/132770044

Fixes #8553.

LGTM=bradfitz
R=golang-codereviews, dave, bradfitz
CC=golang-codereviews
https://golang.org/cl/132750043

Generated by a+c.

R=gobot
CC=golang-codereviews
https://golang.org/cl/126500044

It is called from Go only in tests.

LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, rlh, rsc
https://golang.org/cl/125610043

Cleanup before converting to Go.
Fortunately nobody using it, because it is incorrect:
monotonic runtime time instead of claimed real time.

LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, rsc
https://golang.org/cl/129480043

These are required for chans, semaphores, timers, etc.

LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, rlh, rsc
https://golang.org/cl/123640043

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/128500043

UndefinedBehaviorSanitizer claims it is UB in C:
src/cmd/gc/racewalk.c:422:37: runtime error: member access within null pointer of type 'Node' (aka 'struct Node')
src/cmd/gc/racewalk.c:423:37: runtime error: member access within null pointer of type 'Node' (aka 'struct Node')

LGTM=rsc
R=dave, rsc
CC=golang-codereviews
https://golang.org/cl/125570043

Init GC later as it needs to read GOGC env var.
Fixes #8562.

LGTM=daniel.morsing, rsc
R=golang-codereviews, daniel.morsing, rsc
CC=golang-codereviews, khr, rlh
https://golang.org/cl/130990043

Calling ReadMemStats which does stoptheworld on m0 holding locks
was not a good idea.
Stoptheworld holding locks is a recipe for deadlocks (added check for this).
Stoptheworld on g0 may or may not work (added check for this as well).
As far as I understand scavenger will print incorrect numbers now,
as stack usage is not subtracted from heap. But it's better than deadlocking.

LGTM=khr
R=golang-codereviews, rsc, khr
CC=golang-codereviews, rlh
https://golang.org/cl/124670043

Fixes build.

TBR=khr
R=golang-codereviews
CC=golang-codereviews, khr
https://golang.org/cl/130390044

LGTM=khr
R=golang-codereviews, bradfitz, khr
CC=golang-codereviews
https://golang.org/cl/123680043

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/128230047