Previously, many error messages inconsistantly used either lexlineno
and lineno.  In general this works out okay because they're almost
always the same.  The only exceptional case is after lexing a
multi-line raw string literal, where lineno will be the line number of
the opening quote and lexlineno is the line number of the closing
quote.

This CL makes the compiler's error message more consistent:

- Lexer error messages related to invalid byte sequences (i.e., NUL
bytes, bad UTF-8 sequences, and non-initial BOMs) are emitted at
lexlineno (i.e., the source line that contains the invalid byte
sequence).

- All other error messages (notably the parser's "syntax errors") now
use lineno.  The minor change from this is that bogus input like:

    package `
    bogus`

will emit "syntax error: unexpected string literal, expecting name"
error at line 1, instead of line 2.

- Instead of maintaining prevlineno all the time, just record it
when/where actually needed and not already available elsewhere (which
turns out to be just one function).

- Lastly, we remove the legacy "syntax error near ..." fallback in
Yerror, now that the parser always emits more detailed syntax error
messages.

Change-Id: Iaf5f784223d0385fa3a5b09ef2b2ad447feab02f
Reviewed-on: https://go-review.googlesource.com/19925Reviewed-by: Robert Griesemer <gri@golang.org>

REP-prefixed instructions have a large startup cost.
Avoid them like the plague.

benchmark                  old ns/op     new ns/op     delta
BenchmarkIndexByte10-8     22.4          5.34          -76.16%

Fixes #13983

Change-Id: I857e956e240fc9681d053f2584ccf24c1b272bb3
Reviewed-on: https://go-review.googlesource.com/18703Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #14498.

Change-Id: I4cfab3e45898466179cefbd31c6f7f796da82363
Reviewed-on: https://go-review.googlesource.com/19874Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

_Genqueue and _Gscanenqueue were introduced as part of the GC quiesce
code. The quiesce code was removed by 197aa9e6, but these states and
some associated code stuck around. Remove them.

Change-Id: I69df81881602d4a431556513dac2959668d27c20
Reviewed-on: https://go-review.googlesource.com/19638Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I1fa5e629b2890a8509559ce4ea17b74f47d71925
Reviewed-on: https://go-review.googlesource.com/19637Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently most uses of gcWork use the per-P gcWork, but there are two
places that still use a stack-based gcWork. Simplify things by making
these instead use the per-P gcWork.

Change-Id: I712d012cce9dd5757c8541824e9641ac1c2a329c
Reviewed-on: https://go-review.googlesource.com/19636Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently markroot uses a gcWork on the stack and disposes of it
immediately after marking one root. This used to be necessary because
markroot was called from the depths of parfor, but now that we call it
directly and have ready access to a gcWork at the call site, pass the
gcWork in, use it directly in markroot, and share it across calls to
markroot from the same P.

Change-Id: Id7c3b811bfb944153760e01873c07c8d18909be1
Reviewed-on: https://go-review.googlesource.com/19635Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>

When gcWork was first introduced, the compiler's escape analysis
wasn't good enough to detect that that method receiver didn't escape,
so we had to hack around this.

Now that the compiler can figure out this for itself, remove these
hacks.

Change-Id: I9f73fab721e272410b8b6905b564e7abc03c0dfe
Reviewed-on: https://go-review.googlesource.com/19634Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently we clear the heap bitmap of a span both when we allocate
that span *and* when we free it. There's no point in doing both, and
we definitely have to write the heap bitmap when we allocate a span
for pointer-sized objects, so switch to clearing only when we allocate
a span.

This results in a slight overall performance improvement; however,
most of the benchmarks that get slower are very short, while the
longer benchmarks generally got faster.

name              old time/op  new time/op  delta
XBenchGarbage-12  2.48ms ± 1%  2.47ms ± 1%  -0.58%  (p=0.000 n=91+91)

name                      old time/op    new time/op    delta
BinaryTree17-12              2.85s ± 2%     2.85s ± 2%    ~     (p=0.550 n=20+19)
Fannkuch11-12                2.54s ± 0%     2.47s ± 1%  -2.72%  (p=0.000 n=19+18)
FmtFprintfEmpty-12          51.3ns ± 4%    51.0ns ± 3%    ~     (p=0.223 n=20+20)
FmtFprintfString-12          169ns ± 0%     167ns ± 0%  -1.18%  (p=0.000 n=17+16)
FmtFprintfInt-12             160ns ± 0%     161ns ± 0%  +0.63%  (p=0.000 n=16+15)
FmtFprintfIntInt-12          267ns ± 0%     269ns ± 1%  +0.62%  (p=0.000 n=17+20)
FmtFprintfPrefixedInt-12     234ns ± 1%     240ns ± 0%  +2.80%  (p=0.000 n=20+20)
FmtFprintfFloat-12           316ns ± 0%     313ns ± 0%  -0.76%  (p=0.000 n=20+19)
FmtManyArgs-12              1.04µs ± 0%    1.05µs ± 0%  +0.45%  (p=0.000 n=19+16)
GobDecode-12                7.90ms ± 1%    7.81ms ± 0%  -1.10%  (p=0.000 n=18+18)
GobEncode-12                6.61ms ± 1%    6.58ms ± 0%  -0.46%  (p=0.000 n=20+15)
Gzip-12                      320ms ± 1%     322ms ± 1%  +0.47%  (p=0.030 n=20+20)
Gunzip-12                   42.4ms ± 1%    42.6ms ± 0%  +0.37%  (p=0.000 n=20+20)
HTTPClientServer-12         70.7µs ± 1%    70.6µs ± 2%    ~     (p=0.784 n=18+20)
JSONEncode-12               16.9ms ± 1%    16.8ms ± 0%  -0.64%  (p=0.000 n=20+20)
JSONDecode-12               60.8ms ± 0%    58.6ms ± 1%  -3.50%  (p=0.000 n=17+18)
Mandelbrot200-12            3.92ms ± 0%    3.91ms ± 0%  -0.25%  (p=0.000 n=19+19)
GoParse-12                  3.65ms ± 0%    3.68ms ± 1%  +0.67%  (p=0.000 n=17+16)
RegexpMatchEasy0_32-12       102ns ± 1%     102ns ± 2%  +0.67%  (p=0.009 n=19+19)
RegexpMatchEasy0_1K-12       350ns ± 0%     351ns ± 1%  +0.34%  (p=0.002 n=20+20)
RegexpMatchEasy1_32-12      84.1ns ± 2%    84.2ns ± 2%    ~     (p=0.799 n=20+18)
RegexpMatchEasy1_1K-12       510ns ± 1%     508ns ± 1%  -0.45%  (p=0.000 n=20+17)
RegexpMatchMedium_32-12      132ns ± 1%     134ns ± 1%  +0.85%  (p=0.000 n=20+19)
RegexpMatchMedium_1K-12     40.0µs ± 1%    39.9µs ± 1%  -0.29%  (p=0.014 n=19+18)
RegexpMatchHard_32-12       2.09µs ± 1%    2.05µs ± 0%  -1.76%  (p=0.000 n=20+18)
RegexpMatchHard_1K-12       62.7µs ± 1%    61.8µs ± 1%  -1.39%  (p=0.000 n=20+19)
Revcomp-12                   541ms ± 1%     534ms ± 0%  -1.16%  (p=0.000 n=19+20)
Template-12                 71.1ms ± 0%    69.1ms ± 0%  -2.83%  (p=0.000 n=18+19)
TimeParse-12                 356ns ± 0%     357ns ± 0%  +0.36%  (p=0.000 n=17+19)
TimeFormat-12                358ns ± 0%     372ns ± 1%  +3.74%  (p=0.000 n=15+18)
[Geo mean]                  62.6µs         62.5µs       -0.25%

Change-Id: Ied190b77c7a4d91ec7b2218c592fc31cf7acf362
Reviewed-on: https://go-review.googlesource.com/19633Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The channel code must not allow stack splits between when it assigns a
potential stack pointer to sudog.elem (or sudog.selectdone) and when
it makes the sudog visible to copystack by putting it on the g.waiting
list. We do get this right everywhere, but add a comment about this
subtlety for future eyes.

Change-Id: I941da150437167acff37b0e56983c793f40fcf79
Reviewed-on: https://go-review.googlesource.com/19632Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I9c45aad1c35a99da4c3b8990649dcd962fd23b81
Reviewed-on: https://go-review.googlesource.com/19631Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently the heapBitsSweepSpan comment claims that heapBitsSweepSpan
sets the heap bitmap for the first two words to dead. In fact, it sets
the first *four* words to scalar/dead. This is important because first
two words don't actually have a dead bit, so for objects larger than
two words it *must* set a dead bit in third word to reset the object
to a "noscan" state. For example, we use this in heapBits.hasPointers
to detect that an object larger than two words is noscan.

Change-Id: Ie166a628bed5060851db083475c7377adb349d6c
Reviewed-on: https://go-review.googlesource.com/19630Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

go get -u all command updates all packages including standard
commands. We need to get commands evicted from their cache to
avoid loading old versions of the packages evicted from the
packages cache.

Fixes #14444

Change-Id: Icd581a26e1db34ca634aba595fed62b097094c2f
Reviewed-on: https://go-review.googlesource.com/19899Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #13767

Change-Id: Ib743db7d9d72022ea911bc5ac535243489425642
Reviewed-on: https://go-review.googlesource.com/18725Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I73ae6a6837a6dcf75b3b8f431d97a18348e01a42
Reviewed-on: https://go-review.googlesource.com/19921Reviewed-by: Ian Lance Taylor <iant@golang.org>

This indirectly implements a small fix for runtime/pprof: it used to
look for runtime.gopanic when it should have been looking for
runtime.sigpanic.

Update #11432.

Change-Id: I5e3f5203b2ac5463efd85adf6636e64174aacb1d
Reviewed-on: https://go-review.googlesource.com/19869
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

I can't remember just how this happened to me, but I got an unfortunate
crash with some set of cmd/compile debug options and source code.

Change-Id: Ibef6129c50b68dad0594ac439466bfbc4b32a095
Reviewed-on: https://go-review.googlesource.com/19920
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: Ie89c0945a4cc3aebfa9f7ad7f107bc7ab59ab61c
Reviewed-on: https://go-review.googlesource.com/19685Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I6b14b8eecf125dd74bd40f4e7fff6b49de150e42
Reviewed-on: https://go-review.googlesource.com/19897
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Keith Randall <khr@golang.org>

Simplifies some code as ptrToThis was unreliable under dynamic
linking. Now the same type lookup is used regardless of execution
mode.

A synthetic relocation, R_USETYPE, is introduced to make sure the
linker includes *T on use of T, if *T is carrying methods.

Changes the heap dump format. Anything reading the format needs to
look at the last bool of a type of an interface value to determine
if the type should be the pointer-to type.

Reduces binary size of cmd/go by 0.2%.
For #6853.

Change-Id: I79fcb19a97402bdb0193f3c7f6d94ddf061ee7b2
Reviewed-on: https://go-review.googlesource.com/19695Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Also added a test to ensure the behavior.

Fixes #14150

Change-Id: Ib3ee9fdae59826fa594ce1be3c49b51d740b56eb
Reviewed-on: https://go-review.googlesource.com/19915Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

It looks like the latest DragonFly BSD kernels, at least 4.4 and above,
have finished working on handling of shared IP control blocks. Let's
re-enbale test cases referring to IP control blocks and see what
happens.

Updates #13146.

Change-Id: Icbe2250e788f6a445a648541272c99b598c3013d
Reviewed-on: https://go-review.googlesource.com/19406Reviewed-by: Ian Lance Taylor <iant@golang.org>

It crashes when the node under the test is shaken up.

-- FAIL: TestGoLookupIPWithResolverConfig (11.73s)
panic: interface conversion: error is nil, not *net.DNSError [recovered]
	panic: interface conversion: error is nil, not *net.DNSError

goroutine 23 [running]:
panic(0x2e2620, 0xc820181440)
	/go/src/runtime/panic.go:483 +0x3f3
testing.tRunner.func1(0xc820136d80)
	/go/src/testing/testing.go:467 +0x192
panic(0x2e2620, 0xc820181440)
	/go/src/runtime/panic.go:441 +0x4f6
net.TestGoLookupIPWithResolverConfig(0xc820136d80)
	/go/src/net/dnsclient_unix_test.go:358 +0x7ca
testing.tRunner(0xc820136d80, 0x49ddc0)
	/go/src/testing/testing.go:473 +0x98
created by testing.RunTests
	/go/src/testing/testing.go:582 +0x892
exit status 2

Change-Id: I9631f41a3c73f3269c7e30d679c025ae64d71a98
Reviewed-on: https://go-review.googlesource.com/19870Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: Ic828256efe0f50a3e11a25d85092d7531b342d2e
Reviewed-on: https://go-review.googlesource.com/19873Reviewed-by: Ian Lance Taylor <iant@golang.org>

The actual values assigned to tokens was inherited from the yacc-based
grammar. With the most recent cleanups, all single-char tokens such as
commas, semis, parens, etc., that get returned from lexer.next simply
as their Unicode values are below utf8.RuneSelf (i.e., 7bit ASCII).
Lower the initial starting value for named token constants accordingly.

Change-Id: I7eb8e584dbb3bc7f9dab849d1b68a91320cffebd
Reviewed-on: https://go-review.googlesource.com/19913Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Introduced by (and missed during code review of) golang.org/cl/19847.

Change-Id: I03b76f36e5da69c31730380592dfa1c32570e17f
Reviewed-on: https://go-review.googlesource.com/19912
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #14497.

Change-Id: Ibdd55acf9e416873c64f8751c2f65f7ccdb1d500
Reviewed-on: https://go-review.googlesource.com/19914Reviewed-by: David Crawshaw <crawshaw@golang.org>

If the parsing of an operand completes but the parser thinks there
is more to read, return an "expected end of operand" error message
instead of "expected EOF." This also removes extra "asm: " prefixes
in error strings since "asm: " is already set as the global log
prefix.

Fixes #14071

Change-Id: I7d621c1aea529a0eca3bcba032359bd25b3e1080
Reviewed-on: https://go-review.googlesource.com/19731Reviewed-by: Rob Pike <r@golang.org>

Change-Id: I7fe4d0cdcc284d5319c130ee3c351f23489af273
Reviewed-on: https://go-review.googlesource.com/19902Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

See #13247.

Change-Id: I06636157028d98430eb29277c822270592907856
Reviewed-on: https://go-review.googlesource.com/19910Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Passes toolstash -cmp.

Change-Id: Ia497b51c74a9c760a873e1ed690e4408fd0fe596
Reviewed-on: https://go-review.googlesource.com/19844Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Only tests do this, provide them a hook to disable freeing
after flush.

Change-Id: I810c6c51414a93f476a18ba07b807e16092bf8cf
Reviewed-on: https://go-review.googlesource.com/19907Reviewed-by: Keith Randall <khr@golang.org>

name                        old time/op  new time/op  delta
EncodeValidASCII-4          74.1ns ± 1%  70.1ns ± 1%   -5.46%  (p=0.000 n=10+10)
EncodeValidJapaneseChars-4  61.3ns ± 0%  58.9ns ± 0%   -3.82%  (p=0.000 n=10+10)
EncodeRune-4                13.1ns ± 1%   9.8ns ± 0%  -25.24%   (p=0.000 n=10+9)

Fixes #6957

Change-Id: I9dde6d77420c34c6e2ef3e6213bb6be9b58a3074
Reviewed-on: https://go-review.googlesource.com/19891Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Not every Android contains the /system/framework directory, e.g. Brillo.
Test against other Android-only system files.

Fixes #14489.

Change-Id: I6d9ec1c4d4ceba3803798015e6917d59cf515de8
Reviewed-on: https://go-review.googlesource.com/19904Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Burcu Dogan <jbd@google.com>
Run-TryBot: Burcu Dogan <jbd@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Reuseable cache of Prog entries.

Improves compiler speed by ~10%.

Update #13646

Change-Id: I01bd8606540d989ea8b8ba5131d1275ba380d976
Reviewed-on: https://go-review.googlesource.com/19868Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Don't accumulate a massive list of Prog structs during
compilation and write them all out at the end of compilation.
Instead, convert them to code+relocs (or data+relocs) after each
function is compiled.

Track down a few other places that were keeping Progs alive
and nil them out so the Progs get GCd promptly.

Saves ~20% in peak memory usage for the compiler.  Surprisingly not much
help speed-wise (only because we end up doing more GCs.  With a
compensating GOGC=120, it does help a bit), but this provides a base for
more changes (e.g. reusing a cache of Progs).

Change-Id: I838e01017c228995a687a8110d0cd67bf8596407
Reviewed-on: https://go-review.googlesource.com/19867
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: Icd06d99c42b8299fd931c7da821e1f418684d913
Reviewed-on: https://go-review.googlesource.com/19829Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #14464

Change-Id: Iafc21641cca7d35b7a5631cfc94742ee8e7d5042
Reviewed-on: https://go-review.googlesource.com/19861Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

In CL 14408, the implementation of duffzero on amd64
was changed to replace the use of the MOVQ instructions
by MOVUPS.

However, it broke the build on plan9/amd64, since
Plan 9 doesn't allow floating point in note handler.

This change disables the use of duffzero on Plan 9.
We also take care to not use the MOVUPS instruction.

Fixes #14471.

Change-Id: I8277b485dfe65a68d7d8338e52a048c5d45069bf
Reviewed-on: https://go-review.googlesource.com/19890Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Reduces binary size of cmd/go by 1%.

For #6853.

Change-Id: I6f2992a4dd3699db1b532ab08683e82741b9c2e4
Reviewed-on: https://go-review.googlesource.com/19692Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>