Currently the GODEBUG=gctrace=1 trace line includes "@n.nnns" to
indicate the time that the GC cycle ended relative to the time the
program started. This was meant to be consistent with the utilization
as of the end of the cycle, which is printed next on the trace line,
but it winds up just being confusing and unexpected.

Change the trace line to include the time that the GC cycle started
relative to the time the program started.

Change-Id: I7d64580cd696eb17540716d3e8a74a9d6ae50650
Reviewed-on: https://go-review.googlesource.com/10634Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Due to the requirements of parsing template definitions that mention
other templates that are not yet defined, a Template can be in two states:
defined and undefined. Thus, although one calls New, the resulting
template has no definition even though it exists as a data structure.

Thus, for example, will return nil for a template that is named but not
yet defined.

Fixes #10910
Fixes #10926

Clarify the documentation a little to explain this,
Also tidy up the code a little and remove a spurious call to init.

Change-Id: I22cc083291500bca424e83dc12807e0de7b00b7a
Reviewed-on: https://go-review.googlesource.com/10641Reviewed-by: Andrew Gerrand <adg@golang.org>

This commit implements stack barriers to minimize the amount of
stack re-scanning that must be done during mark termination.

Currently the GC scans stacks of active goroutines twice during every
GC cycle: once at the beginning during root discovery and once at the
end during mark termination. The second scan happens while the world
is stopped and guarantees that we've seen all of the roots (since
there are no write barriers on writes to local stack
variables). However, this means pause time is proportional to stack
size. In particularly recursive programs, this can drive pause time up
past our 10ms goal (e.g., it takes about 150ms to scan a 50MB heap).

Re-scanning the entire stack is rarely necessary, especially for large
stacks, because usually most of the frames on the stack were not
active between the first and second scans and hence any changes to
these frames (via non-escaping pointers passed down the stack) were
tracked by write barriers.

To efficiently track how far a stack has been unwound since the first
scan (and, hence, how much needs to be re-scanned), this commit
introduces stack barriers. During the first scan, at exponentially
spaced points in each stack, the scan overwrites return PCs with the
PC of the stack barrier function. When "returned" to, the stack
barrier function records how far the stack has unwound and jumps to
the original return PC for that point in the stack. Then the second
scan only needs to proceed as far as the lowest barrier that hasn't
been hit.

For deeply recursive programs, this substantially reduces mark
termination time (and hence pause time). For the goscheme example
linked in issue #10898, prior to this change, mark termination times
were typically between 100 and 500ms; with this change, mark
termination times are typically between 10 and 20ms. As a result of
the reduced stack scanning work, this reduces overall execution time
of the goscheme example by 20%.

Fixes #10898.

The effect of this on programs that are not deeply recursive is
minimal:

name                   old time/op    new time/op    delta
BinaryTree17              3.16s ± 2%     3.26s ± 1%  +3.31%  (p=0.000 n=19+19)
Fannkuch11                2.42s ± 1%     2.48s ± 1%  +2.24%  (p=0.000 n=17+19)
FmtFprintfEmpty          50.0ns ± 3%    49.8ns ± 1%    ~     (p=0.534 n=20+19)
FmtFprintfString          173ns ± 0%     175ns ± 0%  +1.49%  (p=0.000 n=16+19)
FmtFprintfInt             170ns ± 1%     175ns ± 1%  +2.97%  (p=0.000 n=20+19)
FmtFprintfIntInt          288ns ± 0%     295ns ± 0%  +2.73%  (p=0.000 n=16+19)
FmtFprintfPrefixedInt     242ns ± 1%     252ns ± 1%  +4.13%  (p=0.000 n=18+18)
FmtFprintfFloat           324ns ± 0%     323ns ± 0%  -0.36%  (p=0.000 n=20+19)
FmtManyArgs              1.14µs ± 0%    1.12µs ± 1%  -1.01%  (p=0.000 n=18+19)
GobDecode                8.88ms ± 1%    8.87ms ± 0%    ~     (p=0.480 n=19+18)
GobEncode                6.80ms ± 1%    6.85ms ± 0%  +0.82%  (p=0.000 n=20+18)
Gzip                      363ms ± 1%     363ms ± 1%    ~     (p=0.077 n=18+20)
Gunzip                   90.6ms ± 0%    90.0ms ± 1%  -0.71%  (p=0.000 n=17+18)
HTTPClientServer         51.5µs ± 1%    50.8µs ± 1%  -1.32%  (p=0.000 n=18+18)
JSONEncode               17.0ms ± 0%    17.1ms ± 0%  +0.40%  (p=0.000 n=18+17)
JSONDecode               61.8ms ± 0%    63.8ms ± 1%  +3.11%  (p=0.000 n=18+17)
Mandelbrot200            3.84ms ± 0%    3.84ms ± 1%    ~     (p=0.583 n=19+19)
GoParse                  3.71ms ± 1%    3.72ms ± 1%    ~     (p=0.159 n=18+19)
RegexpMatchEasy0_32       100ns ± 0%     100ns ± 1%  -0.19%  (p=0.033 n=17+19)
RegexpMatchEasy0_1K       342ns ± 1%     331ns ± 0%  -3.41%  (p=0.000 n=19+19)
RegexpMatchEasy1_32      82.5ns ± 0%    81.7ns ± 0%  -0.98%  (p=0.000 n=18+18)
RegexpMatchEasy1_1K       505ns ± 0%     494ns ± 1%  -2.16%  (p=0.000 n=18+18)
RegexpMatchMedium_32      137ns ± 1%     137ns ± 1%  -0.24%  (p=0.048 n=20+18)
RegexpMatchMedium_1K     41.6µs ± 0%    41.3µs ± 1%  -0.57%  (p=0.004 n=18+20)
RegexpMatchHard_32       2.11µs ± 0%    2.11µs ± 1%  +0.20%  (p=0.037 n=17+19)
RegexpMatchHard_1K       63.9µs ± 2%    63.3µs ± 0%  -0.99%  (p=0.000 n=20+17)
Revcomp                   560ms ± 1%     522ms ± 0%  -6.87%  (p=0.000 n=18+16)
Template                 75.0ms ± 0%    75.1ms ± 1%  +0.18%  (p=0.013 n=18+19)
TimeParse                 358ns ± 1%     364ns ± 0%  +1.74%  (p=0.000 n=20+15)
TimeFormat                360ns ± 0%     372ns ± 0%  +3.55%  (p=0.000 n=20+18)

Change-Id: If8a9bfae6c128d15a4f405e02bcfa50129df82a2
Reviewed-on: https://go-review.googlesource.com/10314Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently there's a race between stopg scanning another G's stack and
the G reaching a preemption point and scanning its own stack. When
this race occurs, the G's stack is scanned twice. Currently this is
okay, so this race is benign.

However, we will shortly be adding stack barriers during the first
stack scan, so scanning will no longer be idempotent. To prepare for
this, this change ensures that each stack is scanned only once during
each GC phase by checking the flag that indicates that the stack has
been scanned in this phase before scanning the stack.

Change-Id: Id9f4d5e2e5b839bc3f200ec1723a4a12dd677ab4
Reviewed-on: https://go-review.googlesource.com/10458Reviewed-by: Rick Hudson <rlh@golang.org>

The stack barrier code will need a bookkeeping structure to keep track
of the overwritten return PCs. This commit introduces and allocates
this structure, but does not yet use the structure.

We don't want to allocate space for this structure during garbage
collection, so this commit allocates it along with the allocation of
the corresponding stack. However, we can't do a regular allocation in
newstack because mallocgc may itself grow the stack (which would lead
to a recursive allocation). Hence, this commit makes the bookkeeping
structure part of the stack allocation itself by stealing the
necessary space from the top of the stack allocation. Since the size
of this bookkeeping structure is logarithmic in the size of the stack,
this has minimal impact on stack behavior.

Change-Id: Ia14408be06aafa9ca4867f4e70bddb3fe0e96665
Reviewed-on: https://go-review.googlesource.com/10313Reviewed-by: Russ Cox <rsc@golang.org>

Currently the runtime assumes that the allocation for the stack is
exactly [stack.lo, stack.hi). We're about to steal a small part of
this allocation for per-stack GC metadata. To prepare for this, this
commit adds a field to the G for the allocated size of the stack.
With this change, stack.lo and stack.hi continue to act as the true
bounds on the stack, but are no longer also used as the bounds on the
stack allocation.

(I also tried this the other way around, where stack.lo and stack.hi
remained the allocation bounds and I introduced a new top of stack.
However, there are far more places that assume stack.hi is the true
top of the stack than there are places that assume it's the top of the
allocation.)

Change-Id: Ifa9d956753be53d286d09cbc73d47fb34a18c0c6
Reviewed-on: https://go-review.googlesource.com/10312Reviewed-by: Russ Cox <rsc@golang.org>

Currently signalstack takes a lower limit and a length and all calls
hard-code the passed length. Change the API to take a *stack and
compute the lower limit and length from the passed stack.

This will make it easier for the runtime to steal some space from the
top of the stack since it eliminates the hard-coded stack sizes.

Change-Id: I7d2a9f45894b221f4e521628c2165530bbc57d53
Reviewed-on: https://go-review.googlesource.com/10311Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Currently we truncate gctrace clock and CPU times to millisecond
precision. As a result, many phases are typically printed as 0, which
is fine for user consumption, but makes gathering statistics and
reports over GC traces difficult.

In 1.4, the gctrace line printed times in microseconds. This was
better for statistics, but not as easy for users to read or interpret,
and it generally made the trace lines longer.

This change strikes a balance between these extremes by printing
milliseconds, but including the decimal part to two significant
figures down to microsecond precision. This remains easy to read and
interpret, but includes more precision when it's useful.

For example, where the code currently prints,

gc #29 @1.629s 0%: 0+2+0+12+0 ms clock, 0+2+0+0/12/0+0 ms cpu, 4->4->2 MB, 4 MB goal, 1 P

this prints,

gc #29 @1.629s 0%: 0.005+2.1+0+12+0.29 ms clock, 0.005+2.1+0+0/12/0+0.29 ms cpu, 4->4->2 MB, 4 MB goal, 1 P

Fixes #10970.

Change-Id: I249624779433927cd8b0947b986df9060c289075
Reviewed-on: https://go-review.googlesource.com/10554Reviewed-by: Russ Cox <rsc@golang.org>

The Error function is a potential XSS vector if a user can control the
error message.

For example, an http.FileServer when given a request for this path
	/<script>alert("xss!")</script>
may return a response with a body like this
	open <script>alert("xss!")</script>: no such file or directory
Browsers that sniff the content may interpret this as HTML and execute
the script. The nosniff header added by this CL should help, but we
should also try santizing the output entirely.

Change-Id: I447f701531329a2fc8ffee2df2f8fa69d546f893
Reviewed-on: https://go-review.googlesource.com/10640Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Allow room for the initial minus sign of negative integers when
computing widths.

Fixes #10945.

Change-Id: I04d80203aaff64611992725d613ec13ed2ae721f
Reviewed-on: https://go-review.googlesource.com/10393Reviewed-by: Rob Pike <r@golang.org>

This change fixes incorrect parsing of literal IP addresses in local
database when the addresses contain IPv6 zone identifiers, are in
dotted-decimal notation or in colon-hexadecimal notation with leading
zeros.

https://golang.org/cl/5851 already fixed the code path using getaddrinfo
via cgo. This change fixes the remaining non-cgo code path.

Fixes #8243.
Fixes #8996.

Change-Id: I48443611cbabed0d69667cc73911ba3de396fd44
Reviewed-on: https://go-review.googlesource.com/10306Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #11014.

Change-Id: I9a18dacd10564d3eaa1fea4d77f1a48e08e79f53
Reviewed-on: https://go-review.googlesource.com/10563Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I22fdba8ecaecf4e9201b845e65d982cac09f254a
Reviewed-on: https://go-review.googlesource.com/10499Reviewed-by: Alan Donovan <adonovan@google.com>

Fixes an error where the compiler did not spot that the shadowed err
value was set again after being read. That second assignment was lost
as the value was redeclared in an inner scope.

Spotted by Gordon Klass, https://groups.google.com/forum/#!topic/golang-nuts/MdDLbvOjb4o

Change-Id: I28f2da6f98c52afcbb45e17d2b4f36c586598f98
Reviewed-on: https://go-review.googlesource.com/10600Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>

Fix missing error check.

Spotted by Gordon Klass, https://groups.google.com/forum/#!topic/golang-nuts/MdDLbvOjb4o

Change-Id: I453a0cf032e0077d2622d5b85030310d159b9c4b
Reviewed-on: https://go-review.googlesource.com/10606Reviewed-by: Nigel Tao <nigeltao@golang.org>

Fix missing error check in test.

Spotted by Gordon Klass, https://groups.google.com/forum/#!topic/golang-nuts/MdDLbvOjb4o

Change-Id: I22f1a438cbb60a2fe1740fc2d43fbf8aa008b6de
Reviewed-on: https://go-review.googlesource.com/10605Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The names of examples were wrong so they were not shown in
the documentation.

Change-Id: Ib1b985b44d2e056c38c008a591cb441e422c4717
Reviewed-on: https://go-review.googlesource.com/10404
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Executing a template involving variadic functions featuring
a []interface{} slice (such as printf) could result in a
panic in reflect.Value.Call, due to incorrect type checking.

The following expressions failed (with a panic):
{{true|printf}}
{{1|printf}}
{{1.1|printf}}
{{'x'|printf}}
{{1+2i|printf}}

Implemented proper type checks for the fixed parameters of the
variadic functions.

Fixes #10946

Change-Id: Ia75333f651f73b3d2e024cb0c47cc30d90cb6852
Reviewed-on: https://go-review.googlesource.com/10403Reviewed-by: Rob Pike <r@golang.org>

The current escape code panics when an action involves chain nodes.
Such nodes can be seen in the following situation:

{{ . | AAA.B }} - AAA being a registered function

The above expression is actually valid, because AAA could return a
map containing a B key. The tests in text/template explicitly
demonstrate this case.

Fix allIdents to cover also chain nodes.

While I was investigating this issue, I realized that the tests
introduced in similar CL 9621 were incorrect. Parse errors were
caught as expected, but for the wrong reason. Fixed them as well.
No changes in text/template code itself.

Fixes #10801

Change-Id: Ic9fe43b63669298ca52c3f499e2725dd2bb818a8
Reviewed-on: https://go-review.googlesource.com/10340Reviewed-by: Rob Pike <r@golang.org>

In 1.6, go doc is more likely to be available.

Change-Id: I970ad1d3317b35273f5c8d830f75713d3570c473
Reviewed-on: https://go-review.googlesource.com/10518Reviewed-by: Andrew Gerrand <adg@golang.org>

When go doc is invoked with a single package name argument (e.g. go doc pkgname)
it needs to find the directory of the requested package sources in GOPATH.

GOPATH might contain directories with the same name as the requested package
that do no contain any *.go files. This change makes "go doc" ignore such
directories when looking for possible package directories.

This fixes #10882

Change-Id: Ib3d4ea69a25801c34cbe7b044de9870ba12f9aa8
Reviewed-on: https://go-review.googlesource.com/10190Reviewed-by: Rob Pike <r@golang.org>

t.init() should be called at the time of template creation
i.e, template.New() and t.New() instead of later in the process.

- Removed calls of t.init() from t.Parse(), t.Execute(), t.Funcs()
- Also got rid of t.common != nil checks as it should never be nil

Fixes #10879

Change-Id: I1b7ac812f02c841ae80037babce7e2b0a2df13e8
Reviewed-on: https://go-review.googlesource.com/10240Reviewed-by: Rob Pike <r@golang.org>

This was a simple oversight: the algorithm to handle recursive types
needed to be applied to the ignore-item case as well.

Fixes #10415.

Change-Id: I39ef31cad680ab8334e141f60d2f8707896785d1
Reviewed-on: https://go-review.googlesource.com/8942Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

runtime.GC() is intentionally very weakly specified. However, it is so
weakly specified that it's difficult to know that it's being used
correctly for its one intended use case: to ensure garbage collection
has run in a test that is garbage-sensitive. In particular, it is
unclear whether it is synchronous or asynchronous. In the old STW
collector this was essentially self-evident; short of queuing up a
garbage collection to run later, it had to be synchronous. However,
with the concurrent collector, there's evidence that people are
inferring that it may be asynchronous (e.g., issue #10986), as this is
both unclear in the documentation and possible in the implementation.

In fact, runtime.GC() runs a fully synchronous STW collection. We
probably don't want to commit to this exact behavior. But we can
commit to the essential property that tests rely on: that runtime.GC()
does not return until the GC has finished.

Change-Id: Ifc3045a505e1898ecdbe32c1f7e80e2e9ffacb5b
Reviewed-on: https://go-review.googlesource.com/10488Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Change-Id: Ic00882735d95d61f7c9d9f28d34cb4acce6a5546
Reviewed-on: https://go-review.googlesource.com/10556Reviewed-by: Minux Ma <minux@golang.org>

As noted in bug #10980, an empty PEM block is encoded as
-----BEGIN foo-----
-----END foo-----

However, Decode failed to process this.

RFC 1421 doesn't answer what the encoding of the empty block should be
because PEM messages always contain at least one header. However, PEM
these days is just the encoding format – nobody uses the rest of PEM any
longer.

Having the empty block not contain a newline seems most correct because
https://tools.ietf.org/html/rfc1421#section-9 clearly says that the
optional “pemtext” carries the leading new-line with it. So if omitted,
the new-line should be omitted too.

None the less, this changes makes encoding/pem permissive, accepting any
number of blank lines in an empty PEM block.

Fixes #10980

Change-Id: If36bdfbf991ee281eccd50b56ddc95f24c6debb2
Reviewed-on: https://go-review.googlesource.com/10516Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>

I think it's worth mentioning.
But the final decision is up to you.

Change-Id: I3959132600ecc554988524ede73a7f6e8eac8353
Reviewed-on: https://go-review.googlesource.com/10551Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

They're each architecture-specific.
Let them share.

Reduces Prog size to 288, which is the
next smaller malloc class.

Reduces inuse_space while compiling the
rotate tests by ~3.2%.

Change-Id: Ica8ec90e466c97b569745fffff0e5acd364e55fa
Reviewed-on: https://go-review.googlesource.com/10514Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

That which cannot happen has not happened.

No immediate changes to Addr or Prog size.

Change-Id: I4cb9315f2c9f5f92eda340bfc4abb46395fa467f
Reviewed-on: https://go-review.googlesource.com/10513Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Printed and Width were unused.

Despite only removing two bytes, due to alignment, 8 bytes are saved
on 64-bit:

Before: unsafe.Sizeof(obj.Prog{}) == 304
 After: unsafe.Sizeof(obj.Prog{}) == 296

The next size class below 320 (304=>19(320)) is 288. Still 8 bytes
away from that.

Change-Id: I8d1632dd40d387f7036c03c65ea4d64e9b6218c3
Reviewed-on: https://go-review.googlesource.com/10511
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

Change-Id: I55a7f455ebbd6b1bd6912aae82c0fcff6f43387c
Reviewed-on: https://go-review.googlesource.com/10512
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Missed in previous CL, causing build failures.

Change-Id: I60aae5a3139aa009cb643d871d91b9d4c47dcbb8
Reviewed-on: https://go-review.googlesource.com/10538Reviewed-by: Russ Cox <rsc@golang.org>

- (*Float).Scan conflicted with fmt.Scanner.Scan; it was also only used
  internally. Removed it, as well as the companion ScanFloat function.

- (*Float).Parse (and thus ParseFloat) can now also parse infinities.
  As a result, more code could be simplified.

- Fixed a bug in rounding (round may implicitly be called for infinite
  values). Found via existing test cases, after simplifying some code.

- Added more test cases.

Fixes issue #10938.

Change-Id: I1df97821654f034965ba8b82b272e52e6dc427f1
Reviewed-on: https://go-review.googlesource.com/10498Reviewed-by: Alan Donovan <adonovan@google.com>

This paves the way for a fmt-compatible (*Float).Format method.
A better name then Text is still desirable (suggestions welcome).

This is partly fixing issue #10938.

Change-Id: I59c20a8cee11f5dba059fe0f38b414fe75f2ab13
Reviewed-on: https://go-review.googlesource.com/10493Reviewed-by: Alan Donovan <adonovan@google.com>

It is almost never set and Addr is large, so having the full struct
in the Prog wastes memory most of the time.

Before (on a 64-bit system):

$ sizeof -p cmd/internal/obj Addr Prog
Addr 80
Prog 376
$

After:

$ sizeof -p cmd/internal/obj Addr Prog
Addr 80
Prog 304
$

Change-Id: I491f201241f87543964a7d0f48b85830759be9d0
Reviewed-on: https://go-review.googlesource.com/10457Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

Change-Id: Iddcd0cfb8f2c2f1c4ad7a94b50a9f65b543862c4
Reviewed-on: https://go-review.googlesource.com/10473Reviewed-by: Minux Ma <minux@golang.org>

Fixes #10968.

Change-Id: I027bc571a71629ac49c2a0ff101b2950af6e7531
Reviewed-on: https://go-review.googlesource.com/10482Reviewed-by: David Symonds <dsymonds@golang.org>
Run-TryBot: David Symonds <dsymonds@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I7aeb9fef3739c17c03fdaadbe00cd945ec9c0d72
Reviewed-on: https://go-review.googlesource.com/10492
Run-TryBot: Andrew Gerrand <adg@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #10979.

Change-Id: Iac25645ba8181a56a75ddfcd29ff6d64c15c4f57
Reviewed-on: https://go-review.googlesource.com/10466Reviewed-by: Alan Donovan <adonovan@google.com>

Memory usage has been reduced.
The tests are still slow,
but that is issue #10571.

/usr/bin/time shows significant variation
in the peak memory usage compiling with tip.
This is unsurprising, given GC.

Using Go 1.4.2, memory is stable at 410mb.
Using tip at d2ee09298,
memory ranges from 470mb (+15%) to 534mb (+30%),
with a mean of 504mb (+23%), with n=50.

Fixes #9933.

Change-Id: Id31f3ae086ec324abf70e8f1a8044c4a0c27e274
Reviewed-on: https://go-review.googlesource.com/10211Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>