Commit 6002e01e authored by Austin Clements's avatar Austin Clements

runtime: allocate black during GC

Currently we allocate white for most of concurrent marking. This is
based on the classical argument that it produces less floating
garbage, since allocations during GC may not get linked into the heap
and allocating white lets us reclaim these. However, it's not clear
how often this actually happens, especially since our write barrier
shades any pointer as soon as it's installed in the heap regardless of
the color of the slot.

On the other hand, allocating black has several advantages that seem
to significantly outweigh this downside.

1) It naturally bounds the total scan work to the live heap size at
the start of a GC cycle. Allocating white does not, and thus depends
entirely on assists to prevent the heap from growing faster than it
can be scanned.

2) It reduces the total amount of scan work per GC cycle by the size
of newly allocated objects that are linked into the heap graph, since
objects allocated black never need to be scanned.

3) It reduces total write barrier work since more objects will already
be black when they are linked into the heap graph.

This gives a slight overall improvement in benchmarks.

name              old time/op  new time/op  delta
XBenchGarbage-12  2.24ms ± 0%  2.21ms ± 1%  -1.32%  (p=0.000 n=18+17)

name                      old time/op    new time/op    delta
BinaryTree17-12              2.60s ± 3%     2.53s ± 3%  -2.56%  (p=0.000 n=20+20)
Fannkuch11-12                2.08s ± 1%     2.08s ± 0%    ~     (p=0.452 n=19+19)
FmtFprintfEmpty-12          45.1ns ± 2%    45.3ns ± 2%    ~     (p=0.367 n=19+20)
FmtFprintfString-12          131ns ± 3%     129ns ± 0%  -1.60%  (p=0.000 n=20+16)
FmtFprintfInt-12             122ns ± 0%     121ns ± 2%  -0.86%  (p=0.000 n=16+19)
FmtFprintfIntInt-12          187ns ± 1%     186ns ± 1%    ~     (p=0.514 n=18+19)
FmtFprintfPrefixedInt-12     189ns ± 0%     188ns ± 1%  -0.54%  (p=0.000 n=16+18)
FmtFprintfFloat-12           256ns ± 0%     254ns ± 1%  -0.43%  (p=0.000 n=17+19)
FmtManyArgs-12               769ns ± 0%     763ns ± 0%  -0.72%  (p=0.000 n=18+18)
GobDecode-12                7.08ms ± 2%    7.00ms ± 1%  -1.22%  (p=0.000 n=20+20)
GobEncode-12                5.88ms ± 0%    5.88ms ± 1%    ~     (p=0.406 n=18+18)
Gzip-12                      214ms ± 0%     214ms ± 1%    ~     (p=0.103 n=17+18)
Gunzip-12                   37.6ms ± 0%    37.6ms ± 0%    ~     (p=0.563 n=17+17)
HTTPClientServer-12         77.2µs ± 3%    76.9µs ± 2%    ~     (p=0.606 n=20+20)
JSONEncode-12               15.1ms ± 1%    15.2ms ± 2%    ~     (p=0.138 n=19+19)
JSONDecode-12               53.3ms ± 1%    53.1ms ± 1%  -0.33%  (p=0.000 n=19+18)
Mandelbrot200-12            4.04ms ± 1%    4.04ms ± 1%    ~     (p=0.075 n=19+18)
GoParse-12                  3.30ms ± 1%    3.29ms ± 1%  -0.57%  (p=0.000 n=18+16)
RegexpMatchEasy0_32-12      69.5ns ± 1%    69.9ns ± 3%    ~     (p=0.822 n=18+20)
RegexpMatchEasy0_1K-12       237ns ± 1%     237ns ± 0%    ~     (p=0.398 n=19+18)
RegexpMatchEasy1_32-12      69.8ns ± 2%    69.5ns ± 1%    ~     (p=0.090 n=20+16)
RegexpMatchEasy1_1K-12       371ns ± 1%     372ns ± 1%    ~     (p=0.178 n=19+20)
RegexpMatchMedium_32-12      108ns ± 2%     108ns ± 3%    ~     (p=0.124 n=20+19)
RegexpMatchMedium_1K-12     33.9µs ± 2%    34.2µs ± 4%    ~     (p=0.309 n=20+19)
RegexpMatchHard_32-12       1.75µs ± 2%    1.77µs ± 4%  +1.28%  (p=0.018 n=19+18)
RegexpMatchHard_1K-12       52.7µs ± 1%    53.4µs ± 4%  +1.23%  (p=0.013 n=15+18)
Revcomp-12                   354ms ± 1%     359ms ± 4%  +1.27%  (p=0.043 n=20+20)
Template-12                 63.6ms ± 2%    63.7ms ± 2%    ~     (p=0.654 n=20+18)
TimeParse-12                 313ns ± 1%     316ns ± 2%  +0.80%  (p=0.014 n=17+20)
TimeFormat-12                332ns ± 0%     329ns ± 0%  -0.66%  (p=0.000 n=16+16)
[Geo mean]                  51.7µs         51.6µs       -0.09%

Change-Id: I2214a6a0e4f544699ea166073249a8efdf080dc0
Reviewed-on: https://go-review.googlesource.com/21323Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
parent 64a26b79
...@@ -690,11 +690,11 @@ func mallocgc(size uintptr, typ *_type, needzero bool) unsafe.Pointer { ...@@ -690,11 +690,11 @@ func mallocgc(size uintptr, typ *_type, needzero bool) unsafe.Pointer {
publicationBarrier() publicationBarrier()
} }
// GCmarkterminate allocates black // Allocate black during GC.
// All slots hold nil so no scanning is needed. // All slots hold nil so no scanning is needed.
// This may be racing with GC so do it atomically if there can be // This may be racing with GC so do it atomically if there can be
// a race marking the bit. // a race marking the bit.
if gcphase == _GCmarktermination || gcBlackenPromptly { if gcphase != _GCoff {
gcmarknewobject(uintptr(x), size, scanSize) gcmarknewobject(uintptr(x), size, scanSize)
} }
......
...@@ -24,6 +24,10 @@ ...@@ -24,6 +24,10 @@
// Hudson, R., and Moss, J.E.B. Copying Garbage Collection without stopping the world. // Hudson, R., and Moss, J.E.B. Copying Garbage Collection without stopping the world.
// Concurrency and Computation: Practice and Experience 15(3-5), 2003. // Concurrency and Computation: Practice and Experience 15(3-5), 2003.
// //
// TODO(austin): The rest of this comment is woefully out of date and
// needs to be rewritten. There is no distinct scan phase any more and
// we allocate black during GC.
//
// 0. Set phase = GCscan from GCoff. // 0. Set phase = GCscan from GCoff.
// 1. Wait for all P's to acknowledge phase change. // 1. Wait for all P's to acknowledge phase change.
// At this point all goroutines have passed through a GC safepoint and // At this point all goroutines have passed through a GC safepoint and
...@@ -244,7 +248,7 @@ var gcBlackenPromptly bool ...@@ -244,7 +248,7 @@ var gcBlackenPromptly bool
const ( const (
_GCoff = iota // GC not running; sweeping in background, write barrier disabled _GCoff = iota // GC not running; sweeping in background, write barrier disabled
_GCmark // GC marking roots and workbufs, write barrier ENABLED _GCmark // GC marking roots and workbufs: allocate black, write barrier ENABLED
_GCmarktermination // GC mark termination: allocate black, P's help GC, write barrier ENABLED _GCmarktermination // GC mark termination: allocate black, P's help GC, write barrier ENABLED
) )
...@@ -467,14 +471,18 @@ func (c *gcControllerState) startCycle() { ...@@ -467,14 +471,18 @@ func (c *gcControllerState) startCycle() {
// It should only be called when gcBlackenEnabled != 0 (because this // It should only be called when gcBlackenEnabled != 0 (because this
// is when assists are enabled and the necessary statistics are // is when assists are enabled and the necessary statistics are
// available). // available).
//
// TODO: Consider removing the periodic controller update altogether.
// Since we switched to allocating black, in theory we shouldn't have
// to change the assist ratio. However, this is still a useful hook
// that we've found many uses for when experimenting.
func (c *gcControllerState) revise() { func (c *gcControllerState) revise() {
// Compute the expected scan work remaining. // Compute the expected scan work remaining.
// //
// Note that the scannable heap size is likely to increase // Note that we currently count allocations during GC as both
// during the GC cycle. This is why it's important to revise // scannable heap (heap_scan) and scan work completed
// the assist ratio throughout the cycle: if the scannable // (scanWork), so this difference won't be changed by
// heap size increases, the assist ratio based on the initial // allocations during GC.
// scannable heap size may target too little scan work.
// //
// This particular estimate is a strict upper bound on the // This particular estimate is a strict upper bound on the
// possible remaining scan work for the current heap. // possible remaining scan work for the current heap.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment