• Austin Clements's avatar
    runtime: make sweep proportional to spans bytes allocated · fc9ca85f
    Austin Clements authored
    Proportional concurrent sweep is currently based on a ratio of spans
    to be swept per bytes of object allocation. However, proportional
    sweeping is performed during span allocation, not object allocation,
    in order to minimize contention and overhead. Since objects are
    allocated from spans after those spans are allocated, the system tends
    to operate in debt, which means when the next GC cycle starts, there
    is often sweep debt remaining, so GC has to finish the sweep, which
    delays the start of the cycle and delays enabling mutator assists.
    
    For example, it's quite likely that many Ps will simultaneously refill
    their span caches immediately after a GC cycle (because GC flushes the
    span caches), but at this point, there has been very little object
    allocation since the end of GC, so very little sweeping is done. The
    Ps then allocate objects from these cached spans, which drives up the
    bytes of object allocation, but since these allocations are coming
    from cached spans, nothing considers whether more sweeping has to
    happen. If the sweep ratio is high enough (which can happen if the
    next GC trigger is very close to the retained heap size), this can
    easily represent a sweep debt of thousands of pages.
    
    Fix this by making proportional sweep proportional to the number of
    bytes of spans allocated, rather than the number of bytes of objects
    allocated. Prior to allocating a span, both the small object path and
    the large object path ensure credit for allocating that span, so the
    system operates in the black, rather than in the red.
    
    Combined with the previous commit, this should eliminate all sweeping
    from GC start up. On the stress test in issue #11911, this reduces the
    time spent sweeping during GC (and delaying start up) by several
    orders of magnitude:
    
                    mean    99%ile     max
        pre fix      1 ms    11 ms   144 ms
        post fix   270 ns   735 ns   916 ns
    
    Updates #11911.
    
    Change-Id: I89223712883954c9d6ec2a7a51ecb97172097df3
    Reviewed-on: https://go-review.googlesource.com/13044Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
    Reviewed-by: 's avatarRuss Cox <rsc@golang.org>
    fc9ca85f
mcentral.go 5.39 KB