Commit 3e214e56 authored by Austin Clements's avatar Austin Clements

runtime: simplify bulkBarrierPreWrite

Currently, bulkBarrierPreWrite uses inheap to decide whether the
destination is in the heap or whether to check for stack or global
data. However, this isn't the best question to ask.

Instead, get the span directly and query its state. This lets us
directly determine whether this might be a global, or is stack memory,
or is heap memory.

At this point, inheap is no longer used in the hot path, so drop it
from the must-be-inlined list and substitute spanOf.

This will help in a circuitous way with #23862, since fixing that is
going to push inheap very slightly over the inline-able threshold on a
few platforms.

Change-Id: I5360fc1181183598502409f12979899e1e4d45f7
Reviewed-on: https://go-review.googlesource.com/95495
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
parent 3e1ac1b0
......@@ -78,10 +78,10 @@ func TestIntendedInlining(t *testing.T) {
"heapBits.morePointers",
"heapBits.next",
"heapBitsForAddr",
"inheap",
"markBits.isMarked",
"muintptr.ptr",
"puintptr.ptr",
"spanOf",
"spanOfUnchecked",
"(*gcWork).putFast",
"(*gcWork).tryGetFast",
......
......@@ -574,13 +574,7 @@ func bulkBarrierPreWrite(dst, src, size uintptr) {
if !writeBarrier.needed {
return
}
if !inheap(dst) {
gp := getg().m.curg
if gp != nil && gp.stack.lo <= dst && dst < gp.stack.hi {
// Destination is our own stack. No need for barriers.
return
}
if s := spanOf(dst); s == nil {
// If dst is a global, use the data or BSS bitmaps to
// execute write barriers.
for _, datap := range activeModules() {
......@@ -596,6 +590,14 @@ func bulkBarrierPreWrite(dst, src, size uintptr) {
}
}
return
} else if s.state != _MSpanInUse || dst < s.base() || s.limit <= dst {
// dst was heap memory at some point, but isn't now.
// It can't be a global. It must be either our stack,
// or in the case of direct channel sends, it could be
// another stack. Either way, no need for barriers.
// This will also catch if dst is in a freed span,
// though that should never have.
return
}
buf := &getg().m.p.ptr().wbBuf
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment