runtime: enable preemption of mark termination goroutine

A mark worker goroutine may attempt to preempt the mark termination
goroutine to scan its stack while the mark termination goroutine is
trying to preempt that worker to flush its work buffer, in rare
cases.

This change makes it so that, like a worker goroutine, the mark
termination goroutine stack is preemptible while it is on the
system stack, attempting to preempt others.

Fixes #28695.

Change-Id: I23bbb191f4fdad293e8a70befd51c9175f8a1171
Reviewed-on: https://go-review.googlesource.com/c/153077Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

src/runtime/mgc.go

@@ -1417,6 +1417,12 @@ top:
 	// Flush all local buffers and collect flushedWork flags.
 	gcMarkDoneFlushed = 0
 	systemstack(func() {
+		gp := getg().m.curg
+		// Mark the user stack as preemptible so that it may be scanned.
+		// Otherwise, our attempt to force all P's to a safepoint could
+		// result in a deadlock as we attempt to preempt a worker that's
+		// trying to preempt us (e.g. for a stack scan).
+		casgstatus(gp, _Grunning, _Gwaiting)
 		forEachP(func(_p_ *p) {
 			// Flush the write barrier buffer, since this may add
 			// work to the gcWork.
@@ -1449,6 +1455,7 @@ top:
 				_p_.gcw.pauseGen = gcWorkPauseGen
 			}
 		})
+		casgstatus(gp, _Gwaiting, _Grunning)
 	})
 
 	if gcMarkDoneFlushed != 0 {