runtime: document rules about unmanaged memory

Updates #17503.

Change-Id: I109d8742358ae983fdff3f3dbb7136973e81f4c3
Reviewed-on: https://go-review.googlesource.com/31452Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

src/runtime/HACKING.md

@@ -2,6 +2,59 @@ This is a very incomplete and probably out-of-date guide to
 programming in the Go runtime and how it differs from writing normal
 Go.
 
+Unmanaged memory
+================
+
+In general, the runtime tries to use regular heap allocation. However,
+in some cases the runtime must allocate objects outside of the garbage
+collected heap, in *unmanaged memory*. This is necessary if the
+objects are part of the memory manager itself or if they must be
+allocated in situations where the caller may not have a P.
+
+There are three mechanisms for allocating unmanaged memory:
+
+* sysAlloc obtains memory directly from the OS. This comes in whole
+  multiples of the system page size, but it can be freed with sysFree.
+
+* persistentalloc combines multiple smaller allocations into a single
+  sysAlloc to avoid fragmentation. However, there is no way to free
+  persistentalloced objects (hence the name).
+
+* fixalloc is a SLAB-style allocator that allocates objects of a fixed
+  size. fixalloced objects can be freed, but this memory can only be
+  reused by the same fixalloc pool, so it can only be reused for
+  objects of the same type.
+
+In general, types that are allocated using any of these should be
+marked `//go:notinheap` (see below).
+
+Objects that are allocated in unmanaged memory **must not** contain
+heap pointers unless the following rules are also obeyed:
+
+1. Any pointers from unmanaged memory to the heap must be added as
+   explicit garbage collection roots in `runtime.markroot`.
+
+2. If the memory is reused, the heap pointers must be zero-initialized
+   before they become visible as GC roots. Otherwise, the GC may
+   observe stale heap pointers. See "Zero-initialization versus
+   zeroing".
+
+Zero-initialization versus zeroing
+==================================
+
+There are two types of zeroing in the runtime, depending on whether
+the memory is already initialized to a type-safe state.
+
+If memory is not in a type-safe state, meaning it potentially contains
+"garbage" because it was just allocated and it is being initialized
+for first use, then it must be *zero-initialized* using
+`memclrNoHeapPointers` or non-pointer writes. This does not perform
+write barriers.
+
+If memory is already in a type-safe state and is simply being set to
+the zero value, this must be done using regular writes, `typedmemclr`,
+or `memclrHasPointers`. This performs write barriers.
+
 Runtime-only compiler directives
 ================================