R=ken2
CC=golang-dev
https://golang.org/cl/7399050

This is a simple refactoring of main.go that will enable the type checker
to be used during vetting.
The change has an unimportant effect on the arguments: it now assumes
that all files named explicitly on the command line belong to the same
package. When run by the go command, this was true already.

Also restore a missing parenthesis from an error message.

R=golang-dev, gri, bradfitz
CC=golang-dev
https://golang.org/cl/7393052

For now, all the callbacks from C use top-level Go functions,
so they use the equivalent C function pointer, and will continue
to do so. But perhaps some day this will be useful for calling
a Go func value (at least if the type is already known).

More importantly, the Windows callback code needs to be able
to use cgocallback_gofunc to call a Go func value.
Should fix the Windows build.

R=ken2
CC=golang-dev
https://golang.org/cl/7388049

R=adonovan
CC=golang-dev
https://golang.org/cl/7386052

Step 4 of http://golang.org/s/go11func.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7393049

Step 3 of http://golang.org/s/go11func.

Fixes #3736.
Fixes #3738.
Fixes #4081.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7393050

We wrap the final '...' argument's type in types.Slice.
Added tests.

Also:
- Function.writeSignature: suppress slice '[]' when printing
  variadic arg '...'.
- Eliminate Package.ImportPath field; redundant
  w.r.t. Package.Types.Path.
- Use "TODO: (opt|fix)" notation more widely.
- Eliminate many redundant/stale TODOs.

R=gri
CC=golang-dev
https://golang.org/cl/7378057

Change ARM context register to R7, to get out of the way
of the register allocator during the compilation of the
prologue statements (it wants to use R0 as a temporary).

Step 2 of http://golang.org/s/go11func.

R=ken2
CC=golang-dev
https://golang.org/cl/7369048

The new src argument is ignored during linking
(that is, CALL r1, r2 is identical to CALL r2 for linking),
but it serves as a hint to the 5g/6g/8g optimizer
that the src register is live on entry to the called
function and must be preserved.

It is possible to avoid exposing this fact to the rest of
the toolchain, keeping it entirely within 5g/6g/8g,
but I think it will help to be able to look in object files
and assembly listings and linker -a / -W output to
see CALL instructions are "Go func value" calls and
which are "C function pointer" calls.

R=ken2
CC=golang-dev
https://golang.org/cl/7364045

This makes Go's quoted-printable decoder more like other
popular ones, allowing through a bare \r or \n, and also
passes through \r\n which looked like a real bug before.

Fixes #4771

R=minux.ma
CC=golang-dev
https://golang.org/cl/7300092

R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/7382049

By appending to the wrong (always empty) list, only the last
anonymous field was being considered for promotion.

Also:
- eliminated "function-local NamedTypes" TODO; nothing to do.
- fixed Function.DumpTo: printing of anon receivers was "( T)",
  now "(T)"; extracted writeSignature into own function.
- eliminated blockNames function;
  thanks to BasicBlock.String, "%s" of []*BasicBlock is fine.
- extracted buildReferrers into own function.

exp/ssa can now build its own transitive closure.

R=gri
CC=golang-dev
https://golang.org/cl/7384054

R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/7395051

runtime: add context argument to gogocall

Too many other things use AX, and at least one
(stack zeroing) cannot be moved onto a different
register. Use the less special DX instead.

Preparation for step 2 of http://golang.org/s/go11func.
Nothing interesting here, just split out so that we can
see it's correct before moving on.

R=ken2
CC=golang-dev
https://golang.org/cl/7395050

- append: nothing to do (nonsemantic change).
- delete: now performs correct conversion (+ test).
- emitCompare: nothing to do.
- emitArith (shifts): nothing to do (+ test).
- "banish untyped types": give up on that.
- real, imag: now do correct conversions.
- added comment to interp.go re zero-size values.

R=gri
CC=golang-dev
https://golang.org/cl/7391046

This significantly speeds up the go tool on
slow file systems (or those with cold caches).

The following numbers were obtained using
an encrypted ext4 file system running on
Linux 3.7.9.

# Before
$ sudo sysctl -w 'vm.drop_caches=3'
$ time go list code.google.com/p/go.net/... | wc -l
9

real	0m16.921s
user	0m0.637s
sys	0m0.317s

# After
$ sudo sysctl -w 'vm.drop_caches=3'
$ time go list code.google.com/p/go.net/... | wc -l
9

real	0m8.175s
user	0m0.220s
sys	0m0.177s

R=rsc, r
CC=golang-dev
https://golang.org/cl/7369044

R=adonovan, minux.ma, alex.brainman, akumar, rminnich
CC=golang-dev, lucio.dere
https://golang.org/cl/7300078

Fixes #4812.

R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/7396051

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7393048

Mention Scanner in docs for ReadLine etc.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7375045

R=jsing, mikioh.mikioh, bradfitz
CC=golang-dev
https://golang.org/cl/7390048

Fixes #4860

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7375047

Revision 5e7fd762f356 has changed the output formatting in a way that
is no longer in line with the format described by the revision
ff0ade0b937b which has introduced this functionality.
When decorating the first line, instead of indenting the whole line,
the current implementation adds indentation right after the "decorate"
part and  before the "log" message.
The fix addresses this issue.

R=golang-dev, iant, minux.ma, r, rsc, remyoudompheng
CC=golang-dev
https://golang.org/cl/7304094

Step 1 of http://golang.org/s/go11func.

R=golang-dev, r, daniel.morsing, remyoudompheng
CC=golang-dev
https://golang.org/cl/7393045

Fixes #4867

R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/7382045

Previously, the func structure contained an inaccurate value for
the args member and a 0 value for the locals member.

This change populates the func structure with args and locals
values computed by the compiler.  The number of args was
already available in the ATEXT instruction.  The number of
locals is now passed through in the new ALOCALS instruction.

This change also switches the unit of args and locals to be
bytes, just like the frame member, instead of 32-bit words.

R=golang-dev, bradfitz, cshapiro, dave, rsc
CC=golang-dev
https://golang.org/cl/7399045

Update #4860

R=adg, rsc, campoy
CC=golang-dev
https://golang.org/cl/7369045

R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/7379045

Update norm and local/collate as well.

R=mpvl
CC=golang-dev
https://golang.org/cl/7395045

Return nice errors and don't panic.

Fixes #4859

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7383046

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7390047

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7397049

lvd changed the old %N to %+hN and these
never got updated.

R=ken2
CC=golang-dev
https://golang.org/cl/7391045

The interpreter's os.Exit now triggers a special panic rather
than kill the test process.  (It's semantically dubious, since
it will run deferred routines.)  Interpret now returns its
exit code rather than calling os.Exit.

Also:
- disabled parts of a few $GOROOT/tests via os.Getenv("GOSSAINTERP").
- remove unnecessary 'slots' param to external functions; they
  are never closures.

Most of the tests are disabled until go/types supports shifts.
They can be reenabled if you patch this workaround:
https://golang.org/cl/7312068

R=iant, bradfitz
CC=golang-dev, gri
https://golang.org/cl/7313062

Assume people who were going to update to Go 1 have done so.
Those with pre-Go 1 trees remaining will need to update first
to Go 1.0 (using its 'go fix') and then to Go 1.1.

Cuts the cmd/fix test time by 99% (3 seconds to 0.03 seconds).

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7402046

By avoiding the need for self-loops following calls to panic,
we reduce the number of basic blocks considerably.

R=gri
CC=golang-dev, iant
https://golang.org/cl/7403043

Current results on linux/amd64:
BenchmarkInterfaces                      20000             80902 ns/op
BenchmarkInterfaceByIndex                50000             71591 ns/op
BenchmarkInterfaceByName                 20000             79908 ns/op
BenchmarkInterfaceAddrs                   2000            836413 ns/op
BenchmarkInterfacesAndAddrs               5000            605946 ns/op
BenchmarkInterfacesAndMulticastAddrs     10000            169029 ns/op

Update #4866.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7368046

Overview: Function.finish() now invokes the "lifting" pass which replaces local allocs and loads and stores to such cells by SSA registers.  We use very standard machinery:

(1) we build the dominator tree for the function's control flow graph (CFG) using the "Simple" Lengauer-Tarjan algorithm.  (Very "simple" in fact: even simple path compression is not yet implemented.)

In sanity-checking mode, we cross check the dominator tree against an alternative implementation using a simple iterative dataflow algorithm.
This all lives in dom.go, along with some diagnostic printing routines.

(2) we build the dominance frontier for the entire CFG using the Cytron et al algorithm.  The DF is represented as a slice of slices, keyed by block index.  See buildDomFrontier() in lift.go.

(3) we determine for each Alloc whether it can be lifted: is it only subject to loads and stores?  If so, we traverse the iterated dominance frontier (IDF) creating φ-nodes; they are not prepended to the blocks yet.
See liftAlloc() in lift.go.

(4) we perform the SSA renaming algorithm from Cytron et al, replacing all loads to lifted Alloc cells by the value stored by the dominating store operation, and deleting the stores and allocs.  See rename() in lift.go.

(5) we eliminate unneeded φ-nodes, then concatenate the remaining ones with the non-deleted instructions of the block into a new slice.  We eliminate any lifted allocs from Function.Locals.

To ease reviewing, I have avoided almost all optimisations at this point, though there are many opportunities to explore.  These will be easier to understand as follow-up changes.

All the existing tests (pending CL 7313062) pass.  (Faster!)

Details:

"NaiveForm" BuilderMode flag suppresses all the new logic.
Exposed as 'ssadump -build=N'.

BasicBlock:
- add .Index field (b.Func[b.Index]==b), simplifying
  algorithms such as Kildall-style dataflow with bitvectors.
- rename the Name field to Comment to better reflect its
  reduced purpose.  It now has a String() method.
- 'dom' field holds dominator tree node; private for now.
- new predIndex method.
- hasPhi is now a method

dom.go:
- domTree: a new struct for a node in a dominator tree.
- buildDomTree builds the dominator tree using the simple
  variant Lengauer/Tarjan algorithm with Georgiadis'
  bucket optimizations.
- sanityCheckDomTree builds dominance relation using
  Kildall-style dataflow and ensures the same result is
  obtained.
- printDomTreeDot prints the CFG/DomTree in GraphViz format.

blockopt.go:
- perform a mark/sweep pass to eliminate unreachable
  cycles; the previous prune() opt would only eliminate
  trivially dead blocks.  (Needed for LT algo.)
- using .Index, fuseblocks can now delete fused blocks directly.
- delete prune().

sanity.go: more consistency checks:
- Phi with missing edge value
- local Alloc instructions must appear in Function.Locals.
- BasicBlock.Index, Func consistency
- CFG edges are all intraprocedural.
- detect nils in BasicBlock.Instrs.
- detect Function.Locals with Heap flag set.
- check fn.Blocks is nil if empty.

Also:
- Phi now has Comment field for debugging.
- Fixed bug in Select.Operands()
  (took address of temporary copy of field)
- new Literal constructor zeroLiteral().
- algorithms steal private fields Alloc.index,
  BasicBlock.gaps to avoid allocating maps.
- We print Function.Locals in DumpTo.
- added profiling support to ssadump.

R=iant, gri
CC=golang-dev
https://golang.org/cl/7229074

mpreinit() is called on the parent thread and with mcache (can allocate memory),
minit() is called on the child thread and can not allocate memory.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7389043

And add a test too, for Alex. :)

Fixes #3734

R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/7399046