1. 04 Mar, 2017 6 commits
  2. 03 Mar, 2017 19 commits
    • Matthew Dempsky's avatar
      cmd/compile/internal/gc: replace Node.Ullman with Node.HasCall · 870d079c
      Matthew Dempsky authored
      Since switching to SSA, the only remaining use for the Ullman field
      was in tracking whether or not an expression contained a function
      call. Give it a new name and encode it in our fancy new bitset field.
      
      Passes toolstash-check.
      
      Change-Id: I95b7f9cb053856320c0d66efe14996667e6011c2
      Reviewed-on: https://go-review.googlesource.com/37721
      Run-TryBot: Matthew Dempsky <mdempsky@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarJosh Bleecher Snyder <josharian@gmail.com>
      870d079c
    • David Lazar's avatar
      cmd/compile: include position info in exported function bodies · 9fd359a2
      David Lazar authored
      This gives accurate line numbers to inlined functions from another
      package. Previously AST nodes from another package would get the line
      number of the import statement for that package.
      
      The following benchmark results show how the size of package export data
      is impacted by this change. These benchmarks were created by compiling
      the go1 benchmark and running `go tool pack x` to extract the export
      data from the resulting .a files.
      
      name                                          old bytes   new bytes    delta
      bufio                                         3.59k ± 0%   4.17k ± 0%  +16.25%
      bytes                                         5.51k ± 0%   6.40k ± 0%  +16.21%
      compress/bzip2                                2.69k ± 0%   3.21k ± 0%  +19.74%
      compress/flate                                5.14k ± 0%   5.57k ± 0%   +8.43%
      compress/gzip                                 8.91k ± 0%  10.46k ± 0%  +17.32%
      container/list                                1.76k ± 0%   2.13k ± 0%  +21.51%
      context                                       4.51k ± 0%   5.47k ± 0%  +21.43%
      crypto                                        1.11k ± 0%   1.13k ± 0%   +1.90%
      crypto/aes                                      475 ± 0%     475 ± 0%   +0.00%
      crypto/cipher                                 1.18k ± 0%   1.18k ± 0%   +0.00%
      crypto/des                                      502 ± 0%     502 ± 0%   +0.00%
      crypto/dsa                                    5.96k ± 0%   6.54k ± 0%   +9.82%
      crypto/ecdsa                                  6.93k ± 0%   7.69k ± 0%  +10.91%
      crypto/elliptic                               6.53k ± 0%   7.17k ± 0%   +9.72%
      crypto/hmac                                     464 ± 0%     464 ± 0%   +0.00%
      crypto/internal/cipherhw                        313 ± 0%     313 ± 0%   +0.00%
      crypto/md5                                      695 ± 0%     711 ± 0%   +2.30%
      crypto/rand                                   5.62k ± 0%   6.21k ± 0%  +10.44%
      crypto/rc4                                      512 ± 0%     512 ± 0%   +0.00%
      crypto/rsa                                    7.31k ± 0%   8.10k ± 0%  +10.86%
      crypto/sha1                                     760 ± 0%     777 ± 0%   +2.24%
      crypto/sha256                                   523 ± 0%     523 ± 0%   +0.00%
      crypto/sha512                                   663 ± 0%     663 ± 0%   +0.00%
      crypto/subtle                                   873 ± 0%    1007 ± 0%  +15.35%
      crypto/tls                                    29.6k ± 0%   33.8k ± 0%  +14.03%
      crypto/x509                                   18.7k ± 0%   21.0k ± 0%  +12.56%
      crypto/x509/pkix                              10.6k ± 0%   12.2k ± 0%  +15.22%
      encoding                                        473 ± 0%     473 ± 0%   +0.00%
      encoding/asn1                                 1.42k ± 0%   1.50k ± 0%   +5.99%
      encoding/base64                               1.69k ± 0%   1.80k ± 0%   +6.88%
      encoding/binary                               2.76k ± 0%   3.51k ± 0%  +27.09%
      encoding/gob                                  13.5k ± 0%   15.2k ± 0%  +12.98%
      encoding/hex                                    857 ± 0%     881 ± 0%   +2.80%
      encoding/json                                 12.5k ± 0%   14.9k ± 0%  +19.37%
      encoding/pem                                    484 ± 0%     484 ± 0%   +0.00%
      errors                                          361 ± 0%     370 ± 0%   +2.49%
      flag                                          10.5k ± 0%   12.1k ± 0%  +14.92%
      fmt                                           1.42k ± 0%   1.42k ± 0%   +0.00%
      go/ast                                        15.8k ± 0%   17.5k ± 0%  +10.31%
      go/parser                                     8.13k ± 0%   9.86k ± 0%  +21.28%
      go/scanner                                    3.94k ± 0%   4.53k ± 0%  +14.73%
      go/token                                      3.53k ± 0%   3.75k ± 0%   +6.11%
      hash                                            507 ± 0%     507 ± 0%   +0.00%
      hash/crc32                                      685 ± 0%     685 ± 0%   +0.00%
      internal/nettrace                               474 ± 0%     474 ± 0%   +0.00%
      internal/poll                                 7.23k ± 0%   8.38k ± 0%  +15.90%
      internal/race                                   511 ± 0%     515 ± 0%   +0.78%
      internal/singleflight                           969 ± 0%    1075 ± 0%  +10.94%
      internal/syscall/unix                           427 ± 0%     427 ± 0%   +0.00%
      io                                            3.52k ± 0%   3.69k ± 0%   +4.82%
      io/ioutil                                     8.48k ± 0%   9.90k ± 0%  +16.72%
      log                                           5.06k ± 0%   5.98k ± 0%  +18.15%
      math                                          4.02k ± 0%   4.35k ± 0%   +8.26%
      math/big                                      9.28k ± 0%   9.94k ± 0%   +7.13%
      math/bits                                     3.47k ± 0%   4.33k ± 0%  +24.83%
      math/rand                                     1.30k ± 0%   1.32k ± 0%   +2.00%
      mime                                          2.98k ± 0%   3.70k ± 0%  +24.21%
      mime/multipart                                3.68k ± 0%   4.22k ± 0%  +14.65%
      mime/quotedprintable                          2.26k ± 0%   2.65k ± 0%  +17.60%
      net                                           23.0k ± 0%   25.7k ± 0%  +11.74%
      net/http                                      59.1k ± 0%   66.7k ± 0%  +13.00%
      net/http/httptest                             35.3k ± 0%   40.9k ± 0%  +15.80%
      net/http/httptrace                            15.3k ± 0%   17.6k ± 0%  +15.26%
      net/http/internal                             2.77k ± 0%   3.27k ± 0%  +17.89%
      net/textproto                                 4.60k ± 0%   5.25k ± 0%  +14.22%
      net/url                                       1.73k ± 0%   1.84k ± 0%   +6.59%
      os                                            14.3k ± 0%   16.4k ± 0%  +14.86%
      path                                            589 ± 0%     606 ± 0%   +2.89%
      path/filepath                                 5.07k ± 0%   6.17k ± 0%  +21.79%
      reflect                                       6.43k ± 0%   6.81k ± 0%   +5.90%
      regexp                                        5.88k ± 0%   6.46k ± 0%   +9.77%
      regexp/syntax                                 3.24k ± 0%   3.29k ± 0%   +1.73%
      runtime                                       13.1k ± 0%   14.9k ± 0%  +13.73%
      runtime/cgo                                     229 ± 0%     229 ± 0%   +0.00%
      runtime/debug                                 4.23k ± 0%   5.15k ± 0%  +21.79%
      runtime/internal/atomic                         905 ± 0%     905 ± 0%   +0.00%
      runtime/internal/sys                          2.04k ± 0%   2.20k ± 0%   +7.64%
      runtime/pprof                                 4.73k ± 0%   5.65k ± 0%  +19.41%
      runtime/trace                                   354 ± 0%     354 ± 0%   +0.00%
      sort                                          1.68k ± 0%   1.85k ± 0%  +10.17%
      strconv                                       1.85k ± 0%   1.95k ± 0%   +5.51%
      strings                                       3.98k ± 0%   4.53k ± 0%  +13.91%
      sync                                          1.52k ± 0%   1.58k ± 0%   +4.28%
      sync/atomic                                   1.60k ± 0%   1.74k ± 0%   +8.50%
      syscall                                       53.3k ± 0%   54.3k ± 0%   +1.84%
      testing                                       8.77k ± 0%  10.09k ± 0%  +14.96%
      testing/internal/testdeps                       598 ± 0%     600 ± 0%   +0.33%
      text/tabwriter                                3.63k ± 0%   4.41k ± 0%  +21.64%
      text/template                                 15.7k ± 0%   18.1k ± 0%  +15.67%
      text/template/parse                           9.12k ± 0%  10.35k ± 0%  +13.48%
      time                                          6.38k ± 0%   7.14k ± 0%  +11.81%
      unicode                                       4.62k ± 0%   4.66k ± 0%   +0.98%
      unicode/utf16                                   707 ± 0%     791 ± 0%  +11.88%
      unicode/utf8                                  1.06k ± 0%   1.20k ± 0%  +12.63%
      vendor/golang_org/x/crypto/chacha20poly1305   1.26k ± 0%   1.43k ± 0%  +13.54%
      vendor/golang_org/x/crypto/curve25519           392 ± 0%     392 ± 0%   +0.00%
      vendor/golang_org/x/crypto/poly1305             426 ± 0%     426 ± 0%   +0.00%
      vendor/golang_org/x/net/http2/hpack           4.75k ± 0%   5.77k ± 0%  +21.42%
      vendor/golang_org/x/net/idna                    355 ± 0%     355 ± 0%   +0.00%
      vendor/golang_org/x/net/lex/httplex             616 ± 0%     644 ± 0%   +4.55%
      vendor/golang_org/x/net/proxy                 7.76k ± 0%   9.58k ± 0%  +23.37%
      vendor/golang_org/x/text/transform            1.31k ± 0%   1.32k ± 0%   +0.46%
      vendor/golang_org/x/text/unicode/norm         5.89k ± 0%   6.84k ± 0%  +16.06%
      vendor/golang_org/x/text/width                1.24k ± 0%   1.27k ± 0%   +2.66%
      [Geo mean]                                    2.51k        2.74k        +9.14%
      
      Change-Id: I9ded911bb0ff63c530795fc85253d76b56d8abbc
      Reviewed-on: https://go-review.googlesource.com/37239
      Run-TryBot: David Lazar <lazard@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarAustin Clements <austin@google.com>
      9fd359a2
    • David Lazar's avatar
      cmd/compile: add flag for debugging PC-value tables · 0824ae6d
      David Lazar authored
      For example, `-d pctab=pctoinline` prints the PC-inline table and
      inlining tree for every function.
      
      Change-Id: Ia6b9ce4d83eed0b494318d40ffe06481ec5d58ab
      Reviewed-on: https://go-review.googlesource.com/37235
      Run-TryBot: David Lazar <lazard@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarAustin Clements <austin@google.com>
      0824ae6d
    • David Lazar's avatar
      cmd/internal/obj: avoid duplicate file name symbols · 301149b9
      David Lazar authored
      The meaning of Version=1 was overloaded: it was reserved for file name
      symbols (to avoid conflicts with non-file name symbols), but was also
      used to mean "give me a fresh version number for this symbol."
      
      With the new inlining tree, the same file name symbol can appear in
      multiple entries, but each one would become a distinct symbol with its
      own version number.
      
      Now, we avoid duplicating symbols by using Version=0 for file name
      symbols and we avoid conflicts with other symbols by prefixing the
      symbol name with "gofile..".
      
      Change-Id: I8d0374053b8cdb6a9ca7fb71871b69b4dd369a9c
      Reviewed-on: https://go-review.googlesource.com/37234
      Run-TryBot: David Lazar <lazard@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarAustin Clements <austin@google.com>
      Reviewed-by: 's avatarRuss Cox <rsc@golang.org>
      301149b9
    • David Lazar's avatar
      runtime: use inlining tables to generate accurate tracebacks · 781fd399
      David Lazar authored
      The code in https://play.golang.org/p/aYQPrTtzoK now produces the
      following stack trace:
      
      goroutine 1 [running]:
      main.(*point).negate(...)
      	/tmp/go/main.go:8
      main.main()
      	/tmp/go/main.go:14 +0x23
      
      Previously the stack trace missed the inlined call:
      
      goroutine 1 [running]:
      main.main()
      	/tmp/go/main.go:14 +0x23
      
      Fixes #10152.
      Updates #19348.
      
      Change-Id: Ib43c67012f53da0ef1a1e69bcafb65b57d9cecb2
      Reviewed-on: https://go-review.googlesource.com/37233
      Run-TryBot: David Lazar <lazard@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarAustin Clements <austin@google.com>
      781fd399
    • David Lazar's avatar
      cmd/compile: copy literals when inlining · 1c6ef9ae
      David Lazar authored
      Without this, literals keep their original source positions through
      inlining, which results in strange jumps in line numbers of inlined
      function bodies. By copying literals, inlining can update their source
      position like other nodes.
      
      Fixes #15453.
      
      Change-Id: Iad5d9bbfe183883794213266dc30e31bab89ee69
      Reviewed-on: https://go-review.googlesource.com/37232
      Run-TryBot: David Lazar <lazard@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMatthew Dempsky <mdempsky@google.com>
      Reviewed-by: 's avatarRuss Cox <rsc@golang.org>
      1c6ef9ae
    • David Lazar's avatar
      cmd/compile,link: generate PC-value tables with inlining information · 699175a1
      David Lazar authored
      In order to generate accurate tracebacks, the runtime needs to know the
      inlined call stack for a given PC. This creates two tables per function
      for this purpose. The first table is the inlining tree (stored in the
      function's funcdata), which has a node containing the file, line, and
      function name for every inlined call. The second table is a PC-value
      table that maps each PC to a node in the inlining tree (or -1 if the PC
      is not the result of inlining).
      
      To give the appearance that inlining hasn't happened, the runtime also
      needs the original source position information of inlined AST nodes.
      Previously the compiler plastered over the line numbers of inlined AST
      nodes with the line number of the call. This meant that the PC-line
      table mapped each PC to line number of the outermost call in its inlined
      call stack, with no way to access the innermost line number.
      
      Now the compiler retains line numbers of inlined AST nodes and writes
      the innermost source position information to the PC-line and PC-file
      tables. Some tools and tests expect to see outermost line numbers, so we
      provide the OutermostLine function for displaying line info.
      
      To keep track of the inlined call stack for an AST node, we extend the
      src.PosBase type with an index into a global inlining tree. Every time
      the compiler inlines a call, it creates a node in the global inlining
      tree for the call, and writes its index to the PosBase of every inlined
      AST node. The parent of this node is the inlining tree index of the
      call. -1 signifies no parent.
      
      For each function, the compiler creates a local inlining tree and a
      PC-value table mapping each PC to an index in the local tree.  These are
      written to an object file, which is read by the linker.  The linker
      re-encodes these tables compactly by deduplicating function names and
      file names.
      
      This change increases the size of binaries by 4-5%. For example, this is
      how the go1 benchmark binary is impacted by this change:
      
      section             old bytes   new bytes   delta
      .text               3.49M ± 0%  3.49M ± 0%   +0.06%
      .rodata             1.12M ± 0%  1.21M ± 0%   +8.21%
      .gopclntab          1.50M ± 0%  1.68M ± 0%  +11.89%
      .debug_line          338k ± 0%   435k ± 0%  +28.78%
      Total               9.21M ± 0%  9.58M ± 0%   +4.01%
      
      Updates #19348.
      
      Change-Id: Ic4f180c3b516018138236b0c35e0218270d957d3
      Reviewed-on: https://go-review.googlesource.com/37231
      Run-TryBot: David Lazar <lazard@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarAustin Clements <austin@google.com>
      699175a1
    • Aliaksandr Valialkin's avatar
      cmd/compile: pack bool fields in Node, Name, Func and Type structs to bitsets · ed70f37e
      Aliaksandr Valialkin authored
      This reduces compiler memory usage by up to 4% - see compilebench
      results below.
      
      name       old time/op     new time/op     delta
      Template       245ms ± 4%      241ms ± 2%  -1.88%  (p=0.029 n=10+10)
      Unicode        126ms ± 3%      124ms ± 3%    ~     (p=0.105 n=10+10)
      GoTypes        805ms ± 2%      813ms ± 3%    ~     (p=0.515 n=8+10)
      Compiler       3.95s ± 2%      3.83s ± 1%  -2.96%  (p=0.000 n=9+10)
      MakeBash       47.4s ± 4%      46.6s ± 1%  -1.59%  (p=0.028 n=9+10)
      
      name       old user-ns/op  new user-ns/op  delta
      Template        324M ± 5%       326M ± 3%    ~     (p=0.935 n=10+10)
      Unicode         186M ± 5%       178M ±10%    ~     (p=0.067 n=9+10)
      GoTypes        1.08G ± 7%      1.09G ± 4%    ~     (p=0.956 n=10+10)
      Compiler       5.34G ± 4%      5.31G ± 1%    ~     (p=0.501 n=10+8)
      
      name       old alloc/op    new alloc/op    delta
      Template      41.0MB ± 0%     39.8MB ± 0%  -3.03%  (p=0.000 n=10+10)
      Unicode       32.3MB ± 0%     31.0MB ± 0%  -4.13%  (p=0.000 n=10+10)
      GoTypes        119MB ± 0%      116MB ± 0%  -2.39%  (p=0.000 n=10+10)
      Compiler       499MB ± 0%      487MB ± 0%  -2.48%  (p=0.000 n=10+10)
      
      name       old allocs/op   new allocs/op   delta
      Template        380k ± 1%       379k ± 1%    ~     (p=0.436 n=10+10)
      Unicode         324k ± 1%       324k ± 0%    ~     (p=0.853 n=10+10)
      GoTypes        1.15M ± 0%      1.15M ± 0%    ~     (p=0.481 n=10+10)
      Compiler       4.41M ± 0%      4.41M ± 0%  -0.12%  (p=0.007 n=10+10)
      
      name       old text-bytes  new text-bytes  delta
      HelloSize       623k ± 0%       623k ± 0%    ~     (all equal)
      CmdGoSize      6.64M ± 0%      6.64M ± 0%    ~     (all equal)
      
      name       old data-bytes  new data-bytes  delta
      HelloSize      5.81k ± 0%      5.81k ± 0%    ~     (all equal)
      CmdGoSize       238k ± 0%       238k ± 0%    ~     (all equal)
      
      name       old bss-bytes   new bss-bytes   delta
      HelloSize       134k ± 0%       134k ± 0%    ~     (all equal)
      CmdGoSize       152k ± 0%       152k ± 0%    ~     (all equal)
      
      name       old exe-bytes   new exe-bytes   delta
      HelloSize       967k ± 0%       967k ± 0%    ~     (all equal)
      CmdGoSize      10.2M ± 0%      10.2M ± 0%    ~     (all equal)
      
      Change-Id: I1f40af738254892bd6c8ba2eb43390b175753d52
      Reviewed-on: https://go-review.googlesource.com/37445Reviewed-by: 's avatarMatthew Dempsky <mdempsky@google.com>
      Run-TryBot: Matthew Dempsky <mdempsky@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      ed70f37e
    • Johan Brandhorst's avatar
      net/http/httptest: add Client and Certificate methods to Server · fbf4dd91
      Johan Brandhorst authored
      Adds a function for easily accessing the x509.Certificate
      of a Server, if there is one. Also adds a helper function
      for getting a http.Client suitable for use with the server.
      
      This makes the steps required to test a httptest
      TLS server simpler.
      
      Fixes #18411
      
      Change-Id: I2e78fe1e54e31bed9c641be2d9a099f698c7bbde
      Reviewed-on: https://go-review.googlesource.com/34639Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      fbf4dd91
    • Matthew Dempsky's avatar
      cmd/compile/internal/ssa: remove Hmul{8,16}{,u} ops · 02e36f8c
      Matthew Dempsky authored
      Change-Id: I90865921584ae4bdfb6c220d439b14593d72b6f9
      Reviewed-on: https://go-review.googlesource.com/37752
      Run-TryBot: Matthew Dempsky <mdempsky@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarCherry Zhang <cherryyz@google.com>
      02e36f8c
    • Cherry Zhang's avatar
      cmd/compile: remove zeroing after newobject · c8eaeb8c
      Cherry Zhang authored
      The Zero op right after newobject has been removed. But this rule
      does not cover Store of constant zero (for SSA-able types). Add
      rules to cover Store op as well.
      
      Updates #19027.
      
      Change-Id: I5d2b62eeca0aa9ce8dc7205b264b779de01c660b
      Reviewed-on: https://go-review.googlesource.com/36836
      Run-TryBot: Cherry Zhang <cherryyz@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarKeith Randall <khr@golang.org>
      c8eaeb8c
    • Cherry Zhang's avatar
      cmd/compile: fix optimization of Zero newobject on amd64p32 · 9b480521
      Cherry Zhang authored
      On amd64p32, PtrSize and RegSize don't agree, and function return
      value is aligned with RegSize. Fix this rule. Other architectures
      are not affected, where PtrSize and RegSize are the same.
      
      Change-Id: If187d3dfde3dc3b931b8e97db5eeff49a781551b
      Reviewed-on: https://go-review.googlesource.com/37720
      Run-TryBot: Cherry Zhang <cherryyz@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarKeith Randall <khr@golang.org>
      9b480521
    • Matthew Dempsky's avatar
      cmd/compile/internal/gc: remove OHMUL Op · d8a0f748
      Matthew Dempsky authored
      Previously the compiler rewrote constant division into OHMUL
      operations, but that rewriting was moved to SSA in CL 37015. Now OHMUL
      is unused, so we can get rid of it.
      
      Change-Id: Ib6fc7c2b6435510bafb5735b3b4f42cfd8ed8cdb
      Reviewed-on: https://go-review.googlesource.com/37750
      Run-TryBot: Matthew Dempsky <mdempsky@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarCherry Zhang <cherryyz@google.com>
      Reviewed-by: 's avatarKeith Randall <khr@golang.org>
      d8a0f748
    • Austin Clements's avatar
      runtime: clarify access to mheap_.busy · 77f64c50
      Austin Clements authored
      There are two accesses to mheap_.busy that are guarded by checks
      against len(mheap_.free). This works because both lists are (and must
      be) the same length, but it makes the code less clear. Change these to
      use len(mheap_.busy) so the access more clearly parallels the check.
      
      Fixes #18944.
      
      Change-Id: I9bacbd3663988df351ed4396ae9018bc71018311
      Reviewed-on: https://go-review.googlesource.com/36354
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
      77f64c50
    • Austin Clements's avatar
      runtime: simplify sweep allocation counting · b50b7285
      Austin Clements authored
      Currently sweep counts the number of allocated objects, computes the
      number of free objects from that, then re-computes the number of
      allocated objects from that. Simplify and clean this up by skipping
      these intermediate steps.
      
      Change-Id: I3ed98e371eb54bbcab7c8530466c4ab5fde35f0a
      Reviewed-on: https://go-review.googlesource.com/34935
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMarvin Stenger <marvin.stenger94@gmail.com>
      Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
      b50b7285
    • Austin Clements's avatar
      runtime: don't rescan finalizers queue during mark termination · f1ba75f8
      Austin Clements authored
      Currently we scan the finalizers queue both during concurrent mark and
      during mark termination. This costs roughly 20ns per queued finalizer
      and about 1ns per unused finalizer queue slot (allocated queue length
      never decreases), which can drive up STW time if there are many
      finalizers.
      
      However, we only add finalizers to this queue during sweeping, which
      means that the second scan will never find anything new. Hence, we can
      fix this by simply not scanning the finalizers queue during mark
      termination. This brings the STW time under the 100µs goal even with
      1,000,000 queued finalizers.
      
      Fixes #18869.
      
      Change-Id: I4ce5620c66fb7f13ebeb39ca313ce57047d1d0fb
      Reviewed-on: https://go-review.googlesource.com/36013
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
      f1ba75f8
    • Austin Clements's avatar
      runtime: remove wbufptr · 98da2d1f
      Austin Clements authored
      Since workbuf is now marked go:notinheap, the write barrier-preventing
      wrapper type wbufptr is no longer necessary. Remove it.
      
      Change-Id: I3e5b5803a1547d65de1c1a9c22458a38e08549b7
      Reviewed-on: https://go-review.googlesource.com/35971
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarRick Hudson <rlh@golang.org>
      98da2d1f
    • Austin Clements's avatar
      cmd/compile: accept string debug flags · 8eb14e9d
      Austin Clements authored
      The compiler's -d flag accepts string-valued flags, but currently only
      for SSA debug flags. Extend it to support string values for other
      flags. This also makes the syntax somewhat more sane so flag=value and
      flag:value now both accept integers and strings.
      
      Change-Id: Idd144d8479a430970cc1688f824bffe0a56ed2df
      Reviewed-on: https://go-review.googlesource.com/37345
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarDavid Chase <drchase@google.com>
      8eb14e9d
    • Cherry Zhang's avatar
      cmd/compile: get rid of "volatile" in SSA · 5bfd1ef0
      Cherry Zhang authored
      A value is "volatile" if it is a pointer to the argument region
      on stack which will be clobbered by function call. This is used
      to make sure the value is safe when inserting write barrier calls.
      The writebarrier pass can tell whether a value is such a pointer.
      Therefore no need to mark it when building SSA and thread this
      information through.
      
      Passes "toolstash -cmp" on std.
      
      Updates #17583.
      
      Change-Id: Idc5fc0d710152b94b3c504ce8db55ea9ff5b5195
      Reviewed-on: https://go-review.googlesource.com/36835
      Run-TryBot: Cherry Zhang <cherryyz@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarKeith Randall <khr@golang.org>
      5bfd1ef0
  3. 02 Mar, 2017 15 commits
    • Will Storey's avatar
      image/gif: handle an extra data sub-block byte. · 4775b7fe
      Will Storey authored
      This changes the decoder's behaviour when there is stray/extra data
      found after an image is decompressed (e.g., data sub-blocks after an LZW
      End of Information Code). Instead of raising an error, we silently skip
      over such data until we find the end of the image data marked by a Block
      Terminator. We skip at most one byte as sample problem GIFs exhibit this
      property.
      
      GIFs should not have and do not need such stray data (though the
      specification is arguably ambiguous). However GIFs with such properties
      have been seen in the wild.
      
      Fixes #16146
      
      Change-Id: Ie7e69052bab5256b4834992304e6ca58e93c1879
      Reviewed-on: https://go-review.googlesource.com/37258Reviewed-by: 's avatarNigel Tao <nigeltao@golang.org>
      Run-TryBot: Nigel Tao <nigeltao@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      4775b7fe
    • Josh Bleecher Snyder's avatar
      runtime/pprof: fix data race between Profile.Add and Profile.WriteTo · 9b15c13d
      Josh Bleecher Snyder authored
      p.m is accessed in WriteTo without holding p.mu.
      Move the access inside the critical section.
      
      The race detector catches this bug using this program:
      
      
      package main
      
      import (
      	"os"
      	"runtime/pprof"
      	"time"
      )
      
      func main() {
      	p := pprof.NewProfile("ABC")
      	go func() {
      		p.WriteTo(os.Stdout, 1)
      		time.Sleep(time.Second)
      	}()
      	p.Add("abc", 0)
      	time.Sleep(time.Second)
      }
      
      
      $ go run -race x.go 
      ==================
      WARNING: DATA RACE
      Write at 0x00c42007c240 by main goroutine:
        runtime.mapassign()
            /Users/josh/go/tip/src/runtime/hashmap.go:485 +0x0
        runtime/pprof.(*Profile).Add()
            /Users/josh/go/tip/src/runtime/pprof/pprof.go:281 +0x255
        main.main()
            /Users/josh/go/tip/src/p.go:15 +0x9d
      
      Previous read at 0x00c42007c240 by goroutine 6:
        runtime/pprof.(*Profile).WriteTo()
            /Users/josh/go/tip/src/runtime/pprof/pprof.go:314 +0xc5
        main.main.func1()
            /Users/josh/go/tip/src/x.go:12 +0x69
      
      Goroutine 6 (running) created at:
        main.main()
            /Users/josh/go/tip/src/x.go:11 +0x6e
      ==================
      ABC profile: total 1
      1 @ 0x110ccb4 0x111aeee 0x1055053 0x107f031
      
      Found 1 data race(s)
      exit status 66
      
      
      (Exit status 66?)
      
      Change-Id: I49d884dc3af9cce2209057a3448fe6bf50653523
      Reviewed-on: https://go-review.googlesource.com/37730
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      9b15c13d
    • Robert Griesemer's avatar
      go/types: don't exclude package unsafe from a Package's Imports list · 72359cf8
      Robert Griesemer authored
      There's no good reason to exclude it and it only makes the code more
      complicated and less consistent. Having it in the list provides an
      easy way to detect if a package uses operations from package unsafe.
      
      Change-Id: I2f9b0485db0a680bd82f3b93a350b048db3f7701
      Reviewed-on: https://go-review.googlesource.com/37694Reviewed-by: 's avatarAlan Donovan <adonovan@google.com>
      72359cf8
    • Robert Griesemer's avatar
      go/types: support type checking of external tests with gotype · 359ca5cc
      Robert Griesemer authored
      - renamed -a flag to -t
      - added -x flag to specify external test files
      - improved documentation and usage string
      
      Change-Id: I7c850bd28a10ceaa55d599c22db07774147aa3f7
      Reviewed-on: https://go-review.googlesource.com/37656Reviewed-by: 's avatarAlan Donovan <adonovan@google.com>
      359ca5cc
    • Robert Griesemer's avatar
      go/types: gotype to always report the same first error now · 9eac1c87
      Robert Griesemer authored
      The old code may have reported different errors given an
      erroneous package depending on the order in which files
      were parsed concurrently. The new code always reports
      errors in "file order", independent of processing order.
      
      Also:
      - simplified parsing code and internal concurrency control
      - removed -seq flag which didn't really add useful functionality
      
      Change-Id: I18e24e630f458f2bc107a7b83926ae761d63c334
      Reviewed-on: https://go-review.googlesource.com/37655Reviewed-by: 's avatarAlan Donovan <adonovan@google.com>
      9eac1c87
    • Josh Bleecher Snyder's avatar
      cmd/dist, cmd/compile: eliminate mergeEnvLists copies · 3a90bfb2
      Josh Bleecher Snyder authored
      This is now handled by os/exec.
      
      Updates #12868
      
      Change-Id: Ic21a6ff76a9b9517437ff1acf3a9195f9604bb45
      Reviewed-on: https://go-review.googlesource.com/37698Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      3a90bfb2
    • Brad Fitzpatrick's avatar
      net/http: clean up Transport.RoundTrip error handling · 9d29be46
      Brad Fitzpatrick authored
      If I put a 10 millisecond sleep at testHookWaitResLoop, before the big
      select in (*persistConn).roundTrip, two flakes immediately started
      happening, TestTransportBodyReadError (#19231) and
      TestTransportPersistConnReadLoopEOF.
      
      The problem was that there are many ways for a RoundTrip call to fail
      (errors reading from Request.Body while writing the response, errors
      writing the response, errors reading the response due to server
      closes, errors due to servers sending malformed responses,
      cancelations, timeouts, etc.), and many of those failures then tear
      down the TCP connection, causing more failures, since there are always
      at least three goroutines involved (reading, writing, RoundTripping).
      
      Because the errors were communicated over buffered channels to a giant
      select, the error returned to the caller was a function of which
      random select case was called, which was why a 10ms delay before the
      select brought out so many bugs. (several fixed in my previous CLs the past
      few days).
      
      Instead, track the error explicitly in the transportRequest, guarded
      by a mutex.
      
      In addition, this CL now:
      
      * differentiates between the two ways writing a request can fail: the
        io.Copy reading from the Request.Body or the io.Copy writing to the
        network. A new io.Reader type notes read errors from the
        Request.Body. The read-from-body vs write-to-network errors are now
        prioritized differently.
      
      * unifies the two mapRoundTripErrorFromXXX methods into one
        mapRoundTripError method since their logic is now the same.
      
      * adds a (*Request).WithT(*testing.T) method in export_test.go, usable
        by tests, to call t.Logf at points during RoundTrip. This is disabled
        behind a constant except when debugging.
      
      * documents and deflakes TestClientRedirectContext
      
      I've tested this CL with high -count values, with/without -race,
      with/without delays before the select, etc. So far it seems robust.
      
      Fixes #19231 (TestTransportBodyReadError flake)
      Updates #14203 (source of errors unclear; they're now tracked more)
      Updates #15935 (document Transport errors more; at least understood more now)
      
      Change-Id: I3cccc3607f369724b5344763e35ad2b7ea415738
      Reviewed-on: https://go-review.googlesource.com/37495
      Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarJosh Bleecher Snyder <josharian@gmail.com>
      9d29be46
    • Mike Danese's avatar
      crypto/tls: make Config.Clone also clone the GetClientCertificate field · 87649d32
      Mike Danese authored
      Using GetClientCertificate with the http client is currently completely
      broken because inside the transport we clone the tls.Config and pass it
      off to the tls.Client. Since tls.Config.Clone() does not pass forward
      the GetClientCertificate field, GetClientCertificate is ignored in this
      context.
      
      Fixes #19264
      
      Change-Id: Ie214f9f0039ac7c3a2dab8ffd14d30668bdb4c71
      Signed-off-by: 's avatarMike Danese <mikedanese@google.com>
      Reviewed-on: https://go-review.googlesource.com/37541Reviewed-by: 's avatarFilippo Valsorda <hi@filippo.io>
      Reviewed-by: 's avatarAdam Langley <agl@golang.org>
      Run-TryBot: Adam Langley <agl@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      87649d32
    • Josh Bleecher Snyder's avatar
      Revert "Revert "cmd/vet/all: remove pprof from the whitelist"" · 2f5aea7c
      Josh Bleecher Snyder authored
      This reverts commit 9bd1cc3f.
      
      Reason for revert: New fixes in from upstream. Try this again.
      
      Change-Id: Iea46f32857e8467f8d5a49b31e20a52fda8bce60
      Reviewed-on: https://go-review.googlesource.com/37693Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      2f5aea7c
    • Josh Bleecher Snyder's avatar
      cmd/vet/all: remove printf hacks · 85bae0a9
      Josh Bleecher Snyder authored
      Now that vet loads from source,
      fmt can always be correctly resolved,
      so the fmt.Formatter type is always available,
      so we can reinstate the check.
      
      Change-Id: I17f0c7fccf6960c9415de8774b15123135d57be8
      Reviewed-on: https://go-review.googlesource.com/37692
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      Reviewed-by: 's avatarRob Pike <r@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      85bae0a9
    • Josh Bleecher Snyder's avatar
      cmd/vet/all: vet using only source · 8a93546d
      Josh Bleecher Snyder authored
      This simplifies the code and speeds it up.
      It also allows us to eliminate some other TODOs;
      those will come in a follow-up CL.
      
      Running for the host platform, before:
      
      real	0m9.907s
      user	0m14.566s
      sys	0m1.058s
      
      After:
      
      real	0m7.841s
      user	0m12.339s
      sys	0m0.572s
      
      Running for a single non-host platform, before:
      
      real	0m8.784s
      user	0m15.451s
      sys	0m3.445s
      
      After:
      
      real	0m7.681s
      user	0m12.122s
      sys	0m0.577s
      
      Running for all platforms, before:
      
      real	7m4.480s
      user	8m43.398s
      sys	1m15.683s
      
      After:
      
      real	4m37.596s
      user	7m30.729s
      sys	0m18.533s
      
      It also makes my laptop considerably more
      responsive while running for all platforms.
      
      Change-Id: I748689fea0d2d4ef61aca2ce5524d03d8fafa5ca
      Reviewed-on: https://go-review.googlesource.com/37691Reviewed-by: 's avatarRob Pike <r@golang.org>
      8a93546d
    • Josh Bleecher Snyder's avatar
      cmd/vet: support importing from source · ddbee9ab
      Josh Bleecher Snyder authored
      Add a -source flag to cmd/vet that instructs
      it to typecheck purely from source code.
      
      Updates #16086
      Fixes #19332
      
      Change-Id: Ic83d0f14d5bb837a329d539b2873aeccdf7bf669
      Reviewed-on: https://go-review.googlesource.com/37690Reviewed-by: 's avatarRob Pike <r@golang.org>
      ddbee9ab
    • Josh Bleecher Snyder's avatar
      cmd/vet: refactor to support alternative importers · 7e74d432
      Josh Bleecher Snyder authored
      Instead of constructing the importer in init, do it lazily as needed.
      This lets us select the importer using a command line flag.
      The addition of the command line flag will come in a follow-up CL.
      
      Change-Id: Ieb3a5f01a34fb5bd220a95086daf5d6b37e83bb5
      Reviewed-on: https://go-review.googlesource.com/37669Reviewed-by: 's avatarRob Pike <r@golang.org>
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      7e74d432
    • Heschi Kreinick's avatar
      cmd/trace: traces may end with pending mark assists · 4de2efe9
      Heschi Kreinick authored
      There's no guarantee that all in-progress mark assists will finish
      before the trace does. Don't crash if that happens.
      
      I haven't added a test because there's quite a bit of ceremony involved
      and the bug is fairly straightforward.
      
      Change-Id: Ia1369a8e2260fc6a328ad204a1eab1063d2e2c90
      Reviewed-on: https://go-review.googlesource.com/37540Reviewed-by: 's avatarAustin Clements <austin@google.com>
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      4de2efe9
    • Josh Bleecher Snyder's avatar
      runtime: delay marking maps as writing until after first alg call · 04fc8877
      Josh Bleecher Snyder authored
      Fixes #19359
      
      Change-Id: I196b47cf0471915b6dc63785e8542aa1876ff695
      Reviewed-on: https://go-review.googlesource.com/37665
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarKeith Randall <khr@golang.org>
      04fc8877