1. 14 Aug, 2017 17 commits
    • Joe Tsai's avatar
      archive/tar: implement specialized logic for PAX format · 7ae95616
      Joe Tsai authored
      Rather than going through writeHeader, which attempts to handle all formats,
      implement writePAXHeader, which only has an understanding of the PAX format.
      
      In PAX, the USTAR header is filled out in a best-effort manner.
      Thus, we change logic of formatString and formatOctal to try their best to
      output something (possibly truncated) in the event of an error.
      
      The new implementation of PAX headers causes several tests to fail.
      An investigation into the new output reveals that the new behavior is correct,
      while the tests had actually locked in incorrect behavior before.
      
      A dump of the differences is listed below (-before, +after):
      
      << writer-big.tar >>
      
      This change is due to fact that we changed the Header.Devminor to force the
      tar.Writer to choose the GNU format over the PAX one.
      The ability to control the output is an open issue (see #18710).
      - 00000150  00 30 30 30 30 30 30 30  00 00 00 00 00 00 00 00  |.0000000........|
      + 00000150  00 ff ff ff ff ff ff ff  ff 00 00 00 00 00 00 00  |................|
      
      << writer-big-long.tar>>
      
      The previous logic generated the GNU magic values for a PAX file.
      The new logic correctly uses the USTAR magic values.
      - 00000100  00 75 73 74 61 72 20 20  00 00 00 00 00 00 00 00  |.ustar  ........|
      - 00000500  00 75 73 74 61 72 20 20  00 67 75 69 6c 6c 61 75  |.ustar  .guillau|
      + 00000100  00 75 73 74 61 72 00 30  30 00 00 00 00 00 00 00  |.ustar.00.......|
      + 00000500  00 75 73 74 61 72 00 30  30 67 75 69 6c 6c 61 75  |.ustar.00guillau|
      
      The previous logic tried to use the specified timestmap in the PAX headers file,
      but this is problematic as this timestamp can overflow, defeating the point
      of using PAX, which is intended to extend tar.
      The new logic uses the zero timestamp similar to what GNU and BSD tar do.
      - 00000080  30 30 30 30 32 33 32 00  31 32 33 33 32 37 37 30  |0000232.12332770|
      + 00000080  30 30 30 30 32 35 36 00  30 30 30 30 30 30 30 30  |0000256.00000000|
      
      The previous logic populated the devminor and devmajor fields.
      The new logic leaves them zeroed just like what GNU and BSD tar do.
      - 00000140  00 00 00 00 00 00 00 00  00 30 30 30 30 30 30 30  |.........0000000|
      - 00000150  00 30 30 30 30 30 30 30  00 00 00 00 00 00 00 00  |.0000000........|
      + 00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
      + 00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
      
      The previous logic uses PAX headers, but fails to add a record for the size.
      The new logic does properly add a record for the size.
      - 00000290  31 36 67 69 67 2e 74 78  74 0a 00 00 00 00 00 00  |16gig.txt.......|
      - 000002a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
      + 00000290  31 36 67 69 67 2e 74 78  74 0a 32 30 20 73 69 7a  |16gig.txt.20 siz|
      + 000002a0  65 3d 31 37 31 37 39 38  36 39 31 38 34 0a 00 00  |e=17179869184...|
      
      The previous logic encoded the size as a base-256 field,
      which is only valid in GNU, but the previous PAX headers implies this should
      be a PAX file. This result in a strange hybrid that is neither GNU nor PAX.
      The new logic uses PAX headers to store the size.
      - 00000470  37 35 30 00 30 30 30 31  37 35 30 00 80 00 00 00  |750.0001750.....|
      - 00000480  00 00 00 04 00 00 00 00  31 32 33 33 32 37 37 30  |........12332770|
      + 00000470  37 35 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |750.0001750.0000|
      + 00000480  30 30 30 30 30 30 30 00  31 32 33 33 32 37 37 30  |0000000.12332770|
      
      << ustar.issue12594.tar >>
      
      The previous logic used the specified timestamp for the PAX headers file.
      The new logic just uses the zero timestmap.
      - 00000080  30 30 30 30 32 33 31 00  31 32 31 30 34 34 30 32  |0000231.12104402|
      + 00000080  30 30 30 30 32 33 31 00  30 30 30 30 30 30 30 30  |0000231.00000000|
      
      The previous logic populated the devminor and devmajor fields.
      The new logic leaves them zeroed just like what GNU and BSD tar do.
      - 00000140  00 00 00 00 00 00 00 00  00 30 30 30 30 30 30 30  |.........0000000|
      - 00000150  00 30 30 30 30 30 30 30  00 00 00 00 00 00 00 00  |.0000000........|
      + 00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
      + 00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
      
      Change-Id: I33419eb1124951968e9d5a10d50027e03133c811
      Reviewed-on: https://go-review.googlesource.com/55231Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      7ae95616
    • Hiroshi Ioka's avatar
      cmd/cgo: use first error position instead of last one · b88e532a
      Hiroshi Ioka authored
      Just like https://golang.org/cl/34783
      
      Given cgo.go:
           1	package main
           2
           3	/*
           4	long double x = 0;
           5	*/
           6	import "C"
           7
           8	func main() {
           9		_ = C.x
          10		_ = C.x
          11	}
      
      Before:
          ./cgo.go:10:6: unexpected: 16-byte float type - long double
      
      After:
          ./cgo.go:9:6: unexpected: 16-byte float type - long double
      
      The above test case is not portable. So it is tested on only amd64.
      
      Change-Id: If0b84cf73d381a22e2ada71c8e9a6e6ec77ffd2e
      Reviewed-on: https://go-review.googlesource.com/54950Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      b88e532a
    • Hiroshi Ioka's avatar
      cmd/link: prefer to use constants in macho.go · 5b43bbe6
      Hiroshi Ioka authored
      We might want to replace some linker's feature by debug/macho in future.
      This CL gathers information of required constants.
      
      Change-Id: Iea14abdb32709a4f5404a17874f9c925d29ba999
      Reviewed-on: https://go-review.googlesource.com/55252Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      5b43bbe6
    • Hiroshi Ioka's avatar
      cmd/link: don't emit default entry symbol in some situations · 6424c40c
      Hiroshi Ioka authored
      Also, fix comment.
      
      Change-Id: Ieb7ba21f34730dc51ab45a652d225e4145d4b861
      Reviewed-on: https://go-review.googlesource.com/54870
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      6424c40c
    • Hiroshi Ioka's avatar
      debug/macho: add some file flags · d03b89bd
      Hiroshi Ioka authored
      Fixes #21414
      
      Change-Id: Idff6e269ae32b33253067c9f32cac25256eb7f1c
      Reviewed-on: https://go-review.googlesource.com/55251Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      d03b89bd
    • Agniva De Sarker's avatar
      encoding/hex: improve tests · 98276d6a
      Agniva De Sarker authored
      The tests for error scenarios were done by manually checking
      error strings. Improved them by checking the actual error type
      instead of just the string.
      
      Printing the actual error in case of failure instead of a
      generic string.
      
      Also added a new scenario with both an invalid byte and an
      invalid length string to verify that the length is checked first
      before doing any computation.
      
      Change-Id: Ic2a19a6d6058912632d597590186ee2d8348cb45
      Reviewed-on: https://go-review.googlesource.com/55256Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      98276d6a
    • Justin Nuß's avatar
      encoding/csv: report line start line in errors · 5d14ac74
      Justin Nuß authored
      Errors returned by Reader contain the line where the Reader originally
      encountered the error. This can be suboptimal since that line does not
      always correspond with the line the current record/field started at.
      
      This can easily happen with LazyQuotes as seen in #19019, but also
      happens for example when a quoted fields has no closing quote and
      the parser hits EOF before it finds another quote.
      
      When this happens finding the erroneous field can be somewhat
      complicated and time consuming, and in most cases it would be better to
      report the line where the record started.
      
      This change updates Reader to keep track of the line on which a record
      begins and uses it for errors instead of the current line, making it
      easier to find errors.
      
      Although a user-visible change, this should have no impact on existing
      code, since most users don't explicitly work with the line in the error
      and probably already expect the new behaviour.
      
      Updates #19019
      
      Change-Id: Ic9bc70fad2651c69435d614d537e7a9266819b05
      Reviewed-on: https://go-review.googlesource.com/52830Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      5d14ac74
    • Hiroshi Ioka's avatar
      cmd/link: remove redundant switch stmt · 342d25fc
      Hiroshi Ioka authored
      Change-Id: I5c4f8dc1e174b3438ef4fb509fac78d6e1b292c9
      Reviewed-on: https://go-review.googlesource.com/54791Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Reviewed-by: 's avatarAvelino <t@avelino.xxx>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      342d25fc
    • Rob Pike's avatar
      cmd/vet: fix a couple of minor word choices in README · b1fab099
      Rob Pike authored
      No semantic change, just clarifying a bit by choosing better words
      in a couple of places.
      
      Change-Id: I4496062ee7909baf83d4d22d25e13ef93b358b4b
      Reviewed-on: https://go-review.googlesource.com/55255Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      b1fab099
    • Tobias Klauser's avatar
      syscall: add utimensat and use it for UtimesNano on BSD and Solaris · 67c36059
      Tobias Klauser authored
      All the BSDs and Solaris support the utimensat syscall, but Darwin
      doesn't. Account for that by adding the //sys lines not to
      syscall_bsd.go but the individual OS's syscall_*.go files and implement
      utimensat on Darwin as just returning ENOSYS, such that UtimesNano will
      fall back to use utimes as it currently does unconditionally.
      
      This also adds the previously missing utimensat syscall number for
      FreeBSD and Dragonfly.
      
      Fixes #16480
      
      Change-Id: I367454c6168eb1f7150b988fa16cf02abff42f34
      Reviewed-on: https://go-review.googlesource.com/55130Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      67c36059
    • Kevin Burke's avatar
      time: remove unused parameter · 6203a79b
      Kevin Burke authored
      lookupName is only called in one location, and one of the return
      values is unused, so let's remove it.
      
      Change-Id: I35e22c7ec611e8eb349deb4f0561e212f7d9de0b
      Reviewed-on: https://go-review.googlesource.com/55232Reviewed-by: 's avatarDaniel Martí <mvdan@mvdan.cc>
      Reviewed-by: 's avatarRob Pike <r@golang.org>
      Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
      6203a79b
    • Austin Clements's avatar
      Revert "Revert "cmd/compile: discard duplicate inline method bodies"" · c8e9fd5d
      Austin Clements authored
      This reverts commit f612cd70.
      
      Reason for revert: We thought the original change had broken the
      linux/amd64 and linux/386 builders, but it turned out to be a problem
      with the build infrastructure, not the change.
      
      Change-Id: Ic3318a63464fcba8d845ac04494115a7ba620364
      Reviewed-on: https://go-review.googlesource.com/55050
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMatthew Dempsky <mdempsky@google.com>
      c8e9fd5d
    • Josh Bleecher Snyder's avatar
      runtime: remove indentation in mapiternext · e0789d73
      Josh Bleecher Snyder authored
      Invert the condition and continue, to remove indentation.
      
      Change-Id: Id62a5d9abc9a4df1193bcf15f95f70f2c2e2abac
      Reviewed-on: https://go-review.googlesource.com/55091
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMartin Möhrmann <moehrmann@google.com>
      e0789d73
    • Josh Bleecher Snyder's avatar
      runtime: simplify hashmap tooManyOverflowBuckets · f5804ce4
      Josh Bleecher Snyder authored
      This generates better code.
      
      Masking B in the return statement should be unnecessary,
      but the compiler is understandably not yet clever enough to see that.
      
      Someday, it'd also be nice for the compiler to generate
      a CMOV for the saturation if statement.
      
      Change-Id: Ie1c157b21f5212610da1f3c7823a93816b3b61b9
      Reviewed-on: https://go-review.googlesource.com/54656
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMartin Möhrmann <moehrmann@google.com>
      f5804ce4
    • Josh Bleecher Snyder's avatar
      runtime: CSE some function arguments in evacuate · aca92f35
      Josh Bleecher Snyder authored
      Shrinks evacuate's machine code a little.
      
      Change-Id: I08874c92abdc7e621bc0737e22f2a6be31542cab
      Reviewed-on: https://go-review.googlesource.com/54652
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMartin Möhrmann <moehrmann@google.com>
      aca92f35
    • Josh Bleecher Snyder's avatar
      runtime: remove indentation in evacuate · a6136ded
      Josh Bleecher Snyder authored
      Combine conditions into a single if statement.
      This is more readable.
      
      It should generate identical machine code, but it doesn't.
      The new code is shorter.
      
      Change-Id: I9bf52f8f288b0df97a2b9b4e4183f6ca74175e8a
      Reviewed-on: https://go-review.googlesource.com/54651
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMartin Möhrmann <moehrmann@google.com>
      a6136ded
    • Josh Bleecher Snyder's avatar
      runtime: make map deletion benchmarks faster to run · 38044eca
      Josh Bleecher Snyder authored
      This reduces the wall time to run these benchmarks by about 30%.
      
      Change-Id: I494a93c93e5acb1514510d85f65796f62e1629a5
      Reviewed-on: https://go-review.googlesource.com/54650
      Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarMartin Möhrmann <moehrmann@google.com>
      38044eca
  2. 13 Aug, 2017 6 commits
    • Austin Clements's avatar
      runtime: support DT_GNU_HASH in VDSO · 9065c3bf
      Austin Clements authored
      Currently we only support finding symbols in the VDSO using the old
      DT_HASH. These days everything uses DT_GNU_HASH instead. To keep up
      with the times and future-proof against DT_HASH disappearing from the
      VDSO in the future, this commit adds support for DT_GNU_HASH and
      prefers it over DT_HASH.
      
      Tested by making sure it found a DT_GNU_HASH section and all of the
      expected symbols in it, and then disabling the DT_GNU_HASH path and
      making sure the old DT_HASH path still found all of the symbols.
      
      Fixes #19649.
      
      Change-Id: I508c8b35a019330d2c32f04f3833b69cb2686f13
      Reviewed-on: https://go-review.googlesource.com/45511
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      9065c3bf
    • Audrey Lim's avatar
      archive/zip: fix Writer to validate file · 816deacc
      Audrey Lim authored
      The ZIP format uses uint16 to contain the length of the file name and
      the length of the Extra section. This change verifies that the length
      of these fields fit in an uint16 prior to writing the ZIP file. If not,
      an error is returned.
      
      Fixes #17402
      
      Change-Id: Ief9a864d2fe16b89ddb9917838283b801a2c58a4
      Reviewed-on: https://go-review.googlesource.com/50250Reviewed-by: 's avatarJoe Tsai <thebrokentoaster@gmail.com>
      Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      816deacc
    • Martin Möhrmann's avatar
      strconv: avoid truncation of output in parse int tests · 09ed0f68
      Martin Möhrmann authored
      If needed cast the test table values to a higher bit size
      integer type instead of casting the result values of the
      tested function to a lower bit size integer type.
      
      Change-Id: Iaa79742b2b1d90c7c7eac324f54032ebea0b1b41
      Reviewed-on: https://go-review.googlesource.com/55137Reviewed-by: 's avatarJoe Tsai <thebrokentoaster@gmail.com>
      09ed0f68
    • Martin Möhrmann's avatar
      strings: speed up FieldsFunc · fee7f2ab
      Martin Möhrmann authored
      Increases performance of FieldsFunc by recording the start and end
      of the fields in an array. The first 32 fields are saved in a pre-allocated
      array on the stack. This avoids the old behavior of iterating over the
      input string two times but uses more allocations when more than 32 fields
      are encountered.
      
      Additionally code for handling non-ASCII containing strings from Fields is
      removed and replaced by a call to the new faster FieldsFunc function.
      
      Overall this still leads to a slowdown for Fields on non-ASCII strings
      while speeding up Fields in general.
      
      name                      old time/op    new time/op     delta
      Fields/ASCII/16              116ns ± 5%      115ns ± 5%       ~     (p=0.480 n=10+10)
      Fields/ASCII/256             765ns ± 1%      761ns ± 2%       ~     (p=0.171 n=10+10)
      Fields/ASCII/4096           12.5µs ± 1%     12.7µs ± 1%     +1.82%  (p=0.000 n=10+10)
      Fields/ASCII/65536           226µs ± 1%      226µs ± 2%       ~     (p=0.739 n=10+10)
      Fields/ASCII/1048576        5.12ms ± 1%     5.12ms ± 1%       ~     (p=0.696 n=8+10)
      Fields/Mixed/16              172ns ± 1%      233ns ± 1%    +35.90%  (p=0.000 n=9+10)
      Fields/Mixed/256            1.18µs ± 2%     2.45µs ± 1%   +107.47%  (p=0.000 n=10+10)
      Fields/Mixed/4096           20.3µs ± 1%     43.1µs ± 2%   +112.41%  (p=0.000 n=10+10)
      Fields/Mixed/65536           364µs ± 1%      704µs ± 1%    +93.56%  (p=0.000 n=9+10)
      Fields/Mixed/1048576        7.07ms ± 2%    13.34ms ± 4%    +88.83%  (p=0.000 n=10+10)
      FieldsFunc/ASCII/16          274ns ± 1%      188ns ± 3%    -31.44%  (p=0.000 n=10+10)
      FieldsFunc/ASCII/256        3.69µs ± 1%     2.06µs ± 2%    -44.26%  (p=0.000 n=10+10)
      FieldsFunc/ASCII/4096       59.9µs ± 1%     35.3µs ± 2%    -41.10%  (p=0.000 n=10+10)
      FieldsFunc/ASCII/65536       958µs ± 1%      567µs ± 1%    -40.82%  (p=0.000 n=10+9)
      FieldsFunc/ASCII/1048576    16.3ms ± 2%     11.0ms ± 3%    -32.52%  (p=0.000 n=10+10)
      FieldsFunc/Mixed/16          309ns ± 1%      213ns ± 0%    -30.98%  (p=0.000 n=10+6)
      FieldsFunc/Mixed/256        3.83µs ± 1%     2.14µs ± 1%    -44.01%  (p=0.000 n=10+10)
      FieldsFunc/Mixed/4096       66.2µs ± 2%     37.8µs ± 1%    -42.85%  (p=0.000 n=10+10)
      FieldsFunc/Mixed/65536      1.09ms ± 1%     0.63ms ± 1%    -42.73%  (p=0.000 n=10+10)
      FieldsFunc/Mixed/1048576    18.6ms ± 3%     12.0ms ± 2%    -35.50%  (p=0.000 n=10+10)
      
      Fixes #17856
      Fixes #19789
      
      Change-Id: I9f5a560e534566fd81963651f342c8f44cfb0469
      Reviewed-on: https://go-review.googlesource.com/42810Reviewed-by: 's avatarJoe Tsai <thebrokentoaster@gmail.com>
      fee7f2ab
    • Martin Möhrmann's avatar
      strconv: fix ParseUint return value on range overflow · fc6b74ce
      Martin Möhrmann authored
      If the value corresponding to the input string cannot be
      represented by an unsigned integer of the given size,
      err.Err = ErrRange and the returned value is the maximum
      magnitude unsigned integer of the appropriate bitSize.
      This is consistent with ParseInt's behavior and the documentation.
      
      Expand tests to test 32 bit test value tables with bitsize 32 set.
      These tests fail without the fix in this CL.
      
      Fixes #21278
      
      Change-Id: I8aab39279ec3e31905fcbf582a916cbf6d9b95da
      Reviewed-on: https://go-review.googlesource.com/55134
      Run-TryBot: Martin Möhrmann <moehrmann@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: 's avatarJoe Tsai <thebrokentoaster@gmail.com>
      fc6b74ce
    • Joe Tsai's avatar
      archive/tar: simplify toASCII and parseString · 1d812515
      Joe Tsai authored
      Use a simple []byte instead of bytes.Buffer to create a string.
      Use bytes.IndexByte instead of our own for loop.
      
      Change-Id: Ic4a1161d79017fd3af086a05c53d5f20a5f09326
      Reviewed-on: https://go-review.googlesource.com/54752Reviewed-by: 's avatarAvelino <t@avelino.xxx>
      Reviewed-by: 's avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
      1d812515
  3. 12 Aug, 2017 4 commits
  4. 11 Aug, 2017 13 commits