Skip to content

N
net
Switch branch/tag
  1. 24 May, 2017 8 commits
  3. 23 May, 2017 1 commit
  5. 22 May, 2017 3 commits
  7. 17 May, 2017 1 commit
  9. 15 May, 2017 1 commit
    • Brad Fitzpatrick's avatar
      http2: reduce the number of select cases in serverConn.server · 34057069
      Brad Fitzpatrick authored 
      This drops the number of select cases in serverConn.server from 10 to
6. It was 7 in Go 1.7. It increased to 10 in Go 1.8.

* replace testing-only testHookCh with always-on grab-bag serveMsgCh
  to be used for both test purposes, and infrequently used message
  types

* remove the settingsTimer.C case for the initial settings timer
  and use serveMsgCh and time.AfterFunc instead

* ... and do the same for the idle timeout timer.

* ... and for the shutdown timer.

* remove wantStartPushCh and just send the *startPushRequest to
  serveMsgCh too

I could go further with this (and plan to, later), but these are the
safe and easy ones that don't require more work elsewhere.

The speed gets better the more the request/response go via the
serverConn.serve loop. (once per Request.Body.Read or
ResponseWriter.Flush):

  name            old time/op    new time/op  delta
  ServerGets-4    138µs ± 3%     134µs ± 2%   -2.54%   (p=0.002 n=10+10)
  ServerPosts-4   176µs ±27%     154µs ± 2%   -12.62%  (p=0.011 n=10+10)

Updates kubernetes/kubernetes#45216
Updates golang/go#20302

Change-Id: I18019554089d7e3d76355d7137b5957e9597e803
Reviewed-on: https://go-review.googlesource.com/43034
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarTom Bergan <tombergan@google.com>
      34057069
  11. 13 May, 2017 1 commit
    • Tom Bergan's avatar
      http2: fix lock contention slowdown due to gracefulShutdownCh · 84f0e6f9
      Tom Bergan authored 
      gracefulShutdownCh is shared by all connections in a server. When a
server accumulates many connections (e.g., 5000 in the kubemark-5000
benchmark), we have 5000 serverConn.serve goroutines selecting on this
channel. This means 5000 goroutines hammer the channel's lock, which
causes severe lock contention.

The fix in this CL is to make a local proxy for gracefulShutdownCh in
each connection so that each connection selects on gracefulShutdownCh
at most once per connection rather than once per serverConn.serve loop
iteration.

This fix is intended to be backported quickly into Go 1.8.2. The
downside of this fix is 2KB extra stack usage per connection. A better
fix will be implemented in Go 1.9.

Unfortunately, I have been unable to reproduce this problem locally.
This fix was verified by the kubernetes team. See:
https://github.com/kubernetes/kubernetes/issues/45216#issuecomment-300830243

Updates golang/go#20302

Change-Id: I19ab19268a6ccab9b6e9dffa0cfbc89b8c7d0f19
Reviewed-on: https://go-review.googlesource.com/43455
Run-TryBot: Tom Bergan <tombergan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      84f0e6f9
  13. 10 May, 2017 1 commit
    • Todd Neal's avatar
      http2/hpack: move initialization to a static table · c9b681d3
      Todd Neal authored 
      This shrinks a binary that just does an http.ListenAndServe by about
90kb due to using less code for the static array.

 syms
 delta   name                                                old-size  new-size  pct-difference
-54206   vendor/golang_org/x/net/http2/hpack.newStaticTable  55233     1027      -98.14%
-4744    runtime.pclntab                                     1041055   1036311   -0.46%
-204     runtime.findfunctab                                 10675     10471     -1.91%
8        runtime.typelink                                    9852      9860      0.08%
41       runtime.gcbss                                       869       910       4.72%
11711    vendor/golang_org/x/net/http2/hpack.init            572       12283     2047.38%

 sections
 delta   name             old-size  new-size  pct-difference
-41888   .text            2185840   2143952   -1.92%
-37644   .rodata          842131    804487    -4.47%
-4744    .gopclntab       1041055   1036311   -0.46%
-3343    .debug_info      981995    978652    -0.34%
-2931    .debug_line      291295    288364    -1.01%
8        .typelink        9852      9860      0.08%
59       .debug_pubnames  81986     82045     0.07%
96       .symtab          186312    186408    0.05%
113      .debug_pubtypes  137500    137613    0.08%
128      .debug_frame     219140    219268    0.06%
220      .strtab          217109    217329    0.10%
2464     .bss             127752    130216    1.93%

Updates golang/go#6853

Change-Id: I3383e63300585539507b75faac1072264d8f37e7
Reviewed-on: https://go-review.googlesource.com/43090Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
      c9b681d3
  15. 09 May, 2017 1 commit
  17. 03 May, 2017 1 commit
  19. 02 May, 2017 1 commit
    • Martin Probst's avatar
      xsrftoken: panic for unsafe zero length keys · 0819898f
      Martin Probst authored 
      Passing a zero length key (or secret) gives no safety against XSRF
attacks. This is a relatively easy mistake to make, e.g. by passing
`make([]byte, 0, 1024)` to `rand.Read` instead of `make([]byte, 1024)`,
and currently fails open, silently.

This uses panic, as the API does not allow returning a structured error,
and catching this programming error is not worth breaking API
compatibility. Passing a zero length secret is also not an error
condition that API callers would handle, so there is little value in
returning a proper error.

Change-Id: Ib6457347675872188d51d2a220eee4b67900f79e
Reviewed-on: https://go-review.googlesource.com/42411Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
      0819898f
  21. 24 Apr, 2017 2 commits
  23. 23 Apr, 2017 1 commit
  25. 21 Apr, 2017 2 commits
  27. 13 Apr, 2017 1 commit
  29. 06 Apr, 2017 1 commit
  31. 29 Mar, 2017 2 commits
  33. 28 Mar, 2017 2 commits
  35. 24 Mar, 2017 2 commits
  37. 23 Mar, 2017 1 commit
  39. 08 Mar, 2017 2 commits
  41. 03 Mar, 2017 1 commit
    • Ian Gudger's avatar
      dns/dnsmessage: add support for parsing and packing of DNS messages · d379faa2
      Ian Gudger authored 
      The Go standard library contains support for packing and unpacking of
DNS messages, but it is not exported, doesn't follow Go style, and is
not very well optimized. Low level DNS functionality is clearly useful
to the Go community as evidenced by the success of
github.com/miekg/dns. This implementation endeavors to avoid the
limitations of both the standard library and github.com/miekg/dns
implementations and is an almost complete rewrite of the code
currently found in on net/dnsmsg.go and net/dnsmsg_test.go.

Goals:
* Minimize heap allocations.
* Allow parsing only what is needed. Avoid unnecessary parsing and
  heap allocations for parts of the message that you don't care about.
  Parsing should be allowed on as small of a granularity as is useful,
  but no smaller as to avoid complicating the interface.
* Parse and pack each byte of the message at most one time.

Updates golang/go#16218
Updates golang/go#10622

Change-Id: Ib754d0007609a617d88be867f21c2feb15b6fcd7
Reviewed-on: https://go-review.googlesource.com/35237
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarMikio Hara <mikioh.mikioh@gmail.com>
      d379faa2
  43. 27 Feb, 2017 3 commits
    • Tom Bergan's avatar
      http2: add configurable knobs for the server's receive window · 906cda95
      Tom Bergan authored 
      Upload performance is poor when BDP is higher than the flow-control window.
Previously, the server's receive window was fixed at 64KB, which resulted in
very poor performance for high-BDP links. The receive window now defaults to
1MB and is configurable. The per-connection and per-stream windows are
configurable separately (both default to 1MB as suggested in golang/go#16512).

Previously, the server created a "fixedBuffer" for each request body. This is no
longer a good idea because a fixedBuffer has fixed size, which means individual
streams cannot use varying amounts of the available connection window. To
overcome this limitation, I replaced fixedBuffer with "dataBuffer", which grows
and shrinks based on current usage. The worst-case fragmentation of dataBuffer
is 32KB wasted memory per stream, but I expect that worst-case will be rare.

A slightly modified version of adg@'s grpcbench program shows a dramatic
improvement when increasing from a 64KB window to a 1MB window, especially at
higher latencies (i.e., higher BDPs). Network latency was simulated with netem,
e.g., `tc qdisc add dev lo root netem delay 16ms`.

Duration        Latency Proto           H2 Window

11ms±4.05ms     0s      HTTP/1.1        -
17ms±1.95ms     0s      HTTP/2.0        65535
8ms±1.75ms      0s      HTTP/2.0        1048576

10ms±1.49ms     1ms     HTTP/1.1        -
47ms±2.91ms     1ms     HTTP/2.0        65535
10ms±1.77ms     1ms     HTTP/2.0        1048576

15ms±1.69ms     2ms     HTTP/1.1        -
88ms±11.29ms    2ms     HTTP/2.0        65535
15ms±1.18ms     2ms     HTTP/2.0        1048576

23ms±1.42ms     4ms     HTTP/1.1        -
152ms±0.77ms    4ms     HTTP/2.0        65535
23ms±0.94ms     4ms     HTTP/2.0        1048576

40ms±1.54ms     8ms     HTTP/1.1        -
288ms±1.67ms    8ms     HTTP/2.0        65535
39ms±1.29ms     8ms     HTTP/2.0        1048576

72ms±1.13ms     16ms    HTTP/1.1        -
559ms±0.68ms    16ms    HTTP/2.0        65535
71ms±1.12ms     16ms    HTTP/2.0        1048576

136ms±1.15ms    32ms    HTTP/1.1        -
1104ms±1.62ms   32ms    HTTP/2.0        65535
135ms±0.96ms    32ms    HTTP/2.0        1048576

264ms±0.95ms    64ms    HTTP/1.1        -
2191ms±2.08ms   64ms    HTTP/2.0        65535
263ms±1.57ms    64ms    HTTP/2.0        1048576

Fixes golang/go#16512
Updates golang/go#17985
Updates golang/go#18404

Change-Id: Ied385aa94588337e98dad9475cf2ece2f39ba346
Reviewed-on: https://go-review.googlesource.com/37226Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
      906cda95
    • Tom Bergan's avatar
      http2/hpack: speedup Encoder.searchTable · bce15e71
      Tom Bergan authored 
      The previous algorithm was linear in the size of the table.
The new algorithm is O(1). The new algorithm should behave exactly
like the old algorthm, except for one unimportant change that
triggered small updates to tests in encode_test.go:

When encoding "Field: Value" where the table has two entries,
[0]={"Field", "X"} and [1]={"Field", "Y"}, we can encode the field
name using either table entry. Previously, we selected the oldest
entry, but now we select the newest entry. The new implementation
should actually generate very slightly better compression because
new entries are encoded with smaller integers than old entries, and
HPACK uses a varint encoding for integers where smaller integers
are encoded in fewer bytes.

I added a synthetic microbenchmark which shows a big speedup in
hpack.Encoder.searchTable:

BenchmarkEncoderSearchTable-40  100000 127440 ns/op  # before
BenchmarkEncoderSearchTable-40   50000  25121 ns/op  # after

Change-Id: Ib87d61b6415d9f0ff38874fe2a719b2f00351590
Reviewed-on: https://go-review.googlesource.com/37406Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      bce15e71
    • Alexander Polcyn's avatar
      http2: Add opt-in option to Framer to allow DataFrame struct reuse · bb807669
      Alexander Polcyn authored 
      The existing Framer in net/http2 allocates a new DataFrame struct
for each DataFrame read on calls to ReadFrame. The SetReuseFrame
option introduced here, if set on a Framer, allows the
Framer to reuse Frame objects and changes the ReadFrame API
so that returned Frame objects are only valid until the next call
to ReadFrame. This opt-in API now only implements reuse of DataFrames,
but it allows the Framer to reuse of any type of Frame.

The footprint caused by creation of new DataFrame structs per data
frame was noticed in micro benchmarks of "gRPC" server "streaming
throuhgput", which uses the Framer in this package. This benchmark
happened to use long lived http2 streams that do client-server "ping-pong"
requests with small data frames, and DataFrames were seen to be a
significant source of allocations.

Running local benchmarks with: (from x/net/http2 directory)

$ go test -run=^$ -bench=BenchmarkServerToClientStream

example output:
* expect an alloc reduction of at least 1 and a small memory reduction between
"BenchmarkServerToClientStreamDefaultOptions" and
"BenchmarkServerToClientStreamReuseFrames"

BenchmarkServerToClientStreamDefaultOptions-12    	   30000
46216 ns/op	     971 B/op	      17 allocs/op
BenchmarkServerToClientStreamReuseFrames-12       	   30000
44952 ns/op	     924 B/op	      16 allocs/op

Fixes golang/go#18502

Change-Id: Iad93420ef6c3918f54249d867098f1dadfa324d8
Reviewed-on: https://go-review.googlesource.com/34812
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      bb807669
  45. 24 Feb, 2017 1 commit
    • Tom Bergan's avatar
      http2: replace fixedBuffer with dataBuffer · 10c134ea
      Tom Bergan authored 
      fixedBuffer was a bad idea for two reasons:

1. It was fixed at a constant 64KB (the current default flow-control
   window) which wastes memory on the server when clients upload many
   small request bodies.

2. A follow-up CL will allow configuring the server's connection and
   stream receive windows. We want to allow individual streams to use
   varying amounts of the available connection window. This is not
   possible when each stream uses a fixedBuffer.

dataBuffer grows and shrinks based on current usage. The worst-case
fragmentation of dataBuffer is 32KB wasted memory per stream, but I
expect that worst-case will be rare. In particular, if the declared
size of a stream's request body is under 1KB, then the server will not
allocate more than 1KB to process that stream's request body.

Updates golang/go#16512
Fixes golang/go#18509

Change-Id: Ibcb18007037e82518a65848ef3baf4937955ac9d
Reviewed-on: https://go-review.googlesource.com/37400
Run-TryBot: Tom Bergan <tombergan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
      10c134ea