• Matthew Dempsky's avatar
    cmd/compile: add indexed export format · ca2f85fd
    Matthew Dempsky authored
    This CL introduces a new indexed data format for package export
    data. This improves on the previous (sequential) binary format by
    allowing the compiler to selectively (and lazily) load only the data
    that's actually needed for compilation.
    
    In large Go projects, the package export data can become very large
    due to transitive type declaration dependencies and inline
    function/method bodies. By lazily loading these declarations and
    bodies as needed, we avoid wasting time and memory processing
    unnecessary and/or redundant data.
    
    In the benchmarks below, "old" is -iexport=false and "new" is
    -iexport=true. The suffixes indicate the compiler concurrency (-c) and
    inlining (-l) settings used for the build (using -gcflags=all=-foo).
    Benchmarks were run on an HP Z620.
    
    Juju is "go build -a github.com/juju/juju/cmd/...":
    
    name          old real-time/op  new real-time/op  delta
    Juju/c=1/l=0        44.0s ± 1%        38.7s ± 9%  -11.97%  (p=0.001 n=7+7)
    Juju/c=1/l=4        53.7s ± 3%        45.3s ± 4%  -15.53%  (p=0.001 n=7+7)
    Juju/c=4/l=0        39.7s ± 8%        32.0s ± 4%  -19.38%  (p=0.001 n=7+7)
    Juju/c=4/l=4        46.3s ± 4%        38.0s ± 4%  -18.06%  (p=0.001 n=7+7)
    
    name          old user-time/op  new user-time/op  delta
    Juju/c=1/l=0         371s ± 1%         300s ± 0%  -19.07%  (p=0.001 n=7+6)
    Juju/c=1/l=4         482s ± 0%         374s ± 1%  -22.37%  (p=0.001 n=7+7)
    Juju/c=4/l=0         410s ± 1%         340s ± 1%  -17.19%  (p=0.001 n=7+7)
    Juju/c=4/l=4         532s ± 1%         424s ± 1%  -20.26%  (p=0.001 n=7+7)
    
    name          old sys-time/op   new sys-time/op   delta
    Juju/c=1/l=0        33.4s ± 1%        28.4s ± 2%  -15.02%  (p=0.001 n=7+7)
    Juju/c=1/l=4        40.7s ± 2%        32.8s ± 3%  -19.51%  (p=0.001 n=7+7)
    Juju/c=4/l=0        39.8s ± 2%        34.4s ± 2%  -13.74%  (p=0.001 n=7+7)
    Juju/c=4/l=4        48.4s ± 2%        40.4s ± 2%  -16.50%  (p=0.001 n=7+7)
    
    Kubelet is "go build -a k8s.io/kubernetes/cmd/kubelet":
    
    name             old real-time/op  new real-time/op  delta
    Kubelet/c=1/l=0        42.0s ± 1%        34.8s ± 1%  -17.27%  (p=0.008 n=5+5)
    Kubelet/c=1/l=4        55.4s ± 3%        45.4s ± 3%  -18.06%  (p=0.002 n=6+6)
    Kubelet/c=4/l=0        37.4s ± 3%        29.9s ± 1%  -20.25%  (p=0.004 n=6+5)
    Kubelet/c=4/l=4        48.1s ± 2%        39.0s ± 5%  -18.93%  (p=0.002 n=6+6)
    
    name             old user-time/op  new user-time/op  delta
    Kubelet/c=1/l=0         291s ± 1%         233s ± 1%  -19.96%  (p=0.002 n=6+6)
    Kubelet/c=1/l=4         385s ± 1%         298s ± 1%  -22.51%  (p=0.002 n=6+6)
    Kubelet/c=4/l=0         325s ± 0%         268s ± 1%  -17.48%  (p=0.004 n=5+6)
    Kubelet/c=4/l=4         429s ± 1%         343s ± 1%  -20.08%  (p=0.002 n=6+6)
    
    name             old sys-time/op   new sys-time/op   delta
    Kubelet/c=1/l=0        25.1s ± 2%        20.9s ± 4%  -16.69%  (p=0.002 n=6+6)
    Kubelet/c=1/l=4        31.2s ± 3%        24.4s ± 0%  -21.67%  (p=0.010 n=6+4)
    Kubelet/c=4/l=0        30.2s ± 2%        25.6s ± 1%  -15.34%  (p=0.002 n=6+6)
    Kubelet/c=4/l=4        37.3s ± 1%        30.9s ± 2%  -17.11%  (p=0.002 n=6+6)
    
    Change-Id: Ie43eb3bbe1392cbb61c86792a17a57b33b9561f0
    Reviewed-on: https://go-review.googlesource.com/106796
    Run-TryBot: Matthew Dempsky <mdempsky@google.com>
    TryBot-Result: Gobot Gobot <gobot@golang.org>
    Reviewed-by: 's avatarRobert Griesemer <gri@golang.org>
    ca2f85fd
export.go 5.63 KB