• Justin Nuß's avatar
    encoding/csv: avoid allocations when reading records · bd06d482
    Justin Nuß authored
    This commit changes parseRecord to allocate a single string per record,
    instead of per field, by using indexes into the raw record.
    
    Benchstat (done with f69991c1)
    
    name                          old time/op    new time/op    delta
    Read-8                          3.17µs ± 0%    2.78µs ± 1%  -12.35%  (p=0.016 n=4+5)
    ReadWithFieldsPerRecord-8       3.18µs ± 1%    2.79µs ± 1%  -12.23%  (p=0.008 n=5+5)
    ReadWithoutFieldsPerRecord-8    4.59µs ± 0%    2.77µs ± 0%  -39.58%  (p=0.016 n=4+5)
    ReadLargeFields-8               57.0µs ± 0%    55.7µs ± 0%   -2.18%  (p=0.008 n=5+5)
    
    name                          old alloc/op   new alloc/op   delta
    Read-8                            660B ± 0%      664B ± 0%   +0.61%  (p=0.008 n=5+5)
    ReadWithFieldsPerRecord-8         660B ± 0%      664B ± 0%   +0.61%  (p=0.008 n=5+5)
    ReadWithoutFieldsPerRecord-8    1.14kB ± 0%    0.66kB ± 0%  -41.75%  (p=0.008 n=5+5)
    ReadLargeFields-8               3.86kB ± 0%    3.94kB ± 0%   +1.86%  (p=0.008 n=5+5)
    
    name                          old allocs/op  new allocs/op  delta
    Read-8                            30.0 ± 0%      18.0 ± 0%  -40.00%  (p=0.008 n=5+5)
    ReadWithFieldsPerRecord-8         30.0 ± 0%      18.0 ± 0%  -40.00%  (p=0.008 n=5+5)
    ReadWithoutFieldsPerRecord-8      50.0 ± 0%      18.0 ± 0%  -64.00%  (p=0.008 n=5+5)
    ReadLargeFields-8                 66.0 ± 0%      24.0 ± 0%  -63.64%  (p=0.008 n=5+5)
    
    For a simple application that I wrote, which reads in a CSV file (via
    ReadAll) and outputs the number of rows read (15857625 rows), this change
    reduces the total time on my notebook from ~58 seconds to ~48 seconds.
    
    This reduces time and allocations (bytes) each by ~6% for a real world
    CSV file at work (~230000 rows, 13 colums).
    
    Updates #16791
    
    Change-Id: Ia07177c94624e55cdd3064a7d2751fb69322d3e4
    Reviewed-on: https://go-review.googlesource.com/24723Reviewed-by: 's avatarBrad Fitzpatrick <bradfitz@golang.org>
    Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
    TryBot-Result: Gobot Gobot <gobot@golang.org>
    bd06d482
Name
Last commit
Last update
.github Loading commit data...
api Loading commit data...
doc Loading commit data...
lib/time Loading commit data...
misc Loading commit data...
src Loading commit data...
test Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
AUTHORS Loading commit data...
CONTRIBUTING.md Loading commit data...
CONTRIBUTORS Loading commit data...
LICENSE Loading commit data...
PATENTS Loading commit data...
README.md Loading commit data...
favicon.ico Loading commit data...
robots.txt Loading commit data...