• Evan Phoenix's avatar
    text/scanner: Fix EOF reporting on strange Readers · 9c037527
    Evan Phoenix authored
    Currently, scanner uses -1 to represent 2 different states:
    
    1. I haven't yet scanned anything, call it "Beginning of File"
    2. I've reached the end of the input, ie EOF
    
    The result of this behavior is that calling Peek() when next()
    has detected the end of the input and set s.ch to scanner.EOF,
    is that Peek() things "oh, s.ch is < 0, which to me means that
    I haven't scanned any next yet, let me try and clear the BOM
    marker."
    
    When this behavior is run on a typical IO, next() will issue
    a Read and get (0, io.EOF) back for the second time without
    blocking and Peek() will return scanner.EOF.
    
    The bug comes into play when, inside a terminal, hitting Control-D.
    This causes the terminal to return a EOF condition to the reader
    but it does not actually close the fd.
    
    So, combining these 2 situations, we arrive at the bug:
    
    What is expected: hitting Control-D in a terminal will make Peek()
    return scanner.EOF instantly.
    
    What actually happens:
    
    0. Code waiting in Next()
    1. User hits Control-D
    2. fd returns EOF condition
    3. EOF bubbles it's way out to line 249 in scanner.go
    4. next() returns scanner.EOF
    5. Next() saves the scanner.EOF to s.ch and returns the previous value
    6. Peek() runs, sees s.ch < 0, mistakenly thinks it hasn't run yet and
       tries to read the BOM marker.
    7. next() sees the buffer is empty and tries to fill it again, blocking
       on line 249.
    
    The fix is simple: use a different code to indicate that no data
    has been scanned.
    
    Change-Id: Iee8f4da5881682c4d4c36b93b9bf397ac5798179
    Reviewed-on: https://go-review.googlesource.com/7913Reviewed-by: 's avatarRobert Griesemer <gri@golang.org>
    9c037527
scanner_test.go 16.5 KB