• Charles L. Dorian's avatar
    math: improve Atan, Asin and Acos accuracy · 35c3afdb
    Charles L. Dorian authored
    pkg/math/all_test.go tests Atan (and therefore Asin and Acos) to a
    relative accuracy of 4e-16, but the test vector misses values where
    the old algorithm was in error by more than that. For example:
    
    x            newError   oldError
    0.414215746  1.41e-16  -4.24e-16
    0.414216076  1.41e-16  -4.24e-16
    0.414217632  1.41e-16  -4.24e-16
    0.414218770  1.41e-16  -4.24e-16
    0.414225466  0         -5.65e-16
    0.414226244  1.41e-16  -4.24e-16
    0.414228756  0         -5.65e-16
    0.414235089  0         -5.65e-16
    0.414237070  0         -5.65e-16
    
    R=rsc, golang-dev
    CC=golang-dev
    https://golang.org/cl/6302093
    35c3afdb
Name
Last commit
Last update
api Loading commit data...
doc Loading commit data...
include Loading commit data...
lib Loading commit data...
misc Loading commit data...
src Loading commit data...
test Loading commit data...
.hgignore Loading commit data...
.hgtags Loading commit data...
AUTHORS Loading commit data...
CONTRIBUTORS Loading commit data...
LICENSE Loading commit data...
PATENTS Loading commit data...
README Loading commit data...
favicon.ico Loading commit data...
robots.txt Loading commit data...