-
Notifications
You must be signed in to change notification settings - Fork 76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
yaml_test: adapt around undefined behavior in float64->int64 casting #42
yaml_test: adapt around undefined behavior in float64->int64 casting #42
Conversation
In TestJSONObjectToYAMLObject the "uint64 big" case accepts a float64 of size 2^63. The value is passed to jsonToYAMLValue() and the float64 branch of the switch is entered. For values that do not fit int64 the first cast to int64() is undefined behavior in most languages and apparently in Golang: http://c0x.coding-guidelines.com/6.3.1.4.pdf This means the value of int64(float64(2^63)) can end up as either -9223372036854775808 or 9223372036854775807. From experimentation, it appears compiler optimization determines the result. The value is then casted back to float64 and matched against the original float64, which may or may not pass. Depending of FPU rounding mode for IEEE754-Doubles any input above 9223372036854775296 may get casted to 9223372036854775808.0. Adapt the unit test for "uint64 big" to not feed a big float64 and enter the problem cases. Instead pass the nearest rounded value of 2^63 as a uint64 and expect to receive the same result: bigUint64 = ((uint64(1) << 63) + 500) / 1000 * 1000
while jsonToYAMLValue is doing something that is not recommended, it appears to match existing logic in https://github.com/go-yaml/yaml, so i left it untouched. /kind bug |
tested this on a real arm64 box:
|
cpu info
|
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dims, neolit123 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
In TestJSONObjectToYAMLObject the "uint64 big" case accepts a float64
of size 2^63. The value is passed to jsonToYAMLValue() and the
float64 branch of the switch is entered. For values that do not
fit int64 the first cast to int64() is undefined behavior
in most languages and apparently in Golang:
http://c0x.coding-guidelines.com/6.3.1.4.pdf
(link for the C standard)
This means the value of int64(float64(2^63)) can end up as either
-9223372036854775808 or 9223372036854775807.
From experimentation, it appears compiler optimization
determines the result.
The value is then casted back to float64 and matched against
the original float64, which may or may not pass. Depending of FPU
rounding mode for IEEE754-Doubles any input above 9223372036854775296
may get casted to 9223372036854775808.0.
Adapt the unit test for "uint64 big" to not feed a big float64 and enter
the problem cases. Instead pass the nearest rounded value of 2^63
as a uint64 and expect to receive the same result:
bigUint64 = ((uint64(1) << 63) + 500) / 1000 * 1000
the observed undefined behavior seems to trip tests on non-AMD64:
fixes #34