Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqrt of integers returns a double NaN #24

Open
ltratt opened this issue Oct 27, 2019 · 1 comment
Open

sqrt of integers returns a double NaN #24

ltratt opened this issue Oct 27, 2019 · 1 comment
Labels
spec Needs specification, or is specification related.

Comments

@ltratt
Copy link

ltratt commented Oct 27, 2019

This program:

sqrter = (
    run = (
        -1 sqrt println.
    )
)

prints NaN which was a little surprising to me -- I wouldn't have expected a conversion to a double in such a case. What we've done in softdevteam/yksom#57 is to throw an error in such cases. Admittedly, on reflection, both the NaN and "thrown an error" behaviour are defensible, though I suspect the latter catches more genuine programmer errors. Maybe something to think about for the SOM spec!

@smarr smarr added the spec Needs specification, or is specification related. label Oct 27, 2019
@smarr
Copy link
Member

smarr commented Oct 27, 2019

Hi @ltratt:

Smalltalks classically try to move between numerical representations for such things.
I am not sure I like that. One problem is that it violates the typing dimension, and makes it value dependent. This leads to surprising results.

---> 4 sqrt
it = 2
---> 4 sqrt class
it = Integer
---> 4.0 sqrt class
it = Double
---> 1 sqrt class
it = Integer
---> 3 sqrt class
it = Double
---> 10 sqrt class
it = Double
---> -1 sqrt class
it = Double

The above behavior (note the 4, 1, and 3 case), is consistent with Pharo. Except for -1 raising an exception.

So, the conversion to a Double and NaN isn't surprising to me and seem consistent with the other behaviors.
Though, I agree that an error as in yksom or Pharo is likely the better result.

I did see the PR earlier, but didn't have an opinion on it before.
Is yksom consistent with sqrt results for 4, 1, and 3?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
spec Needs specification, or is specification related.
Projects
None yet
Development

No branches or pull requests

2 participants