clarifying when approximations are used by effectsize

[IndrajeetPatil]

Since report calls effectsize on model objects to compute effect sizes, effectsize often needs to use an approximation in case insight::get_data() doesn't work.

I think the report output should somehow note this for the user, lest they get confused as to why the values don't match with other package outputs (including effectsize).

For example, #183 (comment)

[mattansb]

effectsize gives a warning, but this is suppressed by report... :/

[DominiqueMakowski]

If not suppressed, can it be muted?

[bwiernik]

We should use tryCatch() to catch the warning and then adjust the report output accordingly.

[leighclark]

If I might be so bold as to make a suggestion: if an error message does get generated in the future, it might be useful to display the "correct" format for getting the "right" results as part of the message, as @mattansb posted in the other thread.

[IndrajeetPatil]

Thinking out loud here:

How about-

• effectsize outputs get a new attribute called approximate with a default value of FALSE
• when insight::get_data() fails, and an approximation is used, this attribute value will change to TRUE
• when report sees that this attribute is TRUE, it will add "(approximate)" to whatever effect size is being reported

For example, here it would change to

Cohen's d (approximate) = 2.03, 95% CI [1.13, 2.91]

[leighclark]

For example, here it would change to

Cohen's d (approximate) = 2.03, 95% CI [1.13, 2.91]

That would be great, and maybe (instead of what I suggested above) it could describe how to "correct" the approximation in the docs or in an example or something :)

[mattansb]

how to "correct" the approximation in the docs or in an example or something :)

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

[leighclark]

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

You are operating on the assumption that novices like me know what "formula notation" is ;)

[bwiernik]

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

Or better, don't use htest functions, just fit a linear model 😜

[mattansb]

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

The formula notation for *.test() functions is when you use a formula to let the function know the structure of you data (:

t.test(mpg ~ am, data = mtcars) # formula

t.test(mtcars$mpg[mtcars$am==0], mtcars$mpg[mtcars$am==1]) # pass values

[bwiernik]

But really @leighclark Don't use the *.test functions. Use lm() instead. https://lindeloev.github.io/tests-as-linear/

[mattansb]

Hmm this is good advice maybe for inference in general - but report will treat it like a linear model (so no Cohen's d...)

[leighclark]

But really @leighclark Don't use the *.test functions. Use lm() instead. https://lindeloev.github.io/tests-as-linear/

hey @bwiernik I think (know) we are at totally different stages of our psychology and statistics careers, so I am still at the stage where I have to do the things that are given to me, but thanks for the tip and I will take a look at it!

Cheers,
Leigh

[bwiernik]

@mattansb easystats/effectsize#351

[LukasWallrich]

This would be a great addition to the docs / examples ... I just needed to find this issue to figure out how to report t.tests

how to "correct" the approximation in the docs or in an example or something :)

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

[mattansb]

Results from effectsize() should have an approximate attribute that can be used.