Cohen's d effect size in a Bayesian context?

[cordphelps]

Is there a way to compare two different groups of evidence to answer the question "is it likely that these two samples come from different populations?" (similar to the Cohen's d effect size)? Thank you.

[mattansb]

Do you mean something like BayesFactor::ttestBF()?

[cordphelps]

hmmm, maybe. Of course it would be nice to have the option to specify priors other than the gaussian. I was not imagining that computing the posterior would be necessary to make the comparison, but perhaps it is. Is it appropriate to compare evidence based on a modes (like MAP and median) beyond just the simple mean ?

[mattansb]

The statistical model behind Cohen's d is the difference between the means in the populations.
You can estimate the MAP of the difference between the means. But if the model you wish to test is the difference between anything else other than the means, then Cohen's d is not the right measure for you, and you will have to specify your own model (in stan / greta / winBUGS, etc...).

[cordphelps]

thank you for your patience with my limited stats background. In my research context, I hypothesize a model and a prior distribution. I then collect two independent sets of evidence.

Ignoring the hypotheses, what can I say about the similarity of these two sets of evidence?

[DominiqueMakowski]

"is it likely that these two samples come from different populations?" (similar to the Cohen's d effect size)

I'm not sure this is what Cohen's d tells you 🙃 rather, it is an absolute measure of magnitude of the difference (i.e., it is a difference between means expressed in a standardized unit)

Bayes factors and other indices mostly aim at being an alternative to hypothesis testing indices (such as the p-value). But this is conceptually independent from the effect size: a difference can be significant, but tiny, or super large, and not significant (even though in practice this is often correlated). So long story short no, there is currently no "purely Bayesian" index of effect size (and I'm not sure it would even make sense, as effect size quantification is an issue orthogonal to the whole "Bayesian vs frequentist" axis).

[cordphelps]

thanks Dominique; I'm trying to mark that as "answered" but I can't see how do do that....

[DominiqueMakowski]

I don't know 🤷‍♂️ let's just leave it as is for now 😅