ArviZ.jl

[kiante-fernandez]

@itsdfish or @DominiqueMakowski
Have either of you tried using ArviZ.jl yet? I am writing up a simple model comparison example for the talk and thought it would be nice to use it rather than ParetoSmooth.jl (to show it a workflow example).

https://julia.arviz.org/ArviZ/stable/quickstart/

[DominiqueMakowski]

For some reason I've not been fully convinced by it (yet). Not that their authors didn't do a great job but I just didn't really have the need for it yet, it seems slightly too Pythonic (although that's to be expected). ^{_{Also, I'm don't like the package name...}}

I I think I read that there is a plan to split it up into even smaller packages, which kind of grinds my gear (I feel like I'm repeating myself but I'm really annoyed by having to load a thousand small packages in Julia to run one function)

[kiante-fernandez]

Yeah, I have found it cumbersome as well. I thought it would be nice because I know folks developing cognitive models in PyMC with the intention to use ArviZ. So I thought the parallel would be nice. But I could not even get a simple thing to work out of the box like this:

using Random
using SequentialSamplingModels
using Turing
using LinearAlgebra

using ArviZ
using ArviZPythonPlots

Random.seed!(5000)

dist = LBA(ν=[3.0, 2.0], A=0.8, k=0.3, τ=0.3)
data = rand(dist, 1000)

@model function model_lba1(data; min_rt=0.2)
    # Priors
    ν ~ MvNormal(fill(3.0, 2), I * 3)
    A ~ truncated(Normal(0.8, 0.4), 0.0, Inf)
    τ ~ Uniform(0.0, min_rt)

    # Likelihood
    for i in 1:length(data)
        data[i] ~ LBA(; ν, A, k = 0.3, τ) #correct 
    end
end

dat = [(choice=data.choice[i], rt=data.rt[i]) for i in 1:length(data.rt)]

chain_lba1 = sample(model_lba1(dat, min_rt=minimum(data.rt)), NUTS(), 1000)

param_mod = model_lba1(dat, min_rt=minimum(data.rt))
log_likelihood = Turing.pointwise_loglikelihoods(param_mod, MCMCChains.get_sections(chain_lba1, :parameters))

idata1 = from_mcmcchains(
    chain_lba1;
    log_likelihood,
    library=Turing
)

ArviZ.loo(idata1)

ERROR: ArgumentError: `var_name` must be specified if there are multiple variables in group
Stacktrace:
 [1] log_likelihood(log_like::Dataset{@NamedTuple{…}, DimensionalData.DimStack{…}}, var_name::Nothing)
   @ InferenceObjectsPosteriorStatsExt ~/.julia/packages/InferenceObjects/DI3dl/ext/InferenceObjectsPosteriorStatsExt/utils.jl:35
 [2] log_likelihood(data::InferenceData{…}, var_name::Nothing)
   @ InferenceObjectsPosteriorStatsExt ~/.julia/packages/InferenceObjects/DI3dl/ext/InferenceObjectsPosteriorStatsExt/utils.jl:17
 [3] loo(data::InferenceData{…}; var_name::Nothing, kwargs::@Kwargs{})
   @ InferenceObjectsPosteriorStatsExt ~/.julia/packages/InferenceObjects/DI3dl/ext/InferenceObjectsPosteriorStatsExt/loo.jl:35
 [4] loo(data::InferenceData{(:posterior, :log_likelihood, :sample_stats, :prior, :sample_stats_prior), Tuple{…}})
   @ InferenceObjectsPosteriorStatsExt ~/.julia/packages/InferenceObjects/DI3dl/ext/InferenceObjectsPosteriorStatsExt/loo.jl:30
 [5] top-level scope
   @ REPL[32]:1

[itsdfish]

I have not used ArviZ much, but the primary maintainer is helpful. My guess is that you need to specify the data name: var_name = :data), but I am not positive. If that doesn't work, I recommend asking via Discourse or their preferred mode of communication.

[DominiqueMakowski]

And it's not helped by the fact that Julia doesn't enforce (and actually encourages the opposite) the clear bulletpoint-like documentation of all the arguments of a function. It makes it super hard to find anything... rant over