Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stDDM to SequentialSamplingModels #59

Merged
merged 11 commits into from
Mar 17, 2024

Conversation

kiante-fernandez
Copy link
Contributor

Iinked to dicssion in #41

Since the model sees extensive use, I figured an accessible simulator could be useful. I still need to write the doc page and test for the model but here is the PR to get us started.

@codecov-commenter
Copy link

codecov-commenter commented Feb 27, 2024

Codecov Report

Attention: Patch coverage is 94.20290% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 90.20%. Comparing base (886b18f) to head (13742ba).
Report is 8 commits behind head on master.

Files Patch % Lines
src/stDDM.jl 95.16% 3 Missing ⚠️
src/type_system.jl 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #59      +/-   ##
==========================================
+ Coverage   89.97%   90.20%   +0.22%     
==========================================
  Files          23       24       +1     
  Lines        1417     1480      +63     
==========================================
+ Hits         1275     1335      +60     
- Misses        142      145       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Feb 27, 2024

Benchmark Results

master 13742ba... master/13742ba01ae4f7...
logpdf/("SequentialSamplingModels.DDM", 10) 1.67 ± 0.12 μs 1.67 ± 0.14 μs 1
logpdf/("SequentialSamplingModels.DDM", 100) 17.3 ± 1.3 μs 17.2 ± 0.78 μs 1
logpdf/("SequentialSamplingModels.LBA", 10) 2.46 ± 0.21 μs 2.46 ± 0.21 μs 0.999
logpdf/("SequentialSamplingModels.LBA", 100) 23.6 ± 0.97 μs 23.5 ± 0.72 μs 1
logpdf/("SequentialSamplingModels.LNR", 10) 1.01 ± 0.18 μs 1 ± 0.15 μs 1.01
logpdf/("SequentialSamplingModels.LNR", 100) 8.6 ± 0.35 μs 8.67 ± 0.36 μs 0.992
logpdf/("SequentialSamplingModels.RDM", 10) 2.64 ± 0.25 μs 2.58 ± 0.21 μs 1.02
logpdf/("SequentialSamplingModels.RDM", 100) 25.1 ± 0.81 μs 25.1 ± 0.87 μs 0.999
logpdf/("SequentialSamplingModels.Wald", 10) 0.225 ± 0.16 μs 0.232 ± 0.16 μs 0.97
logpdf/("SequentialSamplingModels.Wald", 100) 2.01 ± 0.18 μs 1.99 ± 0.052 μs 1.01
logpdf/("SequentialSamplingModels.WaldMixture", 10) 1.08 ± 0.17 μs 1.08 ± 0.17 μs 1
logpdf/("SequentialSamplingModels.WaldMixture", 100) 10.6 ± 0.17 μs 10.6 ± 0.18 μs 1
rand/("SequentialSamplingModels.DDM", 10) 3.99 ± 0.57 μs 3.94 ± 0.57 μs 1.01
rand/("SequentialSamplingModels.DDM", 100) 0.0386 ± 0.0024 ms 0.0385 ± 0.0021 ms 1
rand/("SequentialSamplingModels.LBA", 10) 3.25 ± 1.2 μs 3.24 ± 0.31 μs 1.01
rand/("SequentialSamplingModels.LBA", 100) 16.7 ± 0.96 μs 16.6 ± 0.46 μs 1.01
rand/("SequentialSamplingModels.LCA", 10) 0.787 ± 0.27 ms 0.783 ± 0.28 ms 1.01
rand/("SequentialSamplingModels.LCA", 100) 8.45 ± 0.28 ms 8.49 ± 0.24 ms 0.996
rand/("SequentialSamplingModels.LNR", 10) 1.07 ± 0.18 μs 1.09 ± 0.43 μs 0.984
rand/("SequentialSamplingModels.LNR", 100) 7.89 ± 3.8 μs 7.74 ± 3.8 μs 1.02
rand/("SequentialSamplingModels.RDM", 10) 1.35 ± 0.2 μs 1.37 ± 0.43 μs 0.985
rand/("SequentialSamplingModels.RDM", 100) 11.3 ± 3.7 μs 11.3 ± 3.8 μs 0.996
rand/("SequentialSamplingModels.Wald", 10) 0.471 ± 0.17 μs 0.461 ± 0.16 μs 1.02
rand/("SequentialSamplingModels.Wald", 100) 2.91 ± 0.22 μs 2.9 ± 0.2 μs 1.01
rand/("SequentialSamplingModels.WaldMixture", 10) 1.19 ± 0.16 μs 1.19 ± 0.16 μs 0.996
rand/("SequentialSamplingModels.WaldMixture", 100) 11.4 ± 0.23 μs 11.3 ± 0.21 μs 1.01
simulate/SequentialSamplingModels.DDM 3.78 ± 1.5 μs 3.72 ± 1.5 μs 1.02
simulate/SequentialSamplingModels.LBA 3.78 ± 0.42 μs 3.68 ± 0.45 μs 1.03
simulate/SequentialSamplingModels.LCA 0.139 ± 0.057 ms 0.105 ± 0.023 ms 1.31
simulate/SequentialSamplingModels.RDM 0.103 ± 0.037 ms 0.103 ± 0.031 ms 1
simulate/SequentialSamplingModels.Wald 9.41 ± 4.6 μs 9.21 ± 4.8 μs 1.02
simulate/SequentialSamplingModels.WaldMixture 4.1 ± 1.6 μs 4.04 ± 1.5 μs 1.01
time_to_load 3.95 ± 0.04 s 3.99 ± 0.03 s 0.992

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@itsdfish
Copy link
Owner

Sounds like this will be a nice addition. Thanks!

src/stDDM.jl Outdated Show resolved Hide resolved
@itsdfish
Copy link
Owner

I have not read the paper Amount and time exert independent influences on intertemporal choice in detail, but I am wondering whether this model should be a subtype of AbstractaDDM, or should use some design principles from those models if it does not quite fit coherently. Subtypes of AbstractDDM define an increment function, which increments evidence according to the currently attended attribute or alternative (for example). The location of attention is defined by a fixation function, which is passed to rand. You can define whatever function makes sense to guide shifts in attention, including one that samples from an empirical distribution.

src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Show resolved Hide resolved
src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Show resolved Hide resolved
@kiante-fernandez
Copy link
Contributor Author

@itsdfish I added some docs and tests, along with making changes throughout.

@itsdfish
Copy link
Owner

@kiante-fernandez, awesome! I look forward to reviewing the new model. I will review piecemeal over the week and will ping you when I am done.

src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Outdated Show resolved Hide resolved
Copy link
Owner

@itsdfish itsdfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kiante-fernandez, thanks again for the PR. Aside from the minor noted issues, I think it looks great. Once those are addressed, I will merge. I did not test the docs locally. Please do to ensure they work if you have not done so already. Thanks!

using Random
Random.seed!(8414)

parms = (ν = [1.0, 1.2], α = 0.8, τ = 0.30 , s = 0.0, z = 0.50)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Would you be able to add a second test where you change each parameter just to be safe? At the very least, it would be good to test that the results agree with s > 0.


dist = stDDM(; α, ν = [-3, -3], Δt)
time_steps, evidence = simulate(dist)
@test evidence[end] ≈ 0.0 atol = 0.05
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This first set of tests make sense to me, but I'm not sure why evidence[end] ≈ 0.0 in the second test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simulate can hit (x < α) && (x > 0). It is the corresponding other bound. We would expect that two large negative weights on drift would hit the lower bound.

src/stDDM.jl Outdated Show resolved Hide resolved
src/stDDM.jl Show resolved Hide resolved
docs/src/stDDM.md Outdated Show resolved Hide resolved
docs/src/stDDM.md Outdated Show resolved Hide resolved
@itsdfish itsdfish closed this Mar 17, 2024
@itsdfish itsdfish reopened this Mar 17, 2024
@itsdfish
Copy link
Owner

@kiante-fernandez, thanks for the PR!

@itsdfish itsdfish merged commit 27f27b3 into itsdfish:master Mar 17, 2024
11 checks passed
@DominiqueMakowski
Copy link
Contributor

DominiqueMakowski commented Mar 17, 2024

Out of curiosity, how does stDDM relate to Ratcliff's DDM? Can the two be merged/combined in principle? or are they inherently distinct?

@itsdfish
Copy link
Owner

The article below explains the differences between the two models. From what I have gathered, the DDM is a special case of the stDDM in which the starting time of processing different attributes occurs at the same time.

Piecewise constant averaging methods allow for
fast and accurate hierarchical Bayesian estimation
of drift diffusion models with time-varying
evidence accumulation rates

@DominiqueMakowski
Copy link
Contributor

Which means that cross-trial variability (that #29 implements) could be in-principle added? Or am I getting confused?

@itsdfish
Copy link
Owner

I am fairly certain it is possible to compute the pdf for this model according to equation 9 in the paper. The only thing I am uncertain about is the correlation between the drift rate components here.

I think I am going to change the name. Evidence doesn't seem the most descriptive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants