question about inference.evaluate.eval_fixed

[andreifoldes]

I want to run the simplest of RSA's, but want to make sure I'm doing it right.

I'm focused on the following:

results = rsatoolbox.inference.eval_fixed(mymodel, rdms, method='spearman')

I want to double check whether my neural rdms and model rdm are aligned according to the same feature.
When looking at my model RDM it looks like the following:

rsatoolbox.rdm.RDMs
1 RDM(s) over 6 conditions

dissimilarity_measure = 
rank

dissimilarities[0] = 
[[0. 1. 2. 3. 4. 5.]
 [1. 0. 1. 2. 3. 4.]
 [2. 1. 0. 1. 2. 3.]
 [3. 2. 1. 0. 1. 2.]
 [4. 3. 2. 1. 0. 1.]
 [5. 4. 3. 2. 1. 0.]]

descriptors: 

rdm_descriptors: 
name = ['distance']
index = [0]

pattern_descriptors: 
rank = ['1', '2', '3', '4', '5', '6']
index = [0 1 2 3 4 5]

However I have less insight into how my neural rdms are "ordered" at the time of the ominous correlation, given that they have multiple pattern_descriptors and eval_fixed doesn't allow my to ensure/specify which pattern descriptor (e.g.: "rank") to order by when doing evaluation so I'm worried about misalignment.

[JasperVanDenBosch]

Hi Andrei, the rdms are not reordered during evaluation; in other words, they are kept in the order given by the index pattern descriptor. It is assumed that the order of the model and data rdms are the same.
The relevant functions here would be:

rsatoolbox/src/rsatoolbox/rdm/compare.py

Line 162 in 0f1e77a

def compare_spearman(rdm1, rdm2):

rsatoolbox/src/rsatoolbox/rdm/compare.py

Line 592 in 0f1e77a

def _parse_input_rdms(rdm1, rdm2):

Does that answer your question?

[andreifoldes]

Thanks for the quick reply! What would you recommend is the best way to verify how the "index" pattern descriptor translates to the order of another pattern descriptor? So when looking at my rdms, lets say that I have a pattern descriptor that has the values "A", "B", "C" .. how can I check what index values correspond to those values?

[JasperVanDenBosch]

You can access all the pattern descriptors with RDMs.pattern_descriptors or if you wanna have a look at them in table form I'd recommend RDMs.to_df().

But instead of worrying about the index descriptor, at the stage where you import the data, I would do RDMs.sort_by(myCondition='alpha') where "myCondition" is the ABC descriptor. If you do this on all RDMs objects that you use, before operations, they should be in the same order. Does that make sense?

[andreifoldes]

Yes, thank you! So I can use sort_by(myCondition='alpha') for both mymodel and my rdms variable?

[JasperVanDenBosch]

Exactly; assuming they both have this descriptor called "myCondition", and you want it sorted alphabetically.

[JasperVanDenBosch]

@andreifoldes if you don't mind I'd keep this open, the idea with the "discussion" I think is to also provide something for other people to look for answers to their questions. But maybe you can mark my comment as "Answer"? Thanks!