Clarify more the research ideas for text and images

[elboyran]

The proposal sates the following generic research question:

Can generic properties of the XAI relevance scores be identified by systematic study?

and

Not much research on the difficult “explaining the explainer” problem is done  focus on intuitive
insights from binary classification on simple datasets.
in Scope:

Applied research on XAI methods 1-3 (RISE, LIME
and KernelSHAP) properties on binary classification
of simple image, text and time-series datasets

Make it more concrete, create hypothesis and design initial experiments.

[elboyran]

Practicalities

• Log in experiments and results in dianna-exploration, Issues also there?

• Focus on text and then images

• Use only our simple test datasets:

Stanford sentiment (text) It is possible to browse the dataset and filter it, but it seems positive is encoded blue, while negative is in red.

and

Simple shapes (images). Use the generation script to define test images for the experiments!

• Start with RISE, then LIME and KernalSHAP

• Since we decided not to implement 2 data items side by side in the DIANNA dashboard, one should either use 2 browser tabs with 2 running dashboards or Notebooks for the experiments.

[elboyran]

Hypothesis

With fixed model and (default, demo) model parameters, relevance maps change consistently with the change of input independent of the data modality.

Experiments

Text

• Design lists of 25 positive and negative words using the sentiment scale of the Stanford sentiment treebank.
• Create 25 test sentences of length 3 containing the above words, one sentence each. Perhaps as simple as

This is terrible
This is great
This is marvelous

etc.
Make them similar to, but different from the training data (see the link above), we'll use them for testing.

- Testing the hypothesis
For each XAI method
For a set of continuously changing method parameters
Evaluate:

• Qualitatively: use 2 dashboard instances to to compare pairs of sentences and their relevance maps. Evaluate the relevance score strength of the words of interest. See issue Explore text explanations dianna#439.
• Quantitatively: use a notebook (?) to display the relevancy vs the sentiment scale (per class?).

Images

• Use the generation script to create ~10 test images per shape per shape parameter. Shape parameters are:
• for circles: size and gray-scale contrast? (BG and FG)
• for triangles: size, rotation angle and gray-scale contrast? (BG and FG)

- Testing the hypothesis
For each XAI method
For a set of continuously changing method parameters
For each shape parameter (with others fixed)
Evaluate:

• Qualitatively: use 2 dashboard instances to compare pairs of "neighboring" (in the variable shape parameter) shapes and their relevance maps. Evaluate the (average/mean/max/median?) relevance score strength. See issue Explore text explanations dianna#439.
• Quantitatively: use a notebook (?) to display the (average/mean/max/median?) relevancy vs the shape parameter scale (per shape = class?).

[elboyran]

@loostrum would you like to "review" my ideas?

[loostrum]

I think these are some nice, concrete tasks that we could work on. For the quantitative work indeed it may be easiest to do this with notebooks. When running the dashboard by hand, we should make sure to store our results from that somehow. Perhaps just make screenshots of interesting results?

[elboyran]

When running the dashboard by hand, we should make sure to store our results from that somehow. Perhaps just make screenshots of interesting results?

Good point, I'll add this to the actual issues description.