About Self-CRAG

[innovation64]

When I reproduce your self-CRAG results, it can only reproduce popqa dataset, when I use other dataset that deal with your scritps, it will run IndexError: list index out of range, so What;s the problems.

Here's my instruction:

• for arc

python run_short_form.py \
  --model_name selfrag/selfrag_llama2_7b \
  --input_file ../../CRAG/data/arc_challenge/output/arc_challenge_selfcrag.json\
  --max_new_tokens 50 --threshold 0.2 \
  --output_file ./output/popqa_llama_arcselfcrag.jsonl \
  --metric match --ndocs 5 --use_groundness --use_utility --use_seqscore \
  --task arc_c

-for pub

python run_short_form.py \
  --model_name selfrag/selfrag_llama2_7b \
  --input_file ../../CRAG/data/pubqa/output/pubqa_selfcrag.json \
  --max_new_tokens 50 \
  --threshold 0.2 --output_file ./output/pobqa_llama_pubselfcrag.jsonl \
  --metric match --ndocs 5 \
  --use_groundness --use_utility --use_seqscore \
  --task fever

[ShayekhBinIslam]

Hi @HuskyInSalt @JasonForJoy,
If you kindly address this issue, it would be helpful.