You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
where the original order is recovered to account for the voting result.
Expected Behavior
The code should include the necessary logic to revert the shuffling and account for the voting result.
The text was updated successfully, but these errors were encountered:
HyunggyuJang
changed the title
Shuffling Ensemble: reverting shuffling for the voting process is missing
Shuffling Ensemble: unshuffling ensemble variants for voting is missing in the code
Dec 16, 2023
Issue Description
The code is missing the implementation for reverting shuffling in the voting process. Currently, the shuffling ensemble is implemented in
promptbase/src/promptbase/mmlu/problem_utils.py
Line 63 in dcafda4
However, I couldn't find a section in
promptbase/src/promptbase/mmlu/analyze.py
Lines 77 to 98 in dcafda4
where the original order is recovered to account for the voting result.
Expected Behavior
The code should include the necessary logic to revert the shuffling and account for the voting result.
The text was updated successfully, but these errors were encountered: