-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AttributeError: 'NoneType' object has no attribute 'lower' #5
Comments
Hi @luochenxin, this seems to suggest that the opro/opro/evaluation/eval_utils.py Line 769 in e81b2f5
None elements (instead it should be a list of strings). Could you print out this variable to check its value?
|
Thank you for your suggestion, I tried to print its value and I found that one of the elements in this list has a value of None, I'm not sure why this problem occurs, how can I fix it? |
This sounds weird, especially when only one of the elements is None and the others are normal. To track down the error, could you also print out a few more variables, like opro/opro/evaluation/eval_utils.py Line 648 in e81b2f5
|
Following your suggestion, I printed "raw_prompts_flattened" without any problem, but when I print the value of "raw_answers" at /opro/evaluation/eval_utils.py, line708, I find that the elements in this list, each time the serial number 30 has a value of "None", and |
Hi, have you solved this problem? I have also encountered this problem. |
I think it may be a problem with Gemini. If I switch to gpt-3.5, there will be no problem. |
python optimize_instructions.py --optimizer="gpt-3.5-turbo" --scorer="text-bison" --instruction_pos="Q_end" --dataset="gsm8k" --task="train" --palm_api_key="..." --openai_api_key="..."
I tried to run this code to get such a problem on the gsm8k data collection. How should I solve it?
File "/root/autodl-tmp/LLMasop/opro/evaluation/eval_utils.py", line 802, in evaluate_single_instruction
choices = list(
File "/root/autodl-tmp/LLMasop/opro/evaluation/eval_utils.py", line 804, in
lambda x, y: _parse_prediction(
File "/root/autodl-tmp/LLMasop/opro/evaluation/eval_utils.py", line 794, in _parse_prediction
return metrics.get_normalized_prediction(
File "/root/autodl-tmp/LLMasop/opro/evaluation/metrics.py", line 210, in get_normalized_prediction
prediction_parsed = prediction.lower().strip()
AttributeError: 'NoneType' object has no attribute 'lower'
The text was updated successfully, but these errors were encountered: