google-deepmind / google-deepmind/opro

AttributeError: 'NoneType' object has no attribute 'lower'

Open
#5 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
777
Forks
90
PR merge metrics
No merged PRs in 30d

Description

python optimize_instructions.py --optimizer="gpt-3.5-turbo" --scorer="text-bison" --instruction_pos="Q_end" --dataset="gsm8k" --task="train" --palm_api_key="..." --openai_api_key="..."

I tried to run this code to get such a problem on the gsm8k data collection. How should I solve it?

File "/root/autodl-tmp/LLMasop/opro/evaluation/eval_utils.py", line 802, in evaluate_single_instruction
choices = list(
File "/root/autodl-tmp/LLMasop/opro/evaluation/eval_utils.py", line 804, in
lambda x, y: _parse_prediction(
File "/root/autodl-tmp/LLMasop/opro/evaluation/eval_utils.py", line 794, in _parse_prediction
return metrics.get_normalized_prediction(
File "/root/autodl-tmp/LLMasop/opro/evaluation/metrics.py", line 210, in get_normalized_prediction
prediction_parsed = prediction.lower().strip()
AttributeError: 'NoneType' object has no attribute 'lower'

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.