Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T5 Translation example will not run without replication factor set (even though I set it to be greater than equal to 1) #482

Open
danao413 opened this issue Sep 16, 2024 · 0 comments

Comments

@danao413
Copy link

09/16/2024 09:02:53 - critical - poptorch::python - ValueError: IPUConfig attribute replication_factor must be >= 1. You provided value=0

Traceback (most recent call last):
File "run_translation.py", line 671, in
main()
File "run_translation.py", line 568, in main
trainer = IPUSeq2SeqTrainer(
File "/localdata/u.do100367/poptorch/lib/python3.8/site-packages/optimum/graphcore/trainer_seq2seq.py", line 64, in init
super().init(
File "/localdata/u.do100367/poptorch/lib/python3.8/site-packages/optimum/graphcore/trainer.py", line 282, in init
self.ipu_config.replication_factor = n_ipu // self.ipu_config.ipus_per_replica
File "/localdata/u.do100367/poptorch/lib/python3.8/site-packages/optimum/graphcore/ipu_configuration.py", line 485, in setattr
vfunc(name, value)
File "/localdata/u.do100367/poptorch/lib/python3.8/site-packages/optimum/graphcore/ipu_configuration.py", line 238, in _contents_geq_value_validator
raise ValueError(f"IPUConfig attribute {name} must be >= {floor_value}. You provided {value=}")
ValueError: IPUConfig attribute replication_factor must be >= 1. You provided value=0

I have changed this multiple times in the IPUConfig instantiation to be 1, 100, 10 and each time I get this error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant