NaN in Generated Embeddings

[superctj]

Thank you for open-sourcing this amazing work! I followed the instructions in README to generate embeddings for my use case. However, I found there are vectors consisting of NaNs. Do you have any insights into this issue?

[TianBaoGe]

What graphics card did you use? I'm using a 3090 and encountered the same problem. You can check this issue.

[superctj]

Thank you @TianBaoGe for the pointer! I am using A40 and flash-attn 2.6.3. It seems from the linked issue that the problem has been fixed in an earlier version. Did you fix the problem by upgrading flash-attn?

[claudepi]

I just encountered this as well. On a hunch, I downgraded torch from v2.5 to v2.4 and the nan values are gone. Perhaps check the version of torch that you are using?

[superctj]

Thank you @claudepi for the suggestion! Another alternative can be models from sentence transformers.