Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OOM for first-time build with CUDA Toolkit 12.4 #500

Open
kevinddchen opened this issue Nov 20, 2024 · 0 comments
Open

OOM for first-time build with CUDA Toolkit 12.4 #500

kevinddchen opened this issue Nov 20, 2024 · 0 comments

Comments

@kevinddchen
Copy link

kevinddchen commented Nov 20, 2024

I have noticed that when compiling CUDA extensions for gsplat, e.g. running for first time, my machine freezes and becomes unresponsive at this point:

_C = load_extension(
. I gather that this is an OOM issue because it is resolved by setting MAX_JOBS=4 when running.

I am able to consistently reproduce this issue on my machine. I am using CUDA Toolkit 12.4 and 32 GB of CPU RAM. The GPU is A10G and the TORCH_CUDA_ARCH_LIST env var is not set. This issue does not occur with CUDA Toolkit 11.8 (all other specs unchanged).

Not suggesting any change at the moment, but more curious of others have encountered this issue as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant