considering running CRAG on Mac M1 series computers?

[shuiRong]

Because I heard that CUDA is not actually needed when CRAG is actually running. Is that so?

there is no NIVIDA CUDA on the Mac Apple Silicon series computers.

errors:

  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      fatal: not a git repository (or any of the parent directories): .git
      /private/var/folders/tv/hjs548y55jd99cfcnq4_8_fm0000gn/T/pip-install-3w1lp_js/flash-attn_6d44432be8804728b0a7d01e5a3fc0ab/setup.py:72: UserWarning: flash_attn was requested, but nvcc was not found.  Are you sure your environment has nvcc available?  If you're installing within a container from https://hub.docker.com/r/pytorch/pytorch, only images whose names contain 'devel' will provide nvcc.
        warnings.warn(
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/tv/hjs548y55jd99cfcnq4_8_fm0000gn/T/pip-install-3w1lp_js/flash-attn_6d44432be8804728b0a7d01e5a3fc0ab/setup.py", line 126, in <module>
          CUDAExtension(
        File "/opt/anaconda3/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1076, in CUDAExtension
          library_dirs += library_paths(cuda=True)
                          ^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/anaconda3/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1203, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/anaconda3/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2416, in _join_cuda_home
          raise OSError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
      
      
      torch.__version__  = 2.1.2
      
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

[HuskyInSalt]

Hi @shuiRong , since I have not tried doing experiments on the Mac M1 series, I have no idea about how to run CRAG on them. CUDA was utilized when evaluating the quality of the retrieval with a T5-based model, and generating responses with LLaMA2. Hope it can help you to convert it to Mac M1 series.