Using GEMM files in fastertransformer_backend.

[SnoozingSimian]

While loading both GPTJ and GPT-NeoX models, I get the message
[WARNING] gemm_config.in is not found; using default GEMM algo

This suggests to me that there is a way to add gemm algos while loading these models in, I have generated the gemm_config.in for GPT-NeoX using the FasterTransformer binaries, but I don't know where to place this file so that it can be found by the backend.

Is there any possible way to use it currently?

[byshiue]

You can refer this document https://github.com/NVIDIA/FasterTransformer/blob/main/docs/gptneox_guide.md#run-gpt-neox to generate the gemm_config.ini. We will add the program of gemm test into backend in the future.

[leegohi04517]

i generated gemm_config. but then how to use it. the /opt/tritonserver/bin/tritonserver didn't load the config automatically