Improving ClearML Agent documentation regarding Git / GitHub

[d13g0]

Hello,

It would be great if there was a Git configuration section for the ClearML Agent. Here are some things it woudl be great to have in there:

Personal access tokens

I got stuck because my agent was unable to checkout the code from my repository after configuring it to use SSH authentication, in pip mode. Looking at the debugging info from the agent (-d), I saw that the problem was that the personal token needed write access (not sure why). I was able to fix this by granting the repo privilege in GitHub which is not set by default when generating a personal token So perhaps this would be useful for someone like me who is creating their first agent (?).
See: https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/scopes-for-oauth-apps#available-scopes

Type of keys supported

My agent couldn't connect to GitHub using ed25519 key but it did with anrsa key. Not sure if you can reproduce this but just thought of letting you know that if this is the case, it would be great to have it documented.

Expected behaviour when ...

Not clear what the expected behavior is if you force SSH but you have the user/pass defined. It would be great to have that in the documentation

My environment

I am using ClearML1.12.1 running the agent in AWS Ubuntu Linux 22.04

I am new to ClearML so I have a fresh perspective. I hope you find this feedback useful.

[AlexandruBurlacu]

Hey @d13g0, these are great suggestions, thank you! We'll add these clarifications in upcoming releases of ClearML Agent. That being said I have a few questions.

Can you please double check whether you had your ed25519 key correctly registered on GH? We use the underlying git to fetch/clone data, so it sounds weird that the ed25519 didn't work, while the rsa one did.

Second, can you please paste here the section in clearml.conf file that defines all these settings? Ofc don't forget to obfuscate sensitive information. The reason I'm asking is because it's not clear to me why would you need the GH token if you use SSH auth. And also how is pip mode important here. Seeing the config will make it clearer to me.

[d13g0]

Hi @AlexandruBurlacu thanks for your reply.

Personal access tokens
Glad to help improve the documentation for folks using this option

SSH
In the clearml.conf all I changed was:
force_git_ssh_protocol: true

Just to clarify, I was testing both alternatives. I started with tokens, it worked as I mentioned by providing the right permissions to the token. Then, I moved to SSH and tested it, it worked with rsa but not with ed25519. I will test again.

I mentioned that I am running the agent in pip mode because I imagined (and please correct me if this is not the case), that there are additional considerations if the agent was in docker mode. For example, forwarding SSH agent into a docker container? Not sure how that part works... but in any case this is why I mentioned it.