Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provisioning a host that can only be reached via ProxyJump is extremely difficult #201

Closed
Cu3PO42 opened this issue Sep 5, 2023 · 4 comments
Labels
enhancement New feature or request

Comments

@Cu3PO42
Copy link

Cu3PO42 commented Sep 5, 2023

Is your feature request related to a problem? Please describe.

I wanted to use nixos-anywhere to setup NixOS on a work machine, which I can only reach by going through a seperate login server. For this host, I have an entry in my ~/.ssh/config along the lines of

Host to-be-provisioned
  User myusername
  HostName internal.address
  ProxyJump login.veryrealcompany.com

I was able to SSH into the host successfully and ran nixos-anywhere ... myusername@to-be-provisioned.
Initially, nixos-anywhere was able to connect as well and installed its SSH key.
However, at some point, it switched to trying to connect via internal.address which obviously did not work.

To mitigate this, I added a Host internal.address entry to my ~/.ssh/config and was able to get a step further.
Subsequently, ssh-keyscan failed because apparently it does not read ~/.ssh/config.

As a last resort, I tried to port-forward internal.address:22 to localhost:2222 and target that instead, but again ssh-keyscan failed because it tried to scan username@localhost rather than localhost.

Describe the solution you'd like

I'm sure each of these issues can (and maybe should be) fixed individually.
However, I do believe that my initial attempt should ideally just work.
To that end, nixos-anywhere should stick with the originally provided hostname for its connections or manually specify the appropriate ProxyJump. Further, if a ProxyJump is required, ssh-keyscan must be run on the bastion host.

In case there is a good reason to for nixos-anywhere to switch the hostname it uses for connections in some scenarios, it may be sufficient to enable this behavior through a flag.

Describe alternatives you've considered

Instead of automatically detecting the configured bastion host, it could also be specified via command line args.
While I find that slightly more cumbersome, an argument could be made that that solution is purer since the bastion host needs to be explicitly specified.
That said, the SSH config is currently respected so I believe implicit detection does not go against the spirit of the tool.

Additional context

In this setup, I have no possibility to run nix on the bastion host. I have instead performed the installation manually by running the same steps that nixos-anywhere would, only with the correct SSH options.

@Cu3PO42 Cu3PO42 added the enhancement New feature or request label Sep 5, 2023
@Mic92
Copy link
Member

Mic92 commented Oct 28, 2023

Mhm. It's quite common that jumphosts actually don't allow one to run commands on it. We currently only need the ssh-keyscan in case we need to build on the remote.
So we could improve the situation a bit by not fetching the public key otherwise.

@Mic92
Copy link
Member

Mic92 commented Oct 28, 2023

With #240 you should be able to have the jumphost passed to --ssh-option instead unless you rely on --build-on-remote

@yeoldegrove
Copy link

I can confirm that --ssh-option "ProxyJump=${user}@${host}" works for me.

@Mic92
Copy link
Member

Mic92 commented Nov 6, 2023

Closing this for now as it works in most cases.

@Mic92 Mic92 closed this as completed Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants