Making SSH Nodes work both on Linux and Windows based systems

[dmatej]

When I implemented these tests I had to face quite a lot of issues even on Linux. When I finish, I will write some blog about it, it is another painful episode. However I have to make notes and it will be useful also for users and reviewers. In short:

Assumptions of the test

• The private key doesn't have any passphrase
• DAS uses create-node-ssh --install to "clone" GlassFish installation to another machine.
• Test uses Temurin docker images as a base.

Node side: Environment

• jar is not on PATH everywhere
  • however unzip might be already there - fix: add some autodetection code.

Node side: SSHD Configuration

• SFTP has to be explicitly enabled on some systems
• public key authentication can be disabled. Same applies to other types.
• User interaction can be forbidden.
• Port can be blocked by firewall or antivirus, etc.
• Port can be different.
• ...

DAS side: Environment

• When using scripts, no TTY might be available. Scripts can fail or wait for user input forever.
  • Always set timeout when such risk exists
  • For ssh command is useful sshpass command. Test uses it, however GlassFish uses Jsch library instead.

DAS side: SSH Command Configuration

• Applies to the test and user experiments, not to GlassFish
• Empty passphrase - be careful, when you set it via execInContainer: "''" means apostrophes, not empty passphrase.
• User interaction can be forbidden just like on the server (sshd)
• StrictHostKeyChecking=accept-new should not be used on production; however would be useful if we would not enforce user to run the ssh command on his own and manually accept the server's key. That is TODO.

DAS side: JSCH SSH Implementation

• "raw" and "resolved" password and passphrase. I doubt the implementation is correct.
• logging - fixes will be pushed later
  • race condition - fix: use Jsch logger name.
  • impossible to guess which logger level I should set, overengineered antipattern of "set logger for Jsch once for asadmin, then for kernel, then for cluster, then just jsch, ..." - then you set 4 logger levels and it still doesn't print anything. - fix: always use Jsch logger name.