Respect environment variables in jvm.options

[kaisecheng]

Release notes

Add support for ${VAR:default} syntax in jvm.options file

What does this PR do?

JvmOptionsParser adds support for ${VAR:default} syntax when parsing jvm.options

• allow dynamic resolution of environment variables in the jvm.options file
• enables fallback to default value when the environment variable is not set

Why is it important/What is the impact to the user?

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Fixes: Environment variables in jvm.options are not replaced by the JvmOptionsParser #16444

Use cases

Screenshots

Logs

[donoghuc]

I like that we skip this if we have a commented or empty line. Doing the regex on every "valid" line in config is probably low overhead in practice. Overall I think this will work for the majority of cases but it does have some requirements and edges that may be worth documenting.

1. Spell out the highlights from the regexes \$\{ [A-Z_][A-Z0-9_]* (mainly that the substitution must be in form ${FOO} not $FOO, the name must be all caps and not start with an underscore etc ${FOO} not ${foo} and the default syntax).
2. Nested substitution wont work ${ROOT:${LEAF}}
3. Default values cannot have colons or dollar signs (i think this is relevant because we have an example of such in

   logstash/config/jvm.options

   Line 33 in 01c8e8b

   #-Djava.io.tmpdir=$HOME

   )
4. There is no escape characters (relates to 3)

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 90befd8

History

• 💚 Build #1991 succeeded 70eb054
• 💚 Build #1990 succeeded 45b6e2d

[kaisecheng]

@donoghuc thanks for the edge case checking! I overlooked that we support lowercase variable and the $HOME in jvm.options.

Logstash has documented about the dollar sign variable. $VAR is not the pattern Logstash support. Honestly the variables in jvm.options were interpreted in bash script which has different syntax to logstash env variable. Given that no reported issue on resolving the env var in jvm.options and it has been failing to resolve since 8.0, in this PR, I make it align to Logstash syntax. I noticed that we have different convention in log4j. I will raise the question in weekly sync.

I updated the pattern to align to the current ruby substitution. We do not support nested variable