Support Diagnostic 8.0.1

When running the api type, the diagnostic will run directly against the input host rather than trying to find the master since in cloud deployments the HTTP address information will not result in a valid endpoint.

Support Diagnostic 8.0.0

Updated Help and Interactive Mode
When the diagnostic was just the diagnostic and a limited set of options a set of paragraphs and lists was adequate. But at this stage there's just too much there for that that format and it's obvious from some of the questions that some had difficulty finding the requisite information. It's now been reworked, with the major additions being a table of contents so you can review what's in the documentation and then go right to the place you want and a set of tables with a full list of the available parameters, a short description of what each does, and an example or two of what it should look like. Hopefully that will head off some of the issues.

Related to that, running the diagnostic with no arguments will send it into the new interactive mode. It will use a Q&A process to step the user through the process of picking the inputs rather than making them remember the parameters. A number of the prompts have defaults that serve as examples and/or lists where you pick from a set of options so you don't need to remember the syntax. If someone is having difficulty running the diagnostic, the first suggestion from this point would be to try the interactive prompt if they haven't already. For those who just like to type, command mode is still available, but make sure to check the docs since some of the input names have changed. Diagnostic types, for example.

It's available for all four executions scripts: the diagnostic, scrub, monitoring export, and monitoring import.

New Types of Diagnostic Execution
The convention for installing and running the diagnostic has been constant for quite a while. Download the archive to the node you wish to interrogate, cd to the directory created by the diagnostic archive and run. If there were issues with installing it on a production system due to strict internal controls there was an option to run remotely, but only the REST API's were available. No logs or system calls. 

In this release, you can now run a full diagnostic, including logs and system calls against a node deployed on a different host than the diagnostic is installed on. Host from a different node, developer or administrator workstation, whatever. The diagnostic will create a secure terminal session with the host the specified node is running on, determine the log locations, copy the logs to the host the diagnostic is running on, and run the system calls from that same terminal session. You do need to have the credentials that you'd need if you were going to shell out to that host manually in the form of id/password or an ssh key file. And the account either needs read access for the directory the logs are written to or be able to run sudo. It's similar to if. you did it manually - you just don't have to do it yourself. 
To go with this capability the types have changed so make sure to review them in the documentation. Every archive will now have the type that produced it in the name.  The type local replaces standard and is still the default. The old remote option is now api reflecting the fact that all it will run is the REST API's. And the new mode described above where the diagnostic is deployed on one host and run against a node on a different one for a full set of outputs is now remote. Got that? There's a quiz tomorrow ;). 

Logstash options are lined up exactly with the Elasticsearch options: logstash, logstash-api, and logstash-remote. 

Diagnostic Docker Container Support
In the past, there have been a number of customer requests and questions relating to running the diagnostic in a container. It wasn't all that compelling at the time given the only thing you could reliably run was the old remote mode and there were no protections to prevent it from failing if tried the others.  However since the new remote mode now exists and you can extract a complete diagnostic from a remote host it made sense to revisit this. The diagnostic now has some internal checks to let it know when it is executing from within a container and disable the local execution modes for both Elasticsearch and Logstash. Remote and API types will both work for Elasticsearch and Logstash. 
It's not deployed into a repository at this time, but you can build your own Docker image with the script located in the diagnostic root. There's a dockerfile in the /docker directory it will use. There is also a run script in that directory which illustrates how to run the diagnostic from within a docker image. You just bring up what is essentially an interactive command line session and then run the diagnostic commands exactly as you would if it were installed directly on the host. You do need Docker installed on whichever host you intend to build the image from. And pay attention to the volume that is created since that is the key to getting an output archive. 

Give it a try and let me know ASAP if there are any issues. It's still experimental.
Everything ElseIt is no longer necessary to run the diagnostic from within the installation directory. It will pick up the working directory and adjust the classpath from within the script. Please note, however, that symbolic links are not supported.

Everything Else

• Using "http://" and "https://" in the host name has been a recurring issue so there is now a validation check for that. It will put up a message now when it's encountered. A future release will probably just remove it silently.
• The diagnostic will now check for which node is the current master, and whether it is set up for HTTP access. If this can be verified then all subsequent REST calls will be run against the master rather than the node targeted by the host. Since a number of these calls always go through the master it will minimize the number of round trips overall.
• Fixed issues with Logstash diagnostics failing.
• Bug fixes for monitoring import default parameters that required explicit clustername and index name to be used.
• Monitoring export now requires the stop date and time rather than a start date and time. Rather than having to subtract to calculate when a range of data will start you can simply take the default (now) and specify how far back you want it to go. See the docs for more details.
• Related to that, the date and time have been split into two arguments so that quotes around the date are not now necessary.
• Transform and Data Frame API calls have been added.
• When running against a node in a docker container, all running docker images will have inspect run against them.
• If a node is in a docker container that is local or is a remote node with the appropriate host credentials the system calls will be run for the host containing that docker image.

Support Diagnostic 7.1.5

Backward compatibility fix for remote.

Support Diagnostic 7.1.4

Manifest was not being generated.

Support Diagnostic 7.1.3

Bugfixes only.

• Fixed issue where regression caused reoccurrence of delete utility directory if explicit output not used.
• Fixed issue where cluster name modification of monitoring export was disregarded.
• Fixed issue where proxy password was not being masked on input.
• Fixed issue where proxy host was set incorrectly.
• Fixed issue where v7 monitoring clusters would fail on scroll.

Support Diagnostic 7.1.2

• Minor doc fixes
• Fixed timestamp field in manifest.json.

Support Diagnostic 7.1.1

Fixed issue with --noVerify breaking after implementing pooling.

Support Diagnostic 7.1.0

• Export and import from monitoring indices - up to 12 hours of data for a monitored cluster can be extracted from the monitoring cluster and then indexed into a local monitoring instance for offline viewing.
• Running the jstack and jps commands could fail if the diagnostic was run via a different JVM instance than the one used by Elasticsearch. This became more of an issue with the embedded JVM. The diagnostic now checks for which installation was used to run ES and runs the commands via that same deployment.
• SLM REST calls added.
• Fixed issue with scrub utility when working directory was used instead of explicit output destination.
• Fixed issue with spaces in path names.

Support Diagnostic 7.0.9

Bug Fixes and some new functionality:

Diagnostic

• Fixed issue where hostname was not recognized, thus bypassing system calls and log collections.
• When first call fails due to invalid authentication an explicit message is now displayed in the console.
• Internal cookie spec updated to remedy issues some were having when running through load balancers. They still should not be running through a balancer, but it at least shouldn't end badly and hold things up.

Scrubber:

• Fixed issue where token length caused an error.
• Added option to scrub a single file instead of an entire diagnostic archive to facilitate processing a single log file.

Support Diagnostic 7.0.8

Fixed #289 where message displayed even when superuser role present.
Added bypass of check when security not enabled.
Updated Jackson dependency.