New dredd init & config & CLI

[honzajavorek]

This is a proposal on how to tackle the dredd init + dredd.yml epic.

Parts of Dredd affected

• dredd init wizard
• configuration
• dredd.yml

Requirements

• replace optimist with something maintained [email protected] has license issues #1057
• test and improve dredd init
  • add questions Add hook file question to the dredd init #320
  • improve wording and behavior Improve wording and behavior of dredd init #879
  • more robust detection Interactive config should employ better algorithm to detect installation command in CI config files #525
• test and improve generated CI configurations:
  • migrate to circleci 2.0 Dredd's init should support creating more CI configurations #878, Circle CI tutorials should work with their version 2.0 #978
  • add wercker Dredd's init should support creating more CI configurations #878
  • add appveyor Dredd's init should support creating more CI configurations #878
  • revise travisci Dredd's init should support creating more CI configurations #878
  • make all of them first-class citizen (AFAIK Dredd won't generate travis in favor of circleci now, only update existing travis config)
  • add commands for hooks Commands to install language hook workers in generated CI configuration #322
  • unify CI docs Unify docs how to plug Dredd into CI #353, Circle CI tutorials should work with their version 2.0 #978
  • add integration tests for dredd init
• bonus feature, maybe even a separate epic: allow downloading of the API description document by Dredd directly Could dredd init include fetching APIB file? #366
• refine the CLI and the options
  • allow overriding of blueprint and endpoint CLI params should override dredd.yaml #455
  • rename certain outdated options (such as blueprint and endpoint), but keep backward compatibility (at least for a while)
  • test and fix bugs in hierarchical configuration When a config file is present, adding headers via command line with -h doesn't work #757, Basic Auth from CLI overridden by Auth from document. #915
  • revise Use host option defined in OpenAPI standard #997
  • do not require server address for --names --names requires server address #563
• support YAML file with defaults, i.e. if some values are not specified, they're correctly defaulted

Solutions

• migrate to yargs or something else for CLI arguments parsing Consider using a different library for the Dredd CLI #845
• use nconf for configuration

New CLI

Clearly distinguish commands, arguments, and options. Notes:

• dredd init, --config indicates where to save the file, --init doesn't exist, remove -i
• dredd names <api description filename - or - url> <api description filename - or - url> ... (formerly --names)
• dredd <api description filename - or - url> <server url>
• (dredd.yml exists) dredd <api description filename - or - url - or apiary subdomain/url> <server url> overrides the dredd.yml values
• (dredd.yml exists) dredd --api-description=... overrides the dredd.yml value
• --api-description=... is instead of --path
• (dredd.yml exists) dredd --server-url=... overrides the dredd.yml value
• dredd --version = dredd version (alias)
• dredd --help is command-specific, e.g. dredd names --help
• dredd --hookfiles= changes to dredd --hooks=
• --language is not required if it can be detected from --hooks= files extensions, otherwise there should be aliases (python = python-dredd-hooks binary etc.) or it is treated as a custom hooks handler (works now, but not documented)
• Sandbox is only allowed when --language is resolved as nodejs or javascript or js, otherwise it should fail
• The CLI arguments should fail in general in case they don't make sense together (e.g. --server-wait without --server
• I'm not even sure what --details, -d does and whether anyone uses it
• Clearly distinguish all default values and whether particular option is boolean or not

Downloading API description from Apiary

• Currently there's -j/--custom with custom configuration and apiaryApiName with apiaryApiKey properties, where apiaryApiName is a unique identifier of the API project in Apiary
• We could leave --custom completely to user-specific stuff and pass it on to hooks (see 7bffde7#r29826481) - Access Custom options in Hooks #215
• Dredd could accept the identifier instead of the API description file name or URL: dredd myapi http://localhost:4000 this is dangerous and non-deterministic and we would hate ourselves for such design decision
• Dredd could accept --apiary-api-name
• Call the thing consistently as API NAME, api-name, or apiName, to be consistent with the Apiary CLI
• Dredd could have --apiary-api-key option instead of apiaryApiKey and accept APIARY_API_KEY= env var as well (to work together with the Apiary CLI)
• When testing happens with correct Apiary configuration and there is no local file given, Dredd downloads the API description document as if it was given URL, and tests against it
• When there is both api name and local file, dredd makes a request to Apiary API, checks the checksum and identifies whether the local document is up-to-date. It warns if it's not up-to-date.
• The Tutorial in Apiary turns to $ dredd init --apiary-api-name=myapi --apiary-api-key=9a5.... --reporter=apiary

[honzajavorek]

Depends on

• lcid depends on [email protected], which is missing full license text sindresorhus/lcid#3
• Upgrade os-locale yargs/yargs#1195

[honzajavorek]

• --hooks=... (files)
• --hooks-handler=... (binary path or keyword like python, js, js-sandbox)
• remove --sandbox in favor of the -sandbox suffix (see above)
• js, nodejs, and javascript are aliases
• the --hooks-handler value default is detected from the --hooks value (extensions) or set to JavaScript, so most of the time the option isn't needed at all
• special value custom allows for Allow bypassing handler spawning logic #755, doesn't spawn anything, but sends the data to given hook handler host and port (prints out where it's sending the data... could do this always anyway)
• rename --hooks-worker-* to --hooks-handler-*
• --server=... (binary path)

[artem-zakharchenko]

• Separate validation/normalization/coercing logic during Dredd config application
• Provide proper override priority between CLI args, dredd.yml and default config
• Make config options deprecation more maintainable
• Prevent propagation of aliases into CLI args config (i.e. l, q, etc.), use full option names instead (may require migration to another CLI commands handler than optimist)