Introducing Swagger in Dredd.

Also addressing: - a bit of #353 - closes #496 - closes #389
apiaryio · Jun 14, 2016 · 88bd38d · 88bd38d
1 parent 214635a
commit 88bd38d
Show file tree

Hide file tree

Showing 18 changed files with 679 additions and 446 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -23,7 +23,7 @@ Recommended workflow:
 4. Make sure [test coverage][] didn't drop.
 5. Send a Pull Request.
 
-## Handbook for Contributers and Maintainers
+## Handbook for Contributors and Maintainers
 
 ### Maintainers
 

diff --git a/README.md b/README.md
@@ -9,8 +9,7 @@
 ![Dredd - HTTP API Testing Framework](img/dredd.png?v=3&raw=true)
 
 > **Dredd is a language-agnostic command-line tool for validating
-API description written in [API Blueprint][] format against its backend
-implementation.**
+API description document against backend implementation of the API.**
 
 - [Documentation][]
 - [Changelog][]
@@ -20,6 +19,11 @@ Dredd reads your API description and step by step validates whether your API
 implementation replies with responses as they are described in the
 documentation.
 
+## API Description Formats Support
+
+- [API Blueprint][]
+- [Swagger][] **(experimental)**
+
 ### Supported Hooks Languages
 
 Dredd supports writing [hooks](https://dredd.readthedocs.io/en/latest/hooks/)
@@ -68,6 +72,7 @@ $ npm install -g dredd
 [API Blueprint]: http://apiblueprint.org/
 [API Blueprint tutorial]: https://apiblueprint.org/documentation/tutorial.html
 [API Blueprint examples]: https://github.com/apiaryio/api-blueprint/tree/master/examples
+[Swagger]: http://swagger.io/
 
 [Documentation]: https://dredd.readthedocs.io/en/latest/
 [Changelog]: CHANGELOG.md

diff --git a/docs/example.md b/docs/example.md
@@ -2,7 +2,7 @@
 
 This is an example how to create a simple [Express.js][] API backend application tested with Dredd
 
-## Create API Description File
+## Create an API Description Document
 
 Create a new API description file, for example in the [API Blueprint][] format. Save it as `api-description.apib`
 

diff --git a/docs/execution-lifecycle.html b/docs/execution-lifecycle.html
@@ -0,0 +1,27 @@
+<!DOCTYPE HTML>
+<!--
+    This file exists only to redirect from old URL to new URL.
+    This is more properly done using an HTTP redirect, but unfortunately
+    readthedocs.org's page redirection is broken (see
+    https://github.com/rtfd/readthedocs.org/issues/1826). Once that bug is
+    fixed, the file can be removed.
+-->
+<html lang="en-US">
+    <head>
+        <meta charset="UTF-8">
+        <meta http-equiv="refresh" content="0;url=how-it-works/#execution-life-cycle">
+        <script type="text/javascript">
+            window.location.href = 'how-it-works/#execution-life-cycle' + window.location.hash
+        </script>
+        <title>Page Redirection</title>
+    </head>
+    <body>
+        <p>
+            The page for Dredd usage no longer exists. Please follow
+            the following link:
+        </p>
+        <ul>
+            <li><a href="how-it-works/#execution-life-cycle">How It Works</a></li>
+        </ul>
+    </body>
+</html>
diff --git a/docs/execution-lifecycle.md b/docs/execution-lifecycle.md
diff --git a/docs/execution-lifecycle/index.html b/docs/execution-lifecycle/index.html
@@ -0,0 +1,27 @@
+<!DOCTYPE HTML>
+<!--
+    This file exists only to redirect from old URL to new URL.
+    This is more properly done using an HTTP redirect, but unfortunately
+    readthedocs.org's page redirection is broken (see
+    https://github.com/rtfd/readthedocs.org/issues/1826). Once that bug is
+    fixed, the file can be removed.
+-->
+<html lang="en-US">
+    <head>
+        <meta charset="UTF-8">
+        <meta http-equiv="refresh" content="0;url=../how-it-works/#execution-life-cycle">
+        <script type="text/javascript">
+            window.location.href = '../how-it-works/#execution-life-cycle' + window.location.hash
+        </script>
+        <title>Page Redirection</title>
+    </head>
+    <body>
+        <p>
+          The page for Dredd usage no longer exists. Please follow
+          the following link:
+        </p>
+        <ul>
+            <li><a href="../how-it-works/#execution-life-cycle">How It Works</a></li>
+        </ul>
+    </body>
+</html>
diff --git a/docs/hooks.md b/docs/hooks.md
@@ -19,13 +19,13 @@ Hooks are usually used for:
 
 You can interact with your server implementation in following languages:
 
-- [Ruby](hooks-ruby.md)
-- [Python](hooks-python.md)
-- [Node.js](hooks-nodejs.md)
-- [PHP](hooks-php.md)
 - [Go](hooks-go.md)
+- [JavaScript (Sandboxed)](hooks-js-sandbox.md)
+- [Node.js](hooks-nodejs.md)
 - [Perl](hooks-perl.md)
-- [Sandboxed JavaScript](hooks-js-sandbox.md)
+- [PHP](hooks-php.md)
+- [Python](hooks-python.md)
+- [Ruby](hooks-ruby.md)
 
 Dredd doesn't speak your language? [**It's very easy to write support for your language.**](hooks-new-language.md) Your contribution is more than welcome!
 
@@ -73,7 +73,8 @@ $ dredd single_get.md http://machines.apiary.io --names
 info: Machines > Machines collection > Get Machines
 ```
 
-The `Machines > Machines collection > Get Machines` is the name of a transaction which you can use in your hooks.
+The `Machines > Machines collection > Get Machines` is the name of a transaction which you can use in your hooks. The same approach works also for Swagger documents.
+
 See [Hooks JavaScript API Reference](#hooks-javascript-api-reference) for broader information of how is it used.
 
 ## Types of Hooks

diff --git a/docs/how-it-works.md b/docs/how-it-works.md
@@ -0,0 +1,167 @@
+# How It Works
+
+In a nutshell, Dredd does following:
+
+1. Takes your API description document,
+2. creates expectations based on requests and responses documented in the document,
+3. makes requests to tested API,
+4. checks whether API responses match the documented responses,
+5. reports the results.
+
+## Versioning
+
+Dredd follows [Semantic Versioning][]. To ensure certain stability of your Dredd installation (e.g. in CI), pin the version accordingly. You can also use release tags:
+
+- `npm install dredd` - Installs the latest published version including experimental pre-release versions.
+- `npm install dredd@stable` - Skips experimental pre-release versions. Recommended for CI installations.
+
+If the `User-Agent` header isn't overridden in the API description document, Dredd uses it for sending information about its version number along with every HTTP request it does.
+
+## Execution Life Cycle
+
+Following execution life cycle documentation should help you to understand how Dredd works internally and which action goes after which.
+
+1. Load and parse API description documents
+    - Report parse errors and warnings
+2. Pre-run API description check
+    - Missing example values for URI template parameters
+    - Required parameters present in URI
+    - Report non-parseable JSON bodies
+    - Report invalid URI parameters
+    - Report invalid URI templates
+3. Compile HTTP transactions from API description documents
+    - Inherit headers
+    - Inherit parameters
+    - Expand URI templates with parameters
+4. Load [hooks](hooks.md)
+5. Test run
+    - Report test run `start`
+    - Run `beforeAll` hooks
+    - For each compiled transaction:
+        - Report `test start`
+        - Run `beforeEach` hook
+        - Run `before` hook
+        - Send HTTP request
+        - Receive HTTP response
+        - Run `beforeEachValidation` hook
+        - Run `beforeValidation` hook
+        - [Perform validation](#automatic-expectations)
+        - Run `after` hook
+        - Run `afterEach` hook
+        - Report `test end` with result for in-progress reporting
+    - Run `afterAll` hooks
+6. Report test run `end` with result statistics
+
+## Limitations of API Description Formats
+
+## Automatic Expectations
+
+Dredd automatically generates expectations on HTTP responses based on examples in the API description with use of [Gavel.js][] library. Please refer to [Gavel][] rules if you want know more.
+
+### Response Headers Expectations
+
+- All headers specified in the API description must be present in the response.
+- Names of headers are validated in case-insensitive way.
+- Only values of headers significant for content negotiation are validated.
+- All other headers values can differ.
+
+When using [Swagger][], headers are taken from [`response.headers`](http://swagger.io/specification/#responseHeaders). If document specifies `produces`, it is propagated as response's `Content-Type` header.
+
+### Response Body Expectations
+
+By default Dredd validates HTTP response body against [JSON Schema][] inferred from the API description under test. The effective JSON Schema is taken from following places (the order goes from the highest priority to the lowest):
+
+#### API Blueprint
+
+1. `+ Schema` section - provided custom JSON Schema (draft v4 or v3) will be used.
+2. `+ Attributes` section with data structure description in [MSON][] - API Blueprint parser automatically generates JSON Schema from MSON.
+3. `+ Body` section with sample JSON payload - [Gavel.js][], which is responsible for validation in Dredd, automatically infers some basic expectations described below.
+
+This order [exactly follows the API Blueprint specification](https://github.com/apiaryio/api-blueprint/blob/master/API%20Blueprint%20Specification.md#relation-of-body-schema-and-attributes-sections).
+
+#### Swagger
+
+1. [`response.schema`](http://swagger.io/specification/#responseSchema) - provided JSON Schema will be used.
+2. [`response.examples`](http://swagger.io/specification/#responseExamples) with sample JSON payload - [Gavel.js][], which is responsible for validation in Dredd, automatically infers some basic expectations described below.
+
+#### Gavel's Expectations
+
+- All JSON keys on any level given in the sample must be present in the response's JSON.
+- Response's JSON values must be of the same JSON primitive type.
+- All JSON values can differ.
+- Arrays can have additional items, type or structure of the items is not validated.
+- Plain text must match perfectly.
+
+### Custom Expectations
+
+You can make your own custom expectations in [hooks](hooks.md). For instance, check out how to employ [Chai.js assertions](hooks.md#using-chai-assertions).
+
+## Making Your API Description Ready for Testing
+
+It's very likely that your API description document will not be testable __as is__. This section should help you to learn how to solve the most common issues.
+
+### URI Parameters
+
+Both [API Blueprint][] and [Swagger][] allow usage of URI templates. In order to have an API description which is testable, you need to describe all parameters used in URI (path or query) and provide sample values to make Dredd able to expand URI templates with given sample values.
+
+TODO!!! ...
+
+### Request Headers
+
+TODO!!! ...
+
+#### Swagger
+
+When using [Swagger][], there is no way how to make Dredd send specific HTTP headers except of those significant for content negotiation. This is a known limitation of the Swagger format.
+
+> **Note:** Please mind that Swagger's `"in": "header"` parameter doesn't provide any way to specify HTTP header name. The meaning of the `name` property is _name of the parameter_, not _name of the header_.
+
+- `consumes` is propagated as request's `Content-Type` header.
+- `produces` is propagated as request's `Accept` header.
+- Response's `Content-Type` header overrides any `consumes`.
+
+### Request Body
+
+TODO!!! ...
+
+#### Swagger
+
+Request body is inferred from [operation's `"in": "body"` and `"in": "form"` parameters](http://swagger.io/specification/#parameterObject). Since Swagger doesn't provide any way to specify raw JSON sample for the request body, Dredd's [Swagger Adapter][] generates some sample values from JSON Schema provided in the [`schema`](http://swagger.io/specification/#parameterSchema) property. Following rules apply when the adapter fills values of the properties, ordered by precedence:
+
+1. Value of `default`.
+2. First value from `enum`.
+3. Dummy, generated value.
+
+### How Dredd Works With HTTP Transactions
+
+#### Multiple Requests and Responses (API Blueprint)
+
+TODO!!! ...
+
+#### Default Response (Swagger)
+
+TODO!!! ...
+
+#### Responses with Status Code Other than 2xx (Swagger)
+
+TODO!!! ...
+
+#### Supported Media Types in `produces` and `consumes` (Swagger)
+
+TODO!!! ...
+
+#### Limitations of the Swagger Format
+
+TODO!!! ...
+- Responses with the same Content Type & status code but different examples.
+- Requests with the same Content Type but different examples.
+
+
+[Semantic Versioning]: http://semver.org/
+[API Blueprint]: http://apiblueprint.org/
+[Swagger]: http://swagger.io/
+[Gavel.js]: https://github.com/apiaryio/gavel.js
+[Gavel]: https://www.relishapp.com/apiary/gavel/docs
+[MSON]: https://github.com/apiaryio/mson
+[JSON Schema]: http://json-schema.org/
+[Swagger Adapter]: https://github.com/apiaryio/fury-adapter-swagger/