Running dredd itself and hook server in separate docker containers

[honzatrtik]

Hi, first of all, thanks for dredd, it is great tool!

Im trying to start my hook tcp server in separate docker container and then run dredd tool itself in another container and link them together. The motivation is that I need to bootstrap our app in hook file, create some fixtures etc. but dont want to bloat our production image with dredd.
My idea was something like:

docker run -d --network dredd --name hook-server my-app-image php start-server.php hooks.php  --listen "0.0.0.0" --port 61321
docker run --rm --network dredd -v "/blueprint.apib:/blueprint.apib"--name dredd dred-image dredd /blueprint.apib http://api --hooks-worker-handler-port 61321 --hooks-worker-handler-host hook-server

Is something like this supposed to work? Im not sure, if hook server must be started by dredd itself...

Thanks in advance!

[w-vi]

At the moment the hook server is started by dredd but you can at least in theory pass dredd the custom hook handler command in your case the docker run <your hook server> and dredd will start it. I haven't tried it myself yet but was thinking along these lines. I am happy to help you get this setup running.

[w-vi]

Some info is available in the relevant part of the docuemntation http://dredd.readthedocs.io/en/latest/hooks-new-language/#writing-dredd-hook-handler-for-new-language

[honzatrtik]

but you can at least in theory pass dredd the custom hook handler command in your case the docker run

Im afraid, this is not possible, because you can not run docker container inside docker container (it is actually possible but not in our case), anyway - maybe I can try to pass some dummy command, which will output string "Starting"... I will try and lat you know.
Thanks for the answer!

[w-vi]

I see, In that case I'd just put some script which would run and stop on SIGTERM, something along these lines should suffice.

#!/bin/bash
function term() {
    echo 'Caught SIGTERM, exiting.'
    exit 0
}

trap 'term' SIGTERM

while true
do
   sleep 1
done

[w-vi]

Look at the default values of all the related to hook handlers settings so that you listen on correct port and so on http://dredd.readthedocs.io/en/latest/usage-cli/#-hooks-worker-after-connect-wait

[honzajavorek]

@honzatrtik What @w-vi wrote you is exactly what we do in tests. We use dummy script (e.g. this) as a value for the --language option, so Dredd thinks it's been able to successfully start/run/terminate a hook handler, and then we run a mock hook handler for testing purposes on hook handler port.

So this should work perfectly. Feed Dredd with dummy script and expose the hook handler on correct port. You will just need to start and stop the hook handler yourself.

I'll close this as solved, but please come back and comment in case it doesn't work, so we can re-open and find a solution 🙂 And thanks for using Dredd and for bringing this up! 👍

[martijnarts]

Hey, it'd be super sweet if there was a native solution for this. We're running into this as well now. Maybe null could be an argument to language? The spawnHandler and terminateHandler calls could just be bypassed in that case.

It seems kind of pointless to have the whole handler host/port configuration if Dredd then wants to set it up itself anyway, right?

I was about to create a different issue for this, but this is probably a more appropriate place.

[honzajavorek]

@TotempaaltJ Actually, I think a new issue linking this one would be nice! I agree it's a good idea to give a standard option to bypass the handler logic.

[martijnarts]

Alright I created #755 ^

[ApiaryBot]

🎉 This issue has been resolved in version 5.4.2 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[rayzeller]

Does anyone have an example dredd.yml to achieve the above bypass? included my docker-compose.yml and dredd.yml, and sleep.js but I can't seem to get it working.

https://gist.github.com/rayzeller/255fbbf88f9686b3062e84a67dbfa8fa
https://gist.github.com/rayzeller/77d0df2c33be05a064257590081a749e
https://gist.github.com/rayzeller/eaf49b661f5bbb6fa1b723972193ec81

Current error:
When i run dredd-hooks-ruby ./hooks.rb on api, then
docker-compose up dreddhooks, my tests run, but it doesn't hit the hooks. I have a bunch of logging in place that doesn't hit. And it also doesn't throw any error when the server's not running at all.

It seems that dredd is bypassing any connection to the hook server when passing the sleep.js bypass into the yml file. Was this issue possibly introduced in a new version in the past 1.5 years?

[honzajavorek]

Would --hooks-worker-handler-host and --hooks-worker-handler-port help you?

[rayzeller]

@honzajavorek i have those set in my dredd.yml. does dredd not always respect the yml? should i put them in command line?

hooks-worker-handler-host: http://api
hooks-worker-handler-port: 61321

[honzajavorek]

No, having them in YAML should be enough. Are you able to see anything interesting using --loglevel=debug?

[rayzeller]

AH. i see it now. correct me if i'm wrong, but It looks like if nodejs is the language, then dredd uses the current filesystem for the hooks file. and does not (always?) start a server?

dredd/lib/addHooks.js

Line 56 in 8d517ac

if (!runner.configuration.options.language

anyway, i was able to get hooks to register by changing my sleep.js to the following:

var ignore;

ignore = function() {
  return console.log('ignoring termination');
};

process.on('SIGINT', ignore);


const hooks = require('hooks');

//example hook just to get things going!
hooks.before('auth > Facebook Auth > Callback (Get) > 200 > application/json', (transaction) => {
  transaction.request.headers.Authorization = 'Basic: YWxhZGRpbjpvcGVuc2VzYW1l';
});

setInterval((function() {}), 1000);

Meaning. the hooks server doesn't get called at all.

[honzajavorek]

If the language is nodejs, there is no hook handler started, because Dredd can handle it within its own process in more efficient way, that's correct. In your original post you mentioned Ruby, so I replied in that context. For Ruby Dredd does try to spawn a separate hook handler process.

If you're about to use the JavaScript hooks, this should be enough to get you going:

const hooks = require('hooks');

//example hook just to get things going!
hooks.before('auth > Facebook Auth > Callback (Get) > 200 > application/json', (transaction) => {
  transaction.request.headers.Authorization = 'Basic: YWxhZGRpbjpvcGVuc2VzYW1l';
});

Save it as hooks.js and use --hookfiles to let Dredd to load the file. The hack with endlessly running process is needed only to fool Dredd in case you're using a different language than JS and you want to run the hook handler process in a separate Docker container.

[rayzeller]

Awesome, thanks. It definitely says that in the docs, but I missed it!

[rayzeller]

For anyone else needing a quick POC for using hooks with js in docker, here's an abbreviated version:

docker-compose.yml

version: '2'

services:
  postgis:
    image: mdillon/postgis:11-alpine
    volumes:
      - postgis:/var/lib/postgresql/data
    env_file:
      - '.env'

  api:
    depends_on:
      - 'postgis'
    build: .
    ports:
      - '3000:3000'
    volumes:
      - '.:/app'
    env_file:
      - '.env'

  dredd:
    depends_on:
      - 'api'
    image: apiaryio/dredd
    volumes:
      - './dredd.yml:/dredd.yml'
      - './hooks.js:/hooks.js'

volumes:
  postgis:

dredd.yml

color: true
dry-run: null
hookfiles: hooks.js
language: nodejs
require: null
server: null
server-wait: 3
init: false
custom: {}
names: false
only: []
reporter: apiary
output: []
header: []
sorted: false
user: null
inline-errors: false
details: false
method: []
loglevel: warning
path: []
hooks-worker-timeout: 5000
hooks-worker-connect-timeout: 1500
hooks-worker-connect-retry: 500
hooks-worker-after-connect-wait: 100
hooks-worker-term-timeout: 5000
hooks-worker-term-retry: 500
hooks-worker-handler-host: http://api
hooks-worker-handler-port: 61321
config: ./dredd.yml
blueprint: http://api:3000/apidocs
endpoint: http://api:3000

hooks.js

const hooks = require('hooks');

hooks.beforeEach((transaction) => {
  console.log("YEET");
  transaction.request.headers['X-User-Id'] = '1';
  transaction.request.headers['X-User-Authentication-Token'] = 'notreal'
});

setInterval((function() {}), 1000);

and then
docker-compose up api
docker-compose up dredd

[rayzeller]

OK, it turns out the issue for me was that the hardcoded 127.0.0.1 in dredd_hooks_ruby prevented me from making a connection. changing it to 0.0.0.0 fixes that issue. will make a PR to address the hardcoding.