Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add minimal windows support #3

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@ name: test
on: [push, pull_request]
jobs:
test:
name: Node ${{ matrix.node-version }} and ${{ matrix.os }}
name: Node ${{ matrix.node-version }}, ${{ matrix.os }}, and avoid fdpass ${{ matrix.avoid-fdpass }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
node-version: [10.x, 12.x, 14.x, 15.x]
os: [ubuntu-latest, macos-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
avoid-fdpass: [0,1]
steps:
- uses: actions/checkout@v1
- name: Use Node ${{ matrix.node-version }}
Expand All @@ -19,5 +20,7 @@ jobs:
run: npm i
- name: npm test
run: npm test
env:
MOJO_SERVER_STARTER_AVOID_FDPASS: ${{ matrix.avoid-fdpass }}
- name: npm run lint
run: npm run lint
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
node_modules
package-lock.json

.vscode
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to the rest of the PR.

87 changes: 84 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
# server-starter [![](https://github.com/mojolicious/server-starter/workflows/test/badge.svg)](https://github.com/mojolicious/server-starter/actions)

UNIX superdaemon with support for socket activation.
UNIX, MacOS and Windows platforms superdaemon with support for socket activation.

## Description

This module exists to handle socket activation for TCP servers running in separate processes on UNIX. It is capable of
This module exists to handle socket activation for TCP servers running in separate processes on different platforms. It is capable of
assigning random ports to avoid race conditions when there are many services running in parallel on the same machine.
As is common with large scale testing.

The superdaemon will create the listen socket and pass it to the managed process as `fd=3`, similar to how `systemd`
On UNIX / MacOS platforms the superdaemon will create the listen socket and pass it to the managed process as `fd=3`, similar to how `systemd`
handles socket activation. This also avoids any race conditions between spawning the managed process and sending the
first request, since the listen socket is active the whole time.

For Windows platforms, read also ```Launching servers / platforms without file descriptor pass support``` below.

```js
const http = require('http');

Expand Down Expand Up @@ -77,6 +79,85 @@ t.test('Test the WebSocket chat', async t => {
await server.close();
});
```
## Launching servers / platforms without file descriptor pass support.

This module can be used with standard listening address, for platforms not supporting
file description passing (like windows), or servers that can't reuse sockets passed
as file descriptors.

- Portable listening address

You can build a portable listening address using the ```listenAddress()``` function on ```server``` object. That function will return an absolute url that you can use to configure your server in a portable way.

It will either be the string ```http://*?fd=3``` if file description pass is
allowed, or have a format ```http://<address>:<port>``` that you can use as a listening address or parse it to get the parameters needed by your server (address and port).

- on your test script:
```js
...
const server = await starter.newServer();
await server.launch('node', ['your/path/server.js', server.listenAdress]);
...
```

- then on your server (```your/path/server.js``` file) you will get the listenAdress as a command parameter:
```js
// called as node server.js <listening address>

const http = require('http');
let listen = {fd: 3};
let parts = process.argv[2].match(/http:\/\/([^\/]+):(\d+)/);
if (parts) listen = {port: parts[2], address: parts[1]};
// at this point variable listen will either be {fd: 3} or {port: <port>, address: <address>},
// dependint on the first command argument (process.argv[2])

const server = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello World!');
});
server.listen(listen);
```
Note that depending on the server application required format, listenAddress() string could be exactly all you need, like in the case of a Mojolicious app, to load it in a portable way you just use the returned string as the '-l' command argument:

```js
...
const server = await starter.newServer();
await server.launch('perl', ['your/path/app.pl', '-l', server.listenAdress]);
...
```

- Avoid usage of file description passing of listening socket

You can use the ENV variable ```MOJO_SERVER_STARTER_AVOID_FDPASS```:

```shell
export MOJO_SERVER_STARTER_AVOID_FDPASS=1
```

Default value is 0, and will use fd passing whenever is possible (i.e. except for windows platforms)

- Configurable timeout

When not using fd passing, there is a timeout to wait for the server to start listening. You configure it as option ```connectTimeout```, in mS, when calling the launch() function:

```js
const server = await starter.newServer();
await server.launch(<cmd>, <args>, {connectTimeout: 3000});
```

Default value is 30000 (30 secs).
This parameter has no effect when socket is passed through file descriptor (in that case waiting for the server is not necessary)

- Configurable retry time

When not using fd passing, the launch() function will check if the server is listening every <retryTime> mS. You can configure it as an option:

```js
const server = await starter.newServer();
await server.launch(<cmd>, <args>, {retryTime: 250});
```
Default value is 60 (60 mS).
This parameter has no effect when socket is passed through file descriptor (in that case waiting for the server is not necessary)

## Install

Expand Down
37 changes: 33 additions & 4 deletions lib/server-starter.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ class Server extends EventEmitter {
this._process = undefined;
this._exitHandlers = [];
this._exit = undefined;
this._useFdPass = process.env.MOJO_SERVER_STARTER_AVOID_FDPASS ? false : process.platform !== 'win32';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No Mojo specific code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And let the users decide if they want to use a file descriptor or not. Hidden magic only makes it harder to write portable code.

}

/**
Expand All @@ -64,15 +65,18 @@ class Server extends EventEmitter {
* @param {object} [options] - Optional settings
* @param {boolean} [options.stdout=false] - Forward server output from STDOUT to STDOUT
* @param {boolean} [options.stderr=true] - Forward server output from STDERR to STDERR
* @param {number} [options.connectTimeout=30000] - Max time to wait for server ready, in mS
* @param {number} [options.retryTime=60] - Time to retry for server ready, in mS
* @returns {Promise}
*/
launch (cmd, args, options = {}) {
if (typeof this._process !== 'undefined') throw new Error('Server already launched');
const stdout = typeof options.stdout !== 'undefined' ? options.stdout : false;
const stderr = typeof options.stderr !== 'undefined' ? options.stderr : true;

const proc = (this._process = spawn(cmd, args, { stdio: ['pipe', 'pipe', 'pipe', this._fd] }));

const connectTimeout = typeof options.connectTimeout !== 'undefined' ? options.connectTimeout : 30000;
const retryTime = typeof options.retryTime !== 'undefined' ? options.retryTime : 60;
const spawnOptions = this._useFdPass ? { stdio: ['pipe', 'pipe', 'pipe', this._fd] } : undefined;
const proc = (this._process = spawn(cmd, args, spawnOptions));
proc.on('error', e => this.emit('error', e));
if (stdout) proc.stdout.pipe(process.stdout);
if (stderr) proc.stderr.pipe(process.stderr);
Expand All @@ -86,7 +90,23 @@ class Server extends EventEmitter {
this.emit('exit', code, signal);
});

return new Promise(resolve => this._srv.close(resolve));
return new Promise(resolve => this._srv.close(
() => {
if (this._useFdPass) resolve();
else {
const now = new Date();
const timeToStop = new Date(now.getTime() + connectTimeout);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even my feedback on the unused now constant from the previous PR you've just dismissed without changes.

const port = this.port;
(function loop () {
const connection = net.connect(port, resolve);
connection.on('error', err => {
if (err.code === 'ECONNREFUSED' && new Date() < timeToStop) setTimeout(loop, retryTime);
else resolve(); // this is intented: don't reject, just stop delaying
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of my requirements was that it rejects on errors. Are you sure you don't just want to make your own module?

});
})();
}
}
));
}

/**
Expand Down Expand Up @@ -130,6 +150,15 @@ class Server extends EventEmitter {
const port = this.port;
return `http://${address}:${port}`;
}

/**
* Listen Address to configure service to be launched
* @returns {string}
*/
listenAddress () {
if (this._useFdPass) return 'http://*?fd=3';
return this.url();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're still missing the point. This module is not meant to be Mojolicious specific at all. This could be a generic feature that any other web framework could use just the same.

}

exports = module.exports = new ServerStarter();
Expand Down
5 changes: 1 addition & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,5 @@
},
"engines": {
"node": ">= 10"
},
"os": [
"!win32"
]
}
}
37 changes: 34 additions & 3 deletions test/start_fd.js → test/start.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ const starter = require('..');
t.test('Start and stop a server', async t => {
const server = await starter.newServer();
t.equal(server.pid, null, 'not started');
await server.launch('node', ['test/support/server_fd.js']);
await server.launch('node', ['test/support/server.js', server.listenAddress()]);
t.equal(typeof server.pid, 'number', 'started');
const url = server.url();
t.equal(typeof server.port, 'number', 'port assigned');
Expand All @@ -31,7 +31,7 @@ t.test('Start and stop a server', async t => {
t.test('Do it again', async t => {
const server = await starter.newServer();
t.equal(server.pid, null, 'not started');
await server.launch('node', ['test/support/server_fd.js']);
await server.launch('node', ['test/support/server.js', server.listenAddress()]);
t.equal(typeof server.pid, 'number', 'started');

const res = await fetch(server.url());
Expand All @@ -44,11 +44,42 @@ t.test('Do it again', async t => {
t.equal(server.pid, null, 'stopped');
});

t.test('Slow server', async t => {
const server = await starter.newServer();
t.equal(server.pid, null, 'not started');
await server.launch('node', ['test/support/server.js', server.listenAddress(), 1000]);
t.equal(typeof server.pid, 'number', 'started');

const res = await fetch(server.url());
t.equal(res.ok, true, '2xx code');
t.equal(res.headers.get('Content-Type'), 'text/plain', 'right "Content-Type" header');
const buffer = await res.buffer();
t.equal(buffer.toString('utf8'), 'Hello World!', 'right content');

await server.close();
t.equal(server.pid, null, 'stopped');
});

t.test('Slow server, with wrong (too small) timeout', { skip: !process.env.MOJO_SERVER_STARTER_AVOID_FDPASS && process.platform !== 'win32' }, async t => {
const server = await starter.newServer();
t.equal(server.pid, null, 'not started');
await server.launch('node', ['test/support/server.js', server.listenAddress(), 3000], { connectTimeout: 500 });
t.equal(typeof server.pid, 'number', 'started');

let err;
try { await fetch(server.url()); } catch (e) { err = e; }
t.ok(err, 'request failed');
t.equal(err.errno, 'ECONNREFUSED', 'right error');

await server.close();
t.equal(server.pid, null, 'stopped');
});

t.test('Use a specific port', async t => {
const port = await getPort();
const server = await starter.newServer(port);
t.equal(server.pid, null, 'not started');
await server.launch('node', ['test/support/server_fd.js']);
await server.launch('node', ['test/support/server.js', server.listenAddress()]);
t.equal(typeof server.pid, 'number', 'started');
t.equal(server.port, port, 'right port');

Expand Down
21 changes: 21 additions & 0 deletions test/support/server.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
'use strict';

// Usage: node server-starter.js <listen address> <listen delay (mS)>
// <listen address> is mandatory

const http = require('http');
const delay = process.argv[3] ? process.argv[3] : 0;
let listen = {fd: 3};
let parts = process.argv[2].match(/http:\/\/([^\/]+):(\d+)/);
if (parts) listen = {port: parts[2], address: parts[1]};
const server = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello World!');
});

// delayed start listening
(() => new Promise(resolve => {
setTimeout(
() => resolve(server.listen(listen)),
delay)
}))();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had renamed the test server to server_fd.js the other day specifically so the code doesn't have to get messy for new features.

9 changes: 0 additions & 9 deletions test/support/server_fd.js

This file was deleted.