Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wait for tcp port available (win32) #2

Open
wants to merge 39 commits into
base: windows
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
ed21f8d
Link to blog post
kraih Mar 28, 2021
35cb5c8
Use getters for pid and port
kraih Mar 28, 2021
0d9dd39
Small documentation improvements
kraih Mar 29, 2021
c7cf6c7
Use a little less code
kraih Mar 29, 2021
34bc5e6
The index.js file is not needed
kraih Mar 29, 2021
5c589fd
Bump version
kraih Mar 29, 2021
fbd8076
newServer returns a Promise
kraih Mar 30, 2021
5b75dcd
wait for tcp port available
dmanto Mar 31, 2021
d1bf330
Enable mergify
kraih Mar 31, 2021
5cf785d
Make room for port based tests
kraih Mar 31, 2021
a58db73
Test port assignment
kraih Mar 31, 2021
425fcde
Give the test a little more time to close the port
kraih Apr 1, 2021
7865e33
Wait for the first listen socket to be closed
kraih Apr 1, 2021
cccc7ed
add listenAddress attribute
dmanto Apr 3, 2021
9aef07f
avoid fdpass through env var
dmanto Apr 4, 2021
bc67f2d
fix lint errors
dmanto Apr 4, 2021
1f0f1af
add slow server tests
dmanto Apr 4, 2021
3ef6fc5
fix lint again
dmanto Apr 4, 2021
3493b4b
modify README for non fd passing cases
dmanto Apr 4, 2021
e99fd6e
add minimal windows support
dmanto Apr 4, 2021
030cdbf
small fixes
dmanto Apr 5, 2021
8663e04
add avoidFdPassing option to launch funtion
dmanto Apr 5, 2021
08d5c04
remove listenAddress() function, improve README
dmanto Apr 5, 2021
18c0dd2
add race condition caveat
dmanto Apr 5, 2021
8b1b9f7
fix typos and rephrase race condition caveat
dmanto Apr 5, 2021
293263c
fix typos
dmanto Apr 5, 2021
47797fe
add windows workflow, but skiping start_fd.js tests in this case
dmanto Apr 5, 2021
c850246
reorganize README
dmanto Apr 5, 2021
b8d7ef2
manual merge into former branch
dmanto Apr 5, 2021
2d204f4
fix workflow
dmanto Apr 5, 2021
b05e0d8
leave server_fd.js as in master branch
dmanto Apr 5, 2021
9140764
fix typo
dmanto Apr 5, 2021
45f29f3
add a launchPortable() function instead of checking avoidFdPassing op…
dmanto Apr 6, 2021
e526c69
fix README intralink
dmanto Apr 6, 2021
8ab9dd8
more tests on portable mode
dmanto Apr 6, 2021
cf15c04
fix node 10.x windows cannot find module error
dmanto Apr 6, 2021
575e5d7
fix bad port test on windows node 10.x
dmanto Apr 6, 2021
4fbb40b
fix bad port test, try harder
dmanto Apr 6, 2021
054e412
skip some tests on node 10.x (windows), seems to have error msgs broken
dmanto Apr 7, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
node_modules
package-lock.json

20 changes: 20 additions & 0 deletions .mergify/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
pull_request_rules:
- name: automatic merge
conditions:
- "#approved-reviews-by>=2"
- "#changes-requested-reviews-by=0"
- base=master
actions:
merge:
method: merge
- name: remove outdated reviews
conditions:
- base=master
actions:
dismiss_reviews: {}
- name: ask to resolve conflict
conditions:
- conflict
actions:
comment:
message: This pull request is now in conflicts. Could you fix it @{{author}}? 🙏
69 changes: 64 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
# server-starter [![](https://github.com/mojolicious/server-starter/workflows/test/badge.svg)](https://github.com/mojolicious/server-starter/actions)

UNIX superdaemon with support for socket activation.
UNIX, MacOS and Windows platforms superdaemon with support for socket activation.

## Description

This module exists to handle socket activation for TCP servers running in separate processes on UNIX. It is capable of
This module exists to handle socket activation for TCP servers running in separate processes on different platforms. It is capable of
assigning random ports to avoid race conditions when there are many services running in parallel on the same machine.
As is common with large scale testing.

The superdaemon will create the listen socket and pass it to the managed process as `fd=3`, similar to how `systemd`
On UNIX / MacOS platforms the superdaemon will create the listen socket and pass it to the managed process as `fd=3`, similar to how `systemd`
handles socket activation. This also avoids any race conditions between spawning the managed process and sending the
first request, since the listen socket is active the whole time.

For Windows platforms, read also [Launching servers / platforms without file descriptor pass support](#Launching-servers-without-file-descriptor-pass-support) below.

```js
const http = require('http');

Expand All @@ -22,7 +24,7 @@ const server = http.createServer((req, res) => {
server.listen({ fd: 3 });
```

All the web application has to do is use `fd3` as its listen socket to accept new connections from.
All the web application has to do is use `fd=3` as its listen socket to accept new connections from.

```js
const starter = require('@mojolicious/server-starter');
Expand All @@ -42,7 +44,8 @@ const fetch = require('node-fetch');
```

The managed TCP server does not need to be a Node application. In fact this module was originally developed to test
[Mojolicious](https://mojolicious.org) web applications written in Perl with [Playwright](https://playwright.dev).
[Mojolicious](https://mojolicious.org) web applications written in Perl with [Playwright](https://playwright.dev). For
more details take a look at the [blog post](https://dev.to/kraih/playwright-and-mojolicious-21hn).

```js
const t = require('tap');
Expand Down Expand Up @@ -76,7 +79,63 @@ t.test('Test the WebSocket chat', async t => {
await server.close();
});
```
## Launching servers without file descriptor pass support

This module can be used with other ways to pass the listening address, for platforms not supporting
file description passing (like windows), or servers that can't reuse sockets passed
as file descriptors.

Note that when using this option the module does not avoid the race condition mentioned in the [Description](#description) paragraph. The option exists just because it would allow your colaborators to be able to run your tests in Windows platforms without needing to switch to WSL.

Just as an example, suppose you have a simple js server that will listen in a port passed as a parameter:

```js
// called as node server.js <port>
const http = require('http');

const server = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello World!');
});
// take the port value from the first command line argument (process.argv[2])
server.listen({ port: process.argv[2] });
```
To avoid passing the listen socket with a file descriptor, you have to launch your server using the ```launchPortable()``` function:

```js
const starter = require('@mojolicious/server-starter');
const fetch = require('node-fetch');

(async () => {
const server = await starter.newServer();
await server.launchPortable('node', ['server.js', server.port]);
const url = server.url();

const res = await fetch(url);
const buffer = await res.buffer();
console.log(buffer.toString('utf8'));

await server.close();
```

Note that depending on the acttual command line your server application needs to be started, either ```server.url()``` returned string or ```server.port``` could be exactly all you need to configure as a parameter when calling the ```launchPortable()``` function.
## Configurable timers

```launchPortable()``` promise will not resolve until it can be verified that the launched server is actually listening. This behavior is controlled by two timers:

- ```connectTimeout```, in mS, allows to configure maximum time to wait for the launched server to start listening. Default is 30000 (30 secs).

```js
const server = await starter.newServer();
await server.launchPortable(<cmd>, <args>, { connectTimeout: 3000 });
```

- ```retryTime```, in mS, allows to configure the time to retry a connection with the launched server. Default is 60 (60 mS).

```js
const server = await starter.newServer();
await server.launchPortable(<cmd>, <args>, { retryTime: 250 });
```
## Install

$ npm i @mojolicious/server-starter
8 changes: 0 additions & 8 deletions index.js

This file was deleted.

86 changes: 68 additions & 18 deletions lib/server-starter.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@ const net = require('net');
const { spawn } = require('child_process');

/**
* Class representing a superdaemon that can manage one or more servers
* Class representing a superdaemon that handles socket activation and can manage one or more servers
*/
class ServerStarter {
/**
* Create a server
* @param {number} [port=0] - Optional port to listen on
* @param {string} [address] - Optional address to listen on
* @returns {Server}
* @returns {Promise<Server>}
*/
newServer (port, address) {
const server = new Server(port, address);
Expand All @@ -38,7 +38,7 @@ class Server extends EventEmitter {
super();
this._originalPort = port || 0;
this._originalAddress = address;
this.port = undefined;
this._port = undefined;
this._fd = undefined;
this._srv = undefined;
this._process = undefined;
Expand Down Expand Up @@ -72,25 +72,67 @@ class Server extends EventEmitter {
const stderr = typeof options.stderr !== 'undefined' ? options.stderr : true;

const proc = (this._process = spawn(cmd, args, { stdio: ['pipe', 'pipe', 'pipe', this._fd] }));
this._srv.close();

proc.on('error', e => this.emit('error', e));
proc.stdout.on('data', data => {
if (stdout) process.stdout.write(data.toString('utf8'));
this.emit('stdout', data);
});
proc.stderr.on('data', data => {
if (stderr) process.stderr.write(data.toString('utf8'));
this.emit('stderr', data);
});
if (stdout) proc.stdout.pipe(process.stdout);
if (stderr) proc.stderr.pipe(process.stderr);
proc.stdout.on('data', data => this.emit('stdout', data));
proc.stderr.on('data', data => this.emit('stderr', data));

proc.on('exit', (code, signal) => {
this._exit = true;
this._process = undefined;
this._exitHandlers.forEach(item => item());
this.emit('exit', code, signal);
});

// Should be resolved by the "spawn" event, but that is only supported in Node 15+
return Promise.resolve;
return new Promise(resolve => this._srv.close(resolve));
}

/**
* Launch server in "portable" mode (i.e., not passing a fd)
* @param {string} cmd - Server command to run
* @param {string[]} args - Arguments to use with server command
* @param {object} [options] - Optional settings
* @param {boolean} [options.stdout=false] - Forward server output from STDOUT to STDOUT
* @param {boolean} [options.stderr=true] - Forward server output from STDERR to STDERR
* @param {number} [options.connectTimeout=30000] - Max time to wait for server ready, in mS
* @param {number} [options.retryTime=60] - Time to retry for server ready, in mS
* @returns {Promise}
*/
launchPortable (cmd, args, options = {}) {
if (typeof this._process !== 'undefined') throw new Error('Server already launched');
const stdout = typeof options.stdout !== 'undefined' ? options.stdout : false;
const stderr = typeof options.stderr !== 'undefined' ? options.stderr : true;
const connectTimeout = typeof options.connectTimeout !== 'undefined' ? options.connectTimeout : 30000;
const retryTime = typeof options.retryTime !== 'undefined' ? options.retryTime : 60;

return new Promise((resolve, reject) => this._srv.close(
() => {
const proc = (this._process = spawn(cmd, args));
proc.on('error', e => this.emit('error', e));
if (stdout) proc.stdout.pipe(process.stdout);
if (stderr) proc.stderr.pipe(process.stderr);
proc.stdout.on('data', data => this.emit('stdout', data));
proc.stderr.on('data', data => this.emit('stderr', data));

proc.on('exit', (code, signal) => {
this._exit = true;
this._process = undefined;
this._exitHandlers.forEach(item => item());
this.emit('exit', code, signal);
});
const timeToStop = new Date(Date.now() + connectTimeout);
const port = this.port;
(function loop () {
const connection = net.connect(port, resolve);
connection.on('error', err => {
if (err.code === 'ECONNREFUSED' && new Date() < timeToStop) setTimeout(loop, retryTime);
else reject(err);
});
})();
}
));
}

/**
Expand All @@ -102,7 +144,7 @@ class Server extends EventEmitter {
const srv = net.createServer();
srv.listen(this._originalPort, this._originalAddress, () => {
this._srv = srv;
this.port = srv.address().port;
this._port = srv.address().port;
this._fd = srv._handle.fd;
resolve();
});
Expand All @@ -111,14 +153,22 @@ class Server extends EventEmitter {

/**
* PID of the launched process
* @returns {number|null}
* @type {number|null}
*/
pid () {
get pid () {
return this._process ? this._process.pid : null;
}

/**
* URL of the launched server
* Port of activated socket
* @type {number}
*/
get port () {
return this._port;
}

/**
* Full URL of the launched server with "http" scheme
* @returns {string}
*/
url () {
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@mojolicious/server-starter",
"version": "1.0.12",
"version": "1.0.18",
"description": "UNIX superdaemon with support for socket activation",
"keywords": [
"superdaemon",
Expand Down
45 changes: 0 additions & 45 deletions test/start.js

This file was deleted.

Loading