This document explains the actual protocol smith uses to communicate between agents.
The Transport class handles message framing and msgpack serializing on top of
whatever stream you give it. This can be tcp, pipe, or something else. The
msgpack library used has slightly extended the format to add Buffer and
undefined value types for a fuller range of node primitives. If desired
another transport can be used in place that implements the same interface.
Encoded on top of the serialzation format is reference cycles and callback functions.
Since functions can be serialized, rpc using callbacks is natural. Simply pass your callback as an argument and the other side will get a proxy function when it's deserialized. When they call that proxy function, a message will be sent back and your callback will get called with the deserialized arguments (which can include yet another callback). Since the callbacks are executed on their native side, closure variables and all other state is preserved. Callbacks functions must be called once and only once to prevent memory leaks. If the transport goes down, all pending callbacks will get called with an error object as their first argument. It's advised to use node-style callbacks in your APIs. The named function in the Agent can be called multiple times.
Functions are encoded as an object with one key, $. The value of this object is the
unique function index in the local function repository. Function keys are
integers. An example encoded function can look like {$: 3} where
remote.callbacks[3] is the real function. Numbers are reused as soon as
they are freed (when the function is called).
Cycles are also encoded as $ keyed objects. The value is the path to
the actual value as an array of strings. In this way it works like a file-
system symlink. For example. Given the following cyclic object:
var entry = {
name: "Bob",
boss: { name: "Steve" }
};
entry.self = entry;
entry.manager = entry.boss;The following encoded object is generated by the internal freeze function.
{
name: 'Bob',
boss: { name: 'Steve' },
self: { $: [] },
manager: { $: [ 'boss' ] }
}See that the path [] points to the root object itself, and ['boss'] points to
the boss property in the root.
Every RPC message is a function call. There are three kinds of function calls and they are all encoded the same way. A call is sent over the transport as a flat array. The first item is the function name or key, the rest are arguments to that function. There is no return value since everything is async. Use callbacks to get results.
In these examples, we'll assume the following setup: Given a pair A and B, A has the api function "add", and B has none. They are connected to eachother through some transport.
To call a named function simply pass the function name as the first array value.
B calls A's "add" function and passes it (3, 5, function (err, result) {...})
B->A ["add", 3, 4, {$:1}]
To call a callback, use the integer key the other side gave you and it will route the arguments to the callback.
A responds to B's "add" query.
A->B [1, null, 7]
In the initial connection handshake, both sides call a virtual "ready" function passing in a single callback. The other side will reply using this callback with an array of function names. This result is used to populate the wrapper functions in remote.api.
The handshake in our example would have looked like this.
A->B ["ready", {$:1}]
B->A ["ready", {$:1}]
B->A [1, []]
A->B [1, ["add"]]
Note that both sides can do the handshake at the same time. Also note that the function key numbers are independently namespaced per Remote instance. Since this was the first callback for both sides, they both used 1 as the callback key. B then responded using the callback that it has no api functions by sending an empty array. A responded by saying it has one.
As soon as one side receives the api list for the other side, it will emit the "connect" event and be ready to use.
A very powerful debugging trick is to log all messages in the protocol. On the first line of Remote.prototype._onMessage, add this log statement.
console.log(process.pid, message);This will log the process id of the process receiving the message as well as the message already msgpack decoded (but not yet livened).