v2.0.0
Major improvements since v0.9.2
New Features
- Enabled external client API.
- Revived code for using ibverbs API with flow control.
- Derecho can now be configured with a "restart leader" distinct from its normal leader, and if the
enable_backup_restart_leaders
option is True, it can also use a list of multiple restart leaders in priority order. - Added a build script.
- The type T in Replicated is now aware of the subgroup ID it belongs to.
- Moved the ObjectStore out to the new Cascade project.
- Renamed
rpc_port
tostate_transfer_port
in configuration.
Bugfixes
- Fixed the completion queue overrun issue.
- Avoided relying on
ibv_wc::wr_id
(orfi_cq_err_entry::op_context
) to determine which remote node failed when a request posted to a queue pair fails. Now uses timeout logic to detect the failure. - Refactored
p2p_connection
code. - Stored Persistent state into a file named by SHA256 hash string instead of a type string, which might be longer than filename length limitation.
- Fixed TCP
listen()
backlog issue contributing to the slow startup with many nodes.
Dependencies
- Moved to
spdlog v1.3.1
- Moved to
libfabric v1.7.0
Known Issues
- An update to a subgroup type with Persistent fields is acknowledged (by
derecho::rpc::QueryResults<>
) when all shard members delivered the update. Persistence is processed in the background off the critical path. Applications must uselocal_persistentce_callback
orglobal_persistence_callback
in group constructor to make sure when the updates are persisted locally or globally.
/**
* Bundles together a set of callback functions for message delivery events.
* These will be invoked by MulticastGroup or ViewManager to hand control back
* to the client if it wants to implement custom logic to respond to each
* message's arrival. (Note, this is a client-facing constructor argument,
* not an internal data structure).
*/
struct CallbackSet {
message_callback_t global_stability_callback;
persistence_callback_t local_persistence_callback = nullptr;
persistence_callback_t global_persistence_callback = nullptr;
};
- A Derecho node cannot join multiple subgroups so far. We plan to add this feature, aka overlapping subgroup, in the near future.
- Slow startup (It takes several seconds for RDMC to start, please see #160 for more information)