J.A.C.: Just Another Chord implementation

🎓 School of Electrical & Computer Engineering, NTUA
Project for the course Distributed Systems, Winter Semester 2020-2021

This repo was created for version control of the project implemented during Distributed Systems course in the department of Electrical & Computer Engireering School, NTUA. The main goal was the construction of file sharing application with multiple distributed nodes DHT. Further information can be found in the related pdf file (in greek).

📋 Table of Contents

• Overview
• Architecture
• Installation
  • Using conda
  • Using pip
• Deployment
• Demo
• References

📌 Overview

JAC is a distributed system that provides a look up service of key-value pairs, stored across a cluster of nodes. Any participating node can retrieve the value associated to a particular key or even the value of one of its replicas, if replication of data is enabled. JAC provides out of the box 2 types of consistency, eventually consistency & chain-replication. In more depth, each node offers the basic fuctionalities of a Distributed Hash Table, such as insert, query & delete with some additional commands like overlay, info and others. Further infromation about all the available commands can be found inside UTILITIES.md.

jac-cli@ntua$ help
Usage:   COMMAND [OPTIONS] [ARGS]

Commands:
  delete   Deletes the specified <key>.
  depart   Makes current node to depart.
  exit     Makes current node to depart & exits from shell.
  help     Prints this message and exits.
  info     Displays info for current node.
  insert   Inserts the pair (<key>, <value>).
  join     Inserts a new node.
  overlay  Displays current network topology.
  query    Finds the value of <key>.

📌 Architecture

The basic concept behind jac desing can be found in the diagram offered below. The entire shell is offered with the usage of cmd package and click framework, while the actual execution of each operation is accomplished by the Flask server that runs on each node. After it receives a request, it can potential communicate with other nodes of the cluster, in order to fulfill the user request. Once proper data are gathered, server uses json format to send the requested information back to the click command, which in its terms will output the result.

As far as Chord protocol is concerned, a simpler version was implemented, with the following key concepts:

• No finger tables were constructed (as requested by supervisors). Each node knows only about the next and the previous one.
• Unfortunately a single point of faillure was introduced. The first node that joins the chord, aka the bootstrap node, knows the address & the port of every node inside the system. On node join, each candidate asks this bootstrap node for information about its future position inside the chord.
• Requests are forwarded in both directions, decreasing system's response time.

📌 Installation

One can found the proper steps for downloading and installing jac dependencies in their system (either physical or virtual machines), using two known package managers of python like conda & pip. Steps mentioned above assume that a proper python virtual enviroment has been created in advance.

Option A: Using conda

In this case, an additional package is needed, like setuptools. If this is not already part of the conda enviroment of interest, installation instructions can be obtained from the aforamentioned url.

cd jac/src/
setuptools-conda install-requirements ./

Option B: Using pip

Inside jac/src/requierments.txt, dependency packages' versions were placed, which were tested successfully during development. Due to the fact that no strict restriction was found for any specific package, everyone is more than welcome to test different versions for each package that jac is using.

cd jac/src/
pip install ./requirements.txt

📌 Deployment

At this point, jac is capable of running inside the virtual enviroment by running the following commands.

cd jac/src/jacmodule
# No replication of data
./jac.py
# Replication of data with 2 copies of each key-value pair & chain-replication
./jac.py 2 chain-replication
# Replication of data wit 5 copies of each key-value pair & eventually consistency
./jac.py 5 eventually

⚠️ ATTENTION: JAC was developed in an isolated cluster inside a private network of virtual machines and was never tested in production. Thus, the development server was used instead of the suggested WSGI server.

During development, a cluster of 5 virtual machines were provided. For transfering the src code to the host and from there to each node, the script jac/scripts/send_module.sh was used.

📌 Demo

📌 References

[1] Stoica, Ion, et al. "Chord: A scalable peer-to-peer lookup service for internet applications." ACM SIGCOMM Computer Communication Review 31.4 (2001): 149-160.