VA-Architecture

Visual Analytics architecture implementation for fast/realtime aggregation of data using D3.js, Neith, Locustree, Spark and Spark Jobserver.

This code accompanies the submission to LDAV 2014.

It is an implementation of 3 layered architecture for interactively visualizing Big Data. The 3 layers are:

• Visualization layer
• Locustree layer for abstracting away query and caching
• Backend, running on a cluster

This is what the interface looks like:

It uses the following libraries:

• Neith for a functional tree zipper
• D3.js
• Spark
• Ooyala Spark Job Server
• And a little bit of JQuery.js

Although the architecture consists of 3 layers, the visualization and locustree layer still have to be split.

TODO

• Remove dependency on JQuery.
• Remove return-based and stick with CPS based implementation of Locustree
• Add more information on implementation
• Split visualization and locustree layers.
• Add instructions on how to compile and run
• Clean up where possible