Visual Analytics architecture implementation for fast/realtime aggregation of data using D3.js, Neith, Locustree, Spark and Spark Jobserver.
This code accompanies the submission to LDAV 2014.
It is an implementation of 3 layered architecture for interactively visualizing Big Data. The 3 layers are:
- Visualization layer
- Locustree layer for abstracting away query and caching
- Backend, running on a cluster
This is what the interface looks like:
It uses the following libraries:
- Neith for a functional tree zipper
- D3.js
- Spark
- Ooyala Spark Job Server
- And a little bit of JQuery.js
Although the architecture consists of 3 layers, the visualization and locustree layer still have to be split.
- Remove dependency on JQuery.
- Remove return-based and stick with CPS based implementation of Locustree
- Add more information on implementation
- Split visualization and locustree layers.
- Add instructions on how to compile and run
- Clean up where possible