Granules Logonsf-logodhs-logo
  Overview | Documents | API | Software | Contact | HOME |

Distributed Real time Processing of Data Streams

Granules supports the processing of data streams over a distributed collection of processing elements. Such streams can be generated in settings involving observational and monitoring equipment, simulations, and computational workflows. Salient features in Granules include support for :

  Project News
Granules v3 will be released in March, 2017.
  • Long running, stateful computations with multiple rounds of execution
  • Expressing computations as directed graphs that have cycles in them
  • Execution graphs span multiple machines and can be recursive, data driven, or periodic.
  • Developing computations in C, C++,C#, Java, R, and Python
  • Incremental assimilation of processing elements via a scale-out architecture
  • Orchestrating processing over traditional clusters, collection of desktops, or IaaS VM-based settings
  • Scheduling computations based on data stream activity, periodicity, or a combination thereof
  • Interleaving hundreds of computations on a single resource
  • Management of the lifecycle and finite state machine associated with computations
  • Abstracting the complexities of doing such I/O and the vagaries of execution in distributed setting
  • Scientific extensions to the basic MapReduce framework
  

A broad class of compute and data intensive applications can benefit from the capabilities available in Granules. Some of the application domains that Granules is currently deployed in include brain computer interfaces, epidemiological modeling, handwriting recognition, high-dimensional data clustering algorithms, and bio-informatics (mRNA sequencing). More information can be found in this short overview of Granules.

Our new system, NEPTUNE, builds on Granules and targets extreme throughputs in continuous sensing environments. Our preliminary benchmarks contrasting the system with Storm shows promise with throughput and latency measurements showing signficant performance improvements over Storm.
  
     

 


 © The Granules Project
Department of Computer Science
Colorado State University 		 
     
This research has been supported by funding from the US National Science Foundation's Computer Systems Research Program (CNS-1253908) and the Department of Homeland Security’s Long Range Program.