Distributed Real time Processing of Data Streams

Granules supports the processing of data streams over a distributed collection of processing elements. Such streams can be generated in settings involving observational and monitoring equipment, simulations, and computational workflows. Salient features in Granules include support for :

• Long running, stateful computations with multiple rounds of execution
• Expressing computations as directed graphs that have cycles in them
• Execution graphs span multiple machines and can be recursive, data driven, or periodic.
• Developing computations in C, C++,C#, Java, R, and Python
• Incremental assimilation of processing elements via a scale-out architecture
• Orchestrating processing over traditional clusters, collection of desktops, or IaaS VM-based settings
  

• Scheduling computations based on data stream activity, periodicity, or a combination thereof
• Interleaving hundreds of computations on a single resource
• Management of the lifecycle and finite state machine associated with computations
• Abstracting the complexities of doing such I/O and the vagaries of execution in distributed setting
• Scientific extensions to the basic MapReduce framework