Implementation of a transport system with stateful consumers

Typ
Examensarbete för masterexamen
Master Thesis
Program
Publicerad
2013
Författare
SÖDERBERG, MATHIAS
HALLBERG, GUSTAF
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
When handling large amounts of data, parallel processing is key. Distributed computation makes Big Data work, but it presents new challenges for scaling and reliability. These challenges are made even more interesting when computations are stateful, and done in near real-time as in stream processing. To make a stateful distributed system scalable and resilient to failures, a framework for message passing and state persistence is needed. As of today no such system is available but the components are all there to build one. This thesis gives an outline of how such a framework could be constructed using Apache ZooKeeper and Apache Kafka, describing algorithms and components which could be used for such a system. Also provided are some experimental performance benchmarks of Kafka running in Amazon's EC2 cloud, to use as a measuring stick for its viability as a transport system for messages.
Beskrivning
Ämne/nyckelord
Interaktionsteknik , Interaction Technologies
Citation
Arkitekt (konstruktör)
Geografisk plats
Byggnad (typ)
Byggår
Modelltyp
Skala
Teknik / material
Index