A GUI for application design and performance reporting of data streaming applications
Ladda ner
Typ
Examensarbete för masterexamen
Program
Computer systems and networks (MPCSN), MSc
Publicerad
2021
Författare
Teodorsson, Rikard
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
As a result of more connected devices and users and applications moving to the cloud,
every year the amount of data needed to be processed by different computer systems
increases. The troubles with trying to store all this information, and a growing
demand for real-time processing not possible with traditional batch processing, has
led to an increase in the popularity of stream processing as a way of handling large
amounts of data in a real-time fashion with low latency and high throughput.
The number of Stream Processing Engines (SPEs) available for processing this
streaming data is increasing and each has its own programming style and conventions,
resulting in individually developed SPEs that differ in many aspects. In
general, there is a lack of a common set of tools supporting developers to design,
visualise and monitor streaming applications. For instance, a streaming application
executed by an SPE can be modelled as a Directed Acyclic Graph (DAG), but not
all SPEs have a tool to visualise this graph.
This thesis concerns the design and implementation of a framework in the form of
a Graphical User Interface (GUI) as a first step towards a unified view of streaming
applications, aiding developers with visual representations, statistics, and code
generation for different SPEs by abstracting away the implementation details.
Using Java and JavaFX, a framework is developed and tested with two SPEs (Apache
Flink and Liebre) and then evaluated in terms of performance, functionality, and
generality.
Presented is a functional framework allowing for a user to visualise DAGs of existing
streaming applications, viewing live and offline statistics of executions, and the
possibility to design a streaming application graphically and generating the corresponding
Java code. The implementation is general and can be adapted to work
with other popular SPEs.
The resulting framework is a first step towards a unified view of streaming applications
from different SPEs and adds a tool for developers to use, enabling for
increased productivity and better understanding while working with both new and
already existing streaming applications.
Beskrivning
Ämne/nyckelord
GUI , SPE , stream processing , graphical user interface , framework , graph design , code generation , streaming application , query statistics