Real-time Analysis of NetFlow Data for Generating Network Traffic Statistics using Apache Spark

Authors

ČERMÁK Milan JIRSÍK Tomáš LAŠTOVIČKA Martin

Year of publication 2016
Type Article in Proceedings
Conference NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium
MU Faculty or unit

Institute of Computer Science

Citation
Web http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7502952
Doi http://dx.doi.org/10.1109/NOMS.2016.7502952
Field Informatics
Keywords Apache Spark;Real-time;Network Statistics
Attached files
Description Abstract—In this paper, we present a framework for the realtime generation of network traffic statistics on Apache Spark Streaming, a modern distributed stream processing system. Our previous results showed that stream processing systems provide enough throughput to process a large volume of NetFlow data and hence they are suitable for network traffic monitoring. This paper describes the integration of Apache Spark Streaming into a current network monitoring architecture. We prove that it is possible to implement the same basic methods for NetFlow data analysis in the stream processing framework as in the traditional ones. Moreover, our stream processing implementation discovers new information which is not available when using traditional network monitoring approaches.

You are running an old browser version. We recommend updating your browser to its latest version.

More info