World Scientific
  • Search
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Tue, Oct 25th, 2022 at 2am (EDT)

Existing users will be able to log into the site and access content. However, E-commerce and registration of new users may not be available for up to 12 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.

Packet Capture and Analysis on MEDINA, A Massively Distributed Network Data Caching Platform

    Traffic capture and analysis is key to many domains including network management, security and network forensics. Traditionally, it is performed by a dedicated device accessing traffic at a specific point within the network through a link tap or a port of a node mirroring packets. This approach is problematic because the dedicated device must be equipped with a large amount of computation and storage resources to store and analyze packets. Alternatively, in order to achieve scalability, analysis can be performed by a cluster of hosts. However, this is normally located at a remote location with respect to the observation point, hence requiring to move across the network a large volume of captured traffic. To address this problem, this paper presents an algorithm to distribute the task of capturing, processing and storing packets traversing a network across multiple packet forwarding nodes (e.g., IP routers). Essentially, our solution allows individual nodes on the path of a flow to operate on subsets of packets of that flow in a completely distributed and decentralized manner. The algorithm ensures that each packet is processed by n nodes, where n can be set to 1 to minimize overhead or to a higher value to achieve redundancy. Nodes create a distributed index that enables efficient retrieval of packets they store (e.g., for forensics applications).

    Finally, the basic principles of the presented solution can also be applied, with minimal changes, to the distributed execution of generic tasks on data flowing through a network of nodes with processing and storage capabilities. This has applications in various fields ranging from Fog Computing, to microservice architectures and the Internet of Things.

    References

    • 1. Cisco UCS E-Series Servers. http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-e-series-servers/index.html. [Online; accessed 27-January 2017]. Google Scholar
    • 2. A. Anand, V. Sekar and A. Akella, SmartRE: An architecture for coordinated network-wide redundancy elimination, ACM SIGCOMM Computer Communication Review, 2009. Crossref, ISIGoogle Scholar
    • 3. C.-W. Chang, G. Huang, B. Lin and C.-N. Chuah, LEISURE: Load-balanced network-wide traffic measurement and monitor placement, IEEE Transactions on Parallel and Distributed Systems, 2015. Crossref, ISIGoogle Scholar
    • 4. M. Chiesa, C. Dietzel, G. Antichi, M. Bruyere, I. Castro, M. Gusat, T. King, A. W. Moore, T. D. Nguyen, P. Owezarski, S. Uhlig and M. Canini, Inter-domain networking innovation on steroids: Empowering IXPs with SDN capabilities, IEEE Communications Magazine, 2016. Crossref, ISIGoogle Scholar
    • 5. A. Di Pietro, F. Huici, D. Costantini and S. Niccolini, DECON: Decentralized coordination for large-scale flow monitoring, in IEEE INFOCOM (IEEE, 2010). Google Scholar
    • 6. A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford and F. True, Deriving traffic demands for operational IP networks: Methodology and experience, IEEE/ACM Transactions on Networking, 2001. Crossref, ISIGoogle Scholar
    • 7. M. L. Fredman and R. E. Tarjan, Fibonacci heaps and their uses in improved network optimization algorithms, Journal of the ACM, 1987. Crossref, ISIGoogle Scholar
    • 8. R. Gad, M. Kappes and I. Medina-Bulo, Monitoring traffic in computer networks with dynamic distributed remote packet capturing, in 2015 IEEE International Conference on Communications (ICC) (IEEE, 2015). Google Scholar
    • 9. R. Gad, M. Kappes, R. Mueller-Bady and I. Medina-Bulo, Header field based partitioning of network traffic for distributed packet capturing and processing, in IEEE 28th International Conference on Advanced Information Networking and Applications (IEEE, 2014). Google Scholar
    • 10. A. Gupta, R. Birkner, M. Canini, N. Feamster, C. Mac-Stoker and W. Willinger, Network monitoring as a streaming analytics problem, in 15th ACM Workshop on Hot Topics in Networks (HotNets) (ACM, 2016). Google Scholar
    • 11. N. Kamiyama, T. Mori and R. Kawahara, Autonomic load balancing of flow monitors, Computer Networks, 2013. Crossref, ISIGoogle Scholar
    • 12. A. Kirsch, M. Mitzenmacher and G. Varghese, Hash-based techniques for high-speed packet processing, in Algorithms for Next Generation Networks (Springer, 2010). CrossrefGoogle Scholar
    • 13. A. Krylovskiy, M. Jahn and E. Patti, Designing a smart city internet of things platform with microservice architecture, in 3rd International Conference on Future Internet of Things and Cloud (FiCloud ) (IEEE, 2015). Google Scholar
    • 14. N. Mandagere, P. Zhou, M. A. Smith and S. Uttamchandani, Demystifying data deduplication, in ACM/IFIP/USENIX Middleware Conference Companion (ACM, 2008). Google Scholar
    • 15. D. T. Meyer and W. J. Bolosky, A study of practical deduplication, in 9th USENIX Conference on File and Storage Technologies (FAST ) (USENIX, 2011). Google Scholar
    • 16. M. Pustisek, I. Humar and J. Bester, Empirical analysis and modeling of peer-to-peer traffic flows, in 14th IEEE Mediterranean Electrotechnical Conference (MELECON ) (IEEE, 2008). Google Scholar
    • 17. V. Sekar, A. Gupta, M. K. Reiter and H. Zhang, Coordinated sampling sans origin-destination identifiers: Algorithms and analysis, in 2nd International Conference on Communication Systems and Networks (COMSNETS ) (IEEE, 2010). Google Scholar
    • 18. V. Sekar, M. K. Reiter, W. Willinger, H. Zhang, R. R. Kompella and D. G. Andersen, CSAMP: A system for network-wide flow monitoring, in 5th USENIX Symposium on Networked Systems Design and Implementation (NSDI ) (USENIX, 2008). Google Scholar
    • 19. C. Shannon, D. Moore and K. C. Claffy, Beyond folklore: Observations on fragmented traffic, IEEE/ACM Transactions on Networking, 2002. Crossref, ISIGoogle Scholar
    • 20. M. R. Sharma and J. W. Byers, Scalable coordination techniques for distributed network monitoring, in International Workshop on Passive and Active Network Measurement (PAM ) (Springer, 2005). Google Scholar
    • 21. S.-H. Shen and A. Akella, DECOR: A distributed coordinated resource monitoring system, in 20th International Workshop on Quality of Service (IWQoS ) (IEEE, 2012). Google Scholar
    • 22. L. M. Vaquero and L. Rodero-Merino, Finding your way in the fog: Towards a comprehensive definition of fog computing, ACM SIGCOMM Computer Communication Review, 2014. Crossref, ISIGoogle Scholar
    • 23. B. Xiong, K. Yang, F. Li, X. Chen, J. Zhang, Q. Tang, Y. Luo, The impact of bitwise operators on hash uniformity in network packet processing, International Journal of Communication Systems, 2014. ISIGoogle Scholar