Statistical Analysis of Massive Data Streams: Proceedings of a Workshop (2004)
Chapter: ABSTRACT OF PRESENTATION
Abstract of Presentation
Visualization of Internet Packet Headers
Edward J.Wegman, George Mason University (with Don R.Faxon, Jeffrey L.Solka, and John Rigsby) .
Abstract: We have launched a project with the agreement of the University’s CIO to capture all header information for all Internet traffic in and out of the University. This includes TCP, UDP, SNMP, and ICMP packets. We have installed sniffer and analysis machines and are capable of recording up to a terabyte of traffic data. Preliminary experiments within our small statistics subnet indicate traffic of 65,000 to 150,000 packets per hour. Indications are that we will have terabytes of data traffic daily university-wide, 35–40 megabytes of header traffic per minute, or approximately 50–60 gigabytes of header information per day in the larger University context. Much of the packet traffic is administrative traffic from routers. Ultimately, we are interested in real-time detection of intrusion attacks so that analysis methods for streaming data are necessary. In this talk I will describe our project, including some background on TCP/IP traffic, indicate some recursive methods capable of handling streaming data, illustrate a database tool we have developed, and give some suggestions for visualization procedures we are in the process of implementing. This report is very much a preliminary report. In data mining, 80% to 90% of the effort involves getting the data in shape to analyze, and this project does not deviate from this pattern.
