Cantitate/Preț
Produs

High-speed indexing and archival of network measurement data

Autor Francesco Fusco
en Limba Engleză Paperback – 18 noi 2012
The Internet has became a global IT infrastructure providing ubiquitously accessible, interactive, and secure services used by a large fraction of the global population. To meet users' expectations, network administrators require sophisticated monitoring infrastructures for detecting misconfiguration and faults, for measuring the performance, and for enabling timely reactions to security threats.Passive monitoring has rapidly become the de-facto monitoring approach for getting deep insights into the actual status of production networks. Nowadays networks rely on network probes, which are embedded in network equipments or deployed as special purpose monitoring devices, that constantly monitor important network aspects. Therefore, current monitoring infrastructures are able to create large volumes of monitoring data.Industrial and academic research mostly focused on the generation, collection, processing and analysis of network monitoring data streams with the primary goal of providing live views of diverse network aspects. These efforts have led to mature technologies for processing high-speed data streams in real-time. Nowadays, stream processing represents the foundation for the large majority of software and hardware based monitoring infrastructures deployed for operating current production networks. In a nutshell, the stream processing approach consists of applying a predefined set of queries to one or more data streams in a way that summaries of the data are continuously computed. This approach allows one to have a predefined set of information about the network streams without requiring the streams to be entirely recorded, hence the name single-pass analytics. Unfortunately, this also means that the information not captured by the current query set is lost forever.In many emerging contexts including, but not limited to, cyber-security, this trade-off is undesirable. In particular, large corporations, financial institutions and high-security data centers are increasingly interested in efficient solutions enabling the collection of exact data streams, and the expedient analysis of large-scale repositories of historical network measurements particularly in case of security breaches. Enabling long-term historical analysis of massive volumes of network monitoring data is required to enable forensics, anomaly detection, and information leakage analysis tasks.Advanced data collection systems are required to enable the archival of high-speed streams of network monitoring data and, most importantly, to enable fast explorations of large-scale repositories. Such systems have to support data archiving under extremely high-speed insertion rates and to produce archives still amenable to indexing and search. Current solutions that address the challenge of lossless storage of massive network monitoring data streams use off-the-shelf compression techniques, like GZIP and BZIP2. The main shortcoming of these solutions is that they do not offer efficient query processing, especially for queries targeting a small part of the dataset, as large data blocks are compressed and then retrieved using expensive decompression operations and serial scans of the archives.In this thesis, we first focus on the storage, indexing, and data querying of high-speed streams of network flow information and we propose an architecture built upon novel lossless indexing and compression algorithms carefully optimized for the network monitoring domain. The architecture is capable of compressing high-speed streams of network flow records in real-time while achieving higher compression ratios than popular general-purpose compressors, and, more importantly, produces compressed archives that support partial decompression. Then, we describe an indexing architecture for packet traces that has been integrated into libpcap, the de-facto reference library for accessing packet trace repositories.We make the following important contributions: (a) we propose a novel compressed bitmap index encoding that outperforms the current state-of-theart both in terms of CPU load and disk consumption when indexing network flow traces and packet traces, (b) we introduce an online stream reordering mechanism that boosts both compression ratios and retrieval time of modern compressors and compressed bitmap indexes, and, (c) we describe RasterZIP, a novel lossless compressor that leverages indexes for providing fine-grained decompression granularity. RasterZip achieves higher compression ratios than general purpose compressors by exploiting data patterns, such as the shared prefixes of reordered IP addresses, that are commonly present in networking data.
Citește tot Restrânge

Preț: 27799 lei

Nou

Puncte Express: 417

Preț estimativ în valută:
5323 5542$ 4416£

Carte indisponibilă temporar

Doresc să fiu notificat când acest titlu va fi disponibil:

Preluare comenzi: 021 569.72.76

Specificații

ISBN-13: 9783844014648
ISBN-10: 3844014640
Pagini: 155
Ilustrații: 42 farbige Abbildungen
Dimensiuni: 146 x 208 x 17 mm
Greutate: 0.23 kg
Editura: Shaker Verlag