2024/10/20 - COMPACT Newsletter #2
We are pleased to publish the second e-newsletter, providing an update on the current status of the COMPACT research project. On October 14th, the entire research team held a remote meeting to discuss recent activities and the latest project updates.
The POLIMI team has successfully completed two key activities:- Optimal Storage-Accuracy Management for IoT Forensics: POLIMI researchers have developed a framework to optimize IoT forensic tasks in a commercial Access Point. This framework automatically adjusts traffic analysis parameters—such as the number of statistical features to extract and the quantization levels used to represent these features. This allows the system to meet specific computational and storage constraints while maximizing the accuracy of forensic analysis. A journal paper on this framework has been submitted to the IEEE Transactions on Network and Service Management.
- Probe Request Fingerprinting: The POLIMI team has also developed a method for creating a compact yet highly discriminative fingerprint for probe request frames. This fingerprint, requiring as little as 16 bits of memory per frame, enables accurate matching of probe requests from the same device, which is crucial for applications like device counting with minimal memory usage. The results of this work have been submitted to a top-tier conference (details omitted due to double-blind review policy). Additionally, the POLIMI team is working on other important tasks related to traffic compression, with a focus on Zigbee traffic, CSI data from Wi-Fi transmissions, and time series of cellular network KPIs.
The UniTS Team is also making strides in their respective areas of the project. Specifically, it evaluated different lossy compression techniques for network logs coming from an operational network. The UniTS Team’s goal is to find novel compression techniques for compressing flow records and similar logs, and to this end, it took as a case study network logs produced by the Tstat passive meter. To evaluate the effectiveness of compression and assess rate-accuracy trade-offs, the target metric has been set to the classification of the domain (or website) the user contacted when generating a flow record, while the features are volumetric characteristics of the flow packets (e.g., packet size and timings). The Team evaluated and combined various techniques such as scalar quantization, Principal Component Analysis, and Vector Quantization (testing K-Means and H-DBscan algorithms). The results show that scalar quantization achieves the best trade-off between compression rate and accuracy, while PCA, despite proving good accuracy, achieves low compression rates due to the high entropy it injects into the data. Vector quantization algorithms provide promising results as well, but they can hardly scale to large data (which is still an open problem in the literature). In the following months, the Team will summarize the achievements in a scientific paper to submit to an international top-tier conference (not yet identified). Moreover, the Team will evaluate other compression techniques based on ML, specifically Auto-Encoder architectures.
Stay tuned on our website for more news on our website!
The COMPACT team
2024/05/20 - COMPACT Newsletter #1
The COMPACT research team is happy to publish this first e-newsletter about the status of the project! COMPACT is a PRIN-funded project that will propose methodologies aimed at compressing feature-based representations of network traffic in edge (e.g., IoT) and core (e.g., backbone) scenarios. The entire research team met in Trieste at the end of April, to discuss the recent activities and the latest updates. It was a very productive meeting, with many fruitful discussions. We report in this letter a summary of the activities that have been discussed.
- Optimal storage-accuracy management for IoT forensics: POLIMI is actively working on optimizing traffic storage for IoT forensics tasks, which require capturing, storing, and analyzing network traffic exchanged by consumer IoT devices. The focus has been mainly on Wi-Fi-enabled IoT devices, although the team has recently started working with Zigbee traffic as well. The main objective of this activity is to develop techniques to efficiently store traffic and traffic characteristics relative to IoT devices so that forensic analysis built on such data can be performed with high levels of accuracy. To this end, we built a Wi-Fi access point able to perform online traffic feature extraction and compression (so far using lossy techniques based on quantization) and evaluated its performance on several forensics tasks such as presence and movement detection. At the moment, the team is working on a theoretical framework for optimizing the storage/accuracy performance of the proposed access point as well as on extending the architecture to Zigbee devices, integrating the compression and forensic analysis algorithms into a Zigbee gateway.
- Probe request compression and matching: POLIMI is also actively working towards a compression algorithm tailored to Probe Request frames, which are transmitted by Wi-Fi devices. Such frames are particularly interesting for forensics activities such as localizing or tracking devices. Since the source MAC address contained in Probe Requests is generally randomized for privacy reasons, the objective of the work is to design a fingerprint algorithm that can determine the emitting device of a set of probe requests with high precision while requiring just a fraction of the bits needed to store the entire probe request frame. At the moment, techniques from the field of audio and video fingerprinting are being studied and adapted to the task at hand.
- Darknets traffic analysis and compression: UniTS is working towards implementing effective representation and compression techniques for darknet traffic. Darknets are IP addresses that function as passive probes, recording all received packets without hosting services. The traffic they capture, being unsolicited, makes darknets akin to “network telescopes”. On the web, there exist many open-source lists containing a set of known malicious IP addresses. They are a great resource for network administrators and practitioners, helping to build blocklists to prevent or block cyber attacks. During the first months of the project, the UniTS team has worked to understand to what extent lists can be used to explain darknet traffic — i.e., how many endpoints contacting a darknet appear in the public lists, thus are known malicious actors. Our results have shown that public lists can explain up to 3% of the sources of darknet traffic, but up to the 30% of volume. We are currently working to extend the set of lists considered in the studies and to include the lists of IP addresses of public legitimate scanners. Our next steps include a thorough methodology to group and cluster darknet traffic based on the information contained in public lists, to filter out uninteresting traffic, allowing storing only salient events thus achieving effective compression of the darknet traffic.
Stay tuned on our website for more news on our website!
The COMPACT team
Funding in k€
Project duration in months
Academic partners
Publications