With the collection of big data from network traffic, comes the need for more “intelligent” methods to identify security incidents, mainly because it is possible to find new information through the correlation of information and also because a brute-force analysis would take a long time to finish. In this project, it is proposed the development of a system for analyzing massive volumes of heterogeneous data captured in computer networks, in the scope of the RNP network infrastructure, in order to enable the detection, instantly or early, of attacks, that would not be detected with the today existing systems, and the automatic learning with traffic history.
The objectives to be achieved are:
- Definition of an architecture capable of automatic horizontal and vertical scaling, to perform analysis of big data collected by heterogeneous distributed sensors (packets and application and operating systems logs) in search of attacks.
- Performance evaluation, and also evaluation of false positives and false negatives, of several data correlation techniques in order to evaluate the best one for early detection of attacks.
- Implementation of a learning mechanism prototype for simple attacks to consider the feasibility of a more robust mechanism for multi-stage attacks.
- Development of a web interface for system configuration and for visualization of analysis results, allowing at least: differentiation of attacks by level of criticality, creation of time lines for long-term attacks, such as DDoS attacks, and creation of Risk index graphs.
- Analysis of the feasibility of using the system in the RNP Service Center for Security Incidents (CAIS). It is possible that only a few developed mechanisms be integrated with existing systems and procedures in CAIS, since all will be developed with the intention of being easily integrated with well-defined standards and protocols for exchange data related to information security.
As technological innovations, we highlight: (i) a mechanism capable of correlating large amounts of data from several heterogeneous sources in order to detect attacks that would not be possible to be detected with data from a single source, (ii) a mechanism capable of anticipating attacks based on the data collected from several sensors and (iii) the prototype of a learning mechanism for attacks. The prototype, at the end of the Phase 1 of the project, must be able to perform big data analysis that supports the inclusion of several sources and the generation of visualizations of security incidents, besides their anticipation, using correlation of data from several points of a computer network. This data must come from heterogeneous sources, for instance, packet headers and application logs.
