An approach to the correlation of security events based on machine learning techniques
Tóm tắt
Organizations face the ever growing challenge of providing security within their IT infrastructures. Static approaches to security, such as perimetral defense, have proven less than effective — and, therefore, more vulnerable — in a new scenario characterized by increasingly complex systems and by the evolution and automation of cyber attacks. Moreover, dynamic detection of attacks through IDSs (Instrusion Detection Systems) presents too many false positives to be effective. This work presents an approach on how to collect and normalize, as well as how to fuse and classify, security alerts. This approach involves collecting alerts from different sources and normalizes them according to standardized structures — IDMEF (Intrusion Detection Message Exchange Format). The normalized alerts are grouped into meta-alerts (fusion, or clustering), which are later classified using machine learning techniques into attacks or false alarms. We validate and report an implementation of this approach against the DARPA Challenge and the Scan of the Month, using three different classifications — SVMs, Bayesian Networks and Decision Trees — having achieved high levels of attack detection with little false positives. Our results also indicate that our approach outperforms other works when it comes to detecting new kinds of attacks, making it more suitable to a world of evolving attacks.
Từ khóa
Tài liệu tham khảo
Joosen W, Lagaisse B, Truyen E, Handekyn K: Towards application driven security dashboards in future middleware. J Internet Serv Appl 2012, 3: 107–115. 10.1007/s13174–011–0047–6 10.1007/s13174-011-0047-6 10.1007/s13174-011-0047-6
Hale J, Brusil P: Secur(e/ity) management: A continuing uphill climb. J Netw Syst Manage 2007, 15(4):525–553. 10.1007/s10922-007-9079-4
Ganame AK, Bourgeois J, Bidou R, Spies F: A global security architecture for intrusion detection on computer networks. Elsevier Comput Secur 2008, 27: 30–47. 10.1016/j.cose.2008.03.004
Perner P, Imiya A (Eds): In Alarm clustering for intrusion detection systems in computer networks.. 2005.
Ning P, Cui Y, Reeves DS, Xu D: Techniques and tools for analyzing intrusion alerts. ACM Trans Inf Syst Secur (TISSEC) 2004, 7: 274–318. 10.1145/996943.996947
Boyer S, Dain O, Cunningham R: Stellar: A fusion system for scenario construction and security risk assessment. In Proceedings of the Third IEEE International Workshop on Information Assurance. IEEE Computer Society; 2005:105–116.
Julisch K: Clustering intrusion detection alarms to support root cause analysis. ACM Trans Inf Syst Security 2003, 6: 443–471. 10.1145/950191.950192
Liu P, Zang W, Yu M: Incentive-based modeling and inference of attacker intent, objectives, and strategies. ACM Trans Inf Syst Secur (TISSEC) 2005, 8: 78–118. 10.1145/1053283.1053288
Sabata B: Evidence aggregation in hierarchical evidential reasoning. In UAI Applications Workshop, Uncertainty in AI 2005.. Edinburgh, Scotland; 2005.
Chyssler T, Burschka S, Semling M, Lingvall T, Burbeck K: Alarm reduction and correlation in intrusion detection systems. In Detection of Intrusions and Malware & Vulnerability Assessment workshop (DIMVA).. Dortmund, Deutschland; 2004:9–24.
Ohta S, Kurebayashi R, Kobayashi K: Minimizing false positives of a decision tree classifier for intrusion detection on the internet. J Netw Syst Manage 2008, 16: 399–419. 10.1007/s10922-008-9102-4
Haines JW, Lippmann RP, Fried DJ, Tran E, Boswell S, Zissman MA: The 1999 darpa off-line intrusion detection evaluation. Comput Netw. Int J Comput Telecommunications Netw 2000, 34: 579–595.
Project TH: Know Your Enemy : Learning about Security Threats. Addison-Wesley Professional; 2004.
Sommer R, Paxson V: Outside the closed world: On using machine learning for network intrusion detection. Proceedings of the IEEE Symposium on Security and Privacy 2010.
Bowen T, Chee D, Segal M, Sekar R, Shanbhag T, Uppuluri P: Building survivable systems: An integrated approach based on intrustion detection and damage containment. DARPA Information Survivability Conference (DISCEX) 2000.
Vigna G, Eckmann ST, Kemmerer RA: The stat tool suite. In Proceedings of DISCEX 2000. Hilton Head, IEEE Computer Society Press; 2000.
Lee W, Stolfo SJ, Chan PK, Eskin E, Fan W, Miller M, Hershkop S, Zhang J: Real time data mining-based intrusion detection. In Proc. Second DARPA Information Survivability Conference and Exposition.. Anaheim, USA; 2001:85–100.
Neumann PG, Porras PA: Experience with EMERALD to date. In Proceedings 1st USENIX Workshop on Intrusion Detection and Network Monitoring.. Santa Clara, CA, USA; 2005:73–80.
Grimaila M, Myers J, Mills R, Peterson G: Design and analysis of a dynamically configured log-based distributed security event detection methodology. J Defense Model Simul: Appl Methodolgy Tech 2011, 1–23.
Rieke R, Stoynova Z: Predictive security analysis for eventdriven processes. MMM-ACNS’10 Proceedings of the 5th international conference on Mathematical methods, models and architectures for computer network security 2010.
Valdes A, Skinner K: Probabilistic alert correlation. In Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection (RAID 2001).. Davis, CA, USA; 2001:54–68.
Asif-Iqbal H, Udzir NI, Mahmod R, Ghani AAA: Filtering events using clustering in heterogeneous security logs. Inf Technol J 2011, 10: 798–806.
Corona I, Giacinto G, Mazzariello C, Roli F, Sansone C: Information fusion for computer security: State of the art and open issues. Inf Fusion 2011, 10: 274–284.
Burroughs DJ, Wilson LF, Cybenko GV: Analysis of distributed intrusion detection systems using bayesian methods. In Proceedings of IEEE International Performance Computing and Communication Conference.. Phoenix, AZ, USA; 2002:329–334.
Sabata B, Ornes C: Multisource evidence fusion for cyber-situation assessment. In Proc. SPIE Vol. 6242, 624201 (Apr. 18, 2006).. Orlando, FL, USA; 2006.
Endsley MR: Toward a theory of situation awareness in dynamic systems. Human Factors: J Human Factor Ergon Soc 1995, 37: 32–64. 10.1518/001872095779049543
Debar H, Curry D, Feinstein B: The intrusion detection message exchange format (idmef). 2007.http://tools.ietf.org/html/rfc4765 Internet experimental RFC 4765. Available at
Lan F, Chunlei W, Guoqing M: A framework for network security situation awareness based on knowledge discovery. Computer Engineering and Technology (ICCET) 2010.
Cox K, Gerg C: Managing security with snort and IDS tools. Sebastopol: O’Reilly Media; 2004.
AlFedaghi S, Mahdi F: Events classification in log audit. Int J Netw Secur Appl (IJNSA) 2010, 2: 58–73.
Valdes A, Skinner K, International S: Adaptive, model-based monitoring for cyber attack detection. In Recent Advances in Intrusion Detection (RAID 2000). Springer-Verlag,; 2000:80–92.
Mahoney MV, Chan PK: Learning nonstationary models of normal network traffic for detecting novel attacks. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM; 2002:376–385.
Mukkamala S, Sung AH, Abraham A: Intrusion detection using ensemble of soft computing. In Paradigms, Advances in Soft Computing. Springer Verlag,; 2003:239–248.
Faraoun KM, Boukelif A: Securing network traffic using genetically evolved transformations. Malays J Comput Sci 2006, 19(1):9–28. (ISSN 0127–9084) (ISSN 0127-9084)
Faraoun KM, Boukelif A: Neural networks learning improvement using the k-means clustering algorithm to detect network intrusions. Int J Comput Intell Appl 2006, 6(1):77–99. 10.1142/S1469026806001812
Tandon G, Chan P: Learning rules from system call arguments and sequences for anomaly detection. In ICDM Workshop on Data Mining for Computer Security (DMSEC).. Melbourne, FL, USA; 2003:20–29.
Mukkamala S, Sung AH: Feature ranking and selection for intrusion detection systems using support vector machines. Proceedings of the Second Digital Forensic Research Workshop 2002.
Chang CC, Lin CJ: LIBSVM: a library for support vector machines. 2001.http://www.csie.ntu.edu.tw/~cjlin/libsvm Available at
wei Hsu C, chung Chang C, jen Lin C: A practical guide to support vector classification,” tech. rep., Department of Computer Science, National Taiwan University. 2007.http://www.csie.ntu.edu.tw/~cjlin Available at
Witten IH, Frank E: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann; 2000.
Kayacik HG, Zincir-Heywood AN: Using intrusion detection systems with a firewall: Evaluation on darpa 99 dataset. 2003. Tech. rep., NIMS Technical Report 062003