Article

Smart cybersecurity strategies based on deep reinforcement learning : A Literature Review

This paper presents a literature review on the application of deep reinforcement learning (DRL) in cybersecurity. It focuses on key domains, including intrusion detection, adaptive cyber defense, multi-agent coordination, and automated penetration testing. A total of 18 peer-reviewed studies published between 2022 and 2025 were selected through a structured review process and analyzed using performance metrics such as accuracy, precision, recall, and F1-score.The results indicate that multi-agent DRL approaches generally outperform single-agent models in dynamic attack environments. Hybrid DRL models that integrate deep learning techniques, such as convolutional and recurrent neural networks and attention mechanisms, show improved detection accuracy and adaptability. DRL-based penetration testing methods also demonstrate the ability to autonomously explore vulnerabilities and optimize attack strategies. However, challenges remain, including limited generalization to real-world scenarios, high computational costs, low interpretability, and the lack of standardized datasets. Addressing these issues can enable the development of more adaptive, efficient, and reliable cybersecurity systems.

...
Aleah Abdulkaher Alshamiri
Department of Computer Science, Faculty of Computer and Information Technology, Sana’a University, Sana’a, Yemen
...
Ghaleb H. Al Gaphari
Al Gaphari
12906
A. Venturi, M. Andreolini, M. Marchetti, and M. Colajanni, “Assessing generalizability of Deep Reinforcement Learning algorithms for Automated Vulnerability Assessment and Penetration Testing,” Array, vol. 24, Dec. 2024, https://doi.org/10.1016/j.array.2024.100365
12907
T. Purves, K. G. Kyriakopoulos, S. Jenkins, I. Phillips, and T. Dudman, “Causally aware reinforcement learning agents for autonomous cyber defence,” Knowl Based Syst, vol. 304, Nov.2024,https://doi.org/10.1016/j.knosys.2024.112521.
12908
Y. Tang, J. Sun, H. Wang, J. Deng, L. Tong, and W. Xu, “A method of network attack-defense game and collaborative defense decision-making based on hierarchical multi-agent reinforcement learning,” Comput Secur, vol. 142,Jul.2024,https://doi.org/10.1016/j.cose.2024.103871.
12909
A. A. Hammad, S. R. Ahmed, M. K. Abdul-Hussein, M. R. Ahmed, D. A. Majeed, and S. Algburi, “Deep Reinforcement Learning for Adaptive Cyber Defense in Network Security,” in ACM International Conference Proceeding Series, Association for Computing Machinery, May2024,pp.292,297https://doi.org/10.1145/3660853.3660930
12910
S. H. Oh, J. Kim, J. H. Nah, and J. Park, “Employing Deep Reinforcement Learning to Cyber-Attack Simulation for Enhancing Cybersecurity,” Electronics (Switzerland), vol. 13,no.3,Feb.2024,https://doi.org/10.3390/electronics13030555
12911
Y. Ma, C. Li, Y. Wang, and Y. Wang, “Application of deep reinforcement learning algorithms for automatic threat detection and response in dynamic network environments to improve cybersecurity,” Journal of Computational Methods in Sciences and Engineering, vol. 25, no. 3, pp. 2112–2125, May2025,https://doi.org/10.1177/14727978241309550
12912
B. Reddy Maddireddy and B. Reddy Maddireddy, “The Role of Reinforcement Learning in Dynamic Cyber Defense Strategies,” 2024. Accessed: Aug. 18, 2025. [Online].Available:https://ijaeti.com/index.php/Journal/article/download/306/338
12913
Y. Li, H. Dai, and J. Yan, “Knowledge-Informed Auto-Penetration Testing Based on Reinforcement Learning with Reward Machine,” May 2024, Accessed: Aug. 18, 2025. [Online].Available: https://ieeexplore.ieee.org/document/10650368
12914
Q. Li et al., “DynPen: Automated Penetration Testing in Dynamic Network Scenarios Using Deep Reinforcement Learning,” IEEE Transactions on Information Forensics and Security, vol. 19, pp. 8966–8981, 2024, https://doi.org/10.1109/TIFS.2024.3461950
12915
B. S. Kim, H. W. Suk, Y. H. Choi, D. S. Moon, and M. S. Kim, “Optimal Cyber Attack Strategy Using Reinforcement Learning Based on Common Vulnerability Scoring System,” CMES - Computer Modeling in Engineering and Sciences, vol. 141, no. 2, pp. 1551–1574, 2024, https://doi.org/10.32604/cmes.2024.052375.
12916
I. Jabr, Y. Salman, M. Shqair, and A. Hawash, “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach,” An-Najah University Journal for Research - A (Natural Sciences), vol. 39, no. 1, pp.714,Feb.2025,https://doi.org/10.35552/anujr.a.39.1.2231
12917
W. Yang, A. Acuto, Y. Zhou, and D. Wojtczak, “A Survey for Deep Reinforcement Learning Based Network Intrusion Detection,” Sep. 2024,[Online].Available:https://arxiv.org/abs/2410.07612
12918
A. M. K. Adawadkar and N. Kulkarni, “Cyber-security and reinforcement learning — A brief survey,” Sep. 01, 2022, Elsevier Ltd. https://doi.org/10.1016/j.engappai.2022.105116
12919
T. T. Nguyen and V. J. Reddi, “Deep Reinforcement Learning for Cyber Security,” IEEE Trans Neural Netw Learn Syst, vol. 34, no. 8, pp. 3779–3795, Aug. 2023, doi: 10.1109/TNNLS.2021.3121870:https://ieeexplore.ieee.org/document/9596578
12920
A. Manikandan and S. D. Rajan, “Cyber Attack Detection Using Deep Multi-agent Reinforcement Learning with Beth Dataset,” SN Comput Sci, vol. 6, no. 5, Jun. 2025, https://doi.org/10.1007/s42979-025-03981-8
12921
N. Niknami and J. Wu, “DeepIDPS: An Adaptive DRL-based Intrusion Detection and Prevention System for SDN.” Accessed: Aug. 18, 2025. [Online]. Available: https://ieeexplore.ieee.org/document/10622849
12922
M. M. Al-Nawashi, O. M. Al-Hazaimeh, N. M. Tahat, N. Gharaibeh, W. A. Abu-Ain, and T. Abu-Ain, “Deep Reinforcement Learning-Based Framework for Enhancing Cybersecurity,” International Journal of Interactive Mobile Technologies , vol. 19, no. 3, pp.170–190,Feb.2025, https://doi.org/10.3991/ijim.v19i03.50727
12923
H. S. AlSagri, S. S. Sohail, and S. Sebastian, “The role of deep reinforcement learning in developing adaptive cybersecurity defenses for smart grid systems,” Journal of Information and Optimization Sciences, vol. 45, no. 8, pp. 2299–2307,2024 https://doi.org/10.47974/JIOS-1807.
Download data is not yet available.

Metrics

0
Views
0
Downloads
0
Citations

How to Cite

Smart cybersecurity strategies based on deep reinforcement learning : A Literature Review. (2026). Sana’a University Journal of Applied Sciences and Technology, 4(4), 2025-2033. https://doi.org/10.59628/d80zj848

Similar Articles

You may also start an advanced similarity search for this article.

Most read articles by the same author(s)