Behaviour-Aware Hybrid Deep Networks for Detecting Zero-Day and Ransomware Threats
Madhan Mohan Reddy Chinthala1, Harish Apuri2, Kishore Bitra3
1Madhan Mohan Reddy Chinthala, Department of IT, Franklin Info Tech, Rochester (NY), United States of America (USA).
2Harish Apuri, Department of IT, Induct Inc, Charlotte, NC, United States of America (USA).
3Kishore Bitra, Department of IT, Information and Technology, Baltimore (MD), United States of America (USA).
Manuscript received on 28 March 2026 | First Revised Manuscript received on 01 April 2026 | Second Revised Manuscript received on 07 April 2026 | Manuscript Accepted on 15 April 2026 | Manuscript published on 30 April 2026 | PP: 1-10 | Volume-15 Issue-5, April 2026 | Retrieval Number: 100.1/ijitee.E124615050426 | DOI: 10.35940/ijitee.E1246.15050426
Open Access | Editorial and Publishing Policies | Cite | Zenodo | OJS | Indexing and Abstracting
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Purpose: The rapid escalation of ransomware and zero-day malware attacks poses a significant challenge to conventional signature-based detection systems, which cannot generalise to previously unseen threats. This study aims to develop a robust, scalable, and behaviour-aware malware-detection framework capable of accurately identifying ransomware and zero-day attacks across heterogeneous computing environments. Design/methodology/approach: A novel multi-stage hybrid detection pipeline is proposed that integrates advanced feature selection, deep sequential learning, attention mechanisms, and ensemble classification. Initially, irrelevant and redundant features are eliminated using correlation thresholding, Chi-square analysis, mutual information, and variance-based ranking. To capture latent behavioral patterns, a hybrid Gated Recurrent Unit Temporal Convolutional Network (GRU-TCN) architecture is employed to model long- and short-term temporal dependencies. These representations are further refined using squeeze-and excitation attention-enhanced TCN blocks. Finally, an XG-Fusion framework that combines GRU encoding, dilated residual TCNs, attention-based feature fusion, and focal loss optimisation is introduced to address class imbalance, with XGBoost serving as a meta-classifier for final decision-making. Findings: Experimental evaluations conducted on multiple benchmark datasets demonstrate that the proposed framework consistently outperforms traditional machine learning and baseline deep learning models. Superior performance is achieved in terms of accuracy, precision, recall, F1 Score, and ROC AUC. The hierarchical and attention-driven architecture effectively abstracts malicious behavioral patterns and enhances generalization to previously unseen malware variants. Originality: This work introduces a novel multi-stage hybrid deep learning architecture that synergistically combines sequential behavioural modelling, attention-enhanced feature learning, and ensemble-based classification. The proposed approach offers a forward-looking and reliable solution for proactive detection of ransomware and zero-day malware threats.
Keywords: Zero-Day Attack Detection, Ransomware Analysis, Hybrid Deep Learning (CNN-LSTM, VAE), Feature Selection & Behavioural Embedding, t-SNE Visualisation & Entropy-Based Analysis
Scope of the Article: Computer Science and Engineering