Learning with Fewer Labels, Without Sharing Raw Data: A Lightweight Self-Supervised Federated Transformer for IoT Intrusion Detection at the Edge

Authors

Keywords:

Internet of Things (IoT), Intrusion Detection System (IDS), Lightweight Transformer, Self-Supervised Learning, Federated Learning, Model Compression

Abstract

Internet of Things (IoT) ecosystems have grown quickly and dramatically, creating an enormous cybersecurity attack surface that exposes highly distributed networks to sophisticated attacks. However, traditional centralized intrusion detection systems (IDS) and conventional transformer-based models have significant drawbacks in edge environments, such as high computational complexity, privacy issues, and reliance on large amounts of labeled data. To tackle these problems, this study proposes a lightweight and self-supervised federated transformer framework for edge-based intrusion detection in IoT networks. The proposed method combines a multi-task self-supervised pre-training approach to learn discriminative features from unlabeled traffic data to alleviate the lack of labeled data. Given the limited resources of edge devices, an attention-head pruning and INT8 quantization-based compression-aware optimization mechanism is proposed. Moreover, a federated learning paradigm with an effective aggregation protocol is designed to allow for distributed training without sharing raw network traffic. The framework incorporates a robust aggregation architecture designed to handle statistical heterogeneity and establish a structural foundation for resilience against unreliable client updates. The comprehensive evaluations show that the proposed framework achieves an overall classification accuracy of 99.90% and an F1-score of 0.9990 in a centralized setting, and a highly competitive accuracy of 99.87% and an F1-score of 0.9987 under distributed federated conditions. The compression strategy successfully reduces the memory usage of the model by around 79.7% and provides an inference latency of 0.0465ms per sample, demonstrating the computational efficiency and applicability of the model to real-time edge deployment.

Downloads

Download data is not yet available.

References

1. Afrin, S., et al., Industrial Internet of Things: Implementations, challenges, and potential solutions across various industries. Computers in Industry, 2025. 170: p. 104317.

2. Yazdani M, Rassafi AA, Yazdani S. The feasibility study of a smart black spot warning system and its impact on driving speeds in rural highways. International Journal of Intelligent Transportation Systems Research. 2025. 23(2):1301-14.

3. Slaibi, T., N. Ivaki, and M. Vieira, IoT security assessment: A systematic literature review. Journal of Systems and Software, 2026. 237: p. 112833.

4. Rajendran, N., et al., Robust Cyberattack Detection in Smart Grids Using Chaotic Feature Selection and Belief-Guided Deep Learning Models. Energy Storage and Saving, 2026.

5. Komal, A. and S. Li, Intrusion Detection in the Internet of Things: A Comprehensive Review of Techniques, Architectures, Datasets, and Emerging Trends. Sensors (Basel), 2026. 26(11).

6. Bhatia, M. and K.M. Charul, Emerging security paradigms in IoT: A scientometric analysis of research trends and future prospects. Computer Science Review, 2026. 59: p. 100840.

7. Ahanger, T.A., et al., Machine learning-inspired intrusion detection system for IoT: Security issues and future challenges. Computers and Electrical Engineering, 2025. 123: p. 110265.

8. Lakshan Yasarathna, T. and N.-A. Le-Khac, ASEADOS-SDN-IoT: A novel SDN-IoT network intrusion detection dataset and framework. Internet of Things, 2026. 36: p. 101891.

9. Gugueoth, V., S. Safavat, and S. Shetty, Security of Internet of Things (IoT) using federated learning and deep learning — Recent advancements, issues and prospects. ICT Express, 2023. 9(5): p. 941–960.

10. Hossain, M.D., et al., The role of microservice approach in edge computing: Opportunities, challenges, and research directions. ICT Express, 2023. 9(6): p. 1162–1182.

11. Dong, X. and Y. Xie, Research on cloud computing network security mechanism and optimization in university education management informatization based on OpenFlow. Systems and Soft Computing, 2025. 7: p. 200225.

12. Yurdem, B., et al., Federated learning: Overview, strategies, applications, tools and future directions. Heliyon, 2024. 10(19): p. e38137.

13. Pinto, D., et al., A review on intrusion detection datasets: tools, processes, and features. Computer Networks, 2025. 262: p. 111177.

14. Wang, Y., et al., MTRC: A self-supervised network intrusion detection framework based on multiple Transformers enabled data reconstruction with contrastive learning. Journal of Network and Computer Applications, 2025. 243: p. 104300.

15. Mangkunegara, I.S., et al., Transformer models in deep learning: Foundations, advances, challenges and future directions. Buletin Ilmiah Sarjana Teknik Elektro, 2025. 7(2): p. 231–241.

16. Aldhaheri, A., et al., Deep learning for cyber threat detection in IoT networks: A review. Internet of Things and cyber-physical systems, 2024. 4: p. 110–128.

17. Al-Haija, Q.A. and A. Droos, A comprehensive survey on deep learning-based intrusion detection systems in Internet of Things (IoT). Expert Systems, 2025. 42(2): p. e13726.

18. Eshmawi, A.A., A. Aldrees, and R. Alharthi, Smart framework for industrial IoT and cloud computing network intrusion detection using a ConvLSTM-based deep learning model. Frontiers in Computer Science, 2025. Volume 7 - 2025.

19. Akuthota, U.C. and L. Bhargava, Transformer-Based Intrusion Detection for IoT Networks. IEEE Internet of Things Journal, 2025. 12(5): p. 6062–6067.

20. Kheddar, H., Transformers and large language models for efficient intrusion detection systems: A comprehensive survey. Information Fusion, 2025. 124: p. 103347.

21. Neto, E.C.P., et al., Deep learning for intrusion detection in emerging technologies: a comprehensive survey and new perspectives. Artificial Intelligence Review, 2025. 58(11): p. 340.

22. Agrawal, S., et al., Federated Learning for intrusion detection system: Concepts, challenges and future directions. Computer Communications, 2022. 195: p. 346–361.

23. Fu, X., et al., A comprehensive survey of federated learning for intrusion detection systems in Internet of Things environments. Computer Science Review, 2026. 63: p. 101049.

24. Zhang, H., et al., Survey of federated learning in intrusion detection. Journal of Parallel and Distributed Computing, 2025. 195: p. 104976.

25. Rezaei, H., et al., A Survey on Security and Privacy in Federated Learning-Based Intrusion Detection Systems for 5G and Beyond Networks. IEEE Open Journal of the Communications Society, 2026. 7: p. 253–300.

26. Makris, I., et al., A comprehensive survey of Federated Intrusion Detection Systems: Techniques, challenges and solutions. Computer Science Review, 2025. 56: p. 100717.

27. Wang, W., et al., Robust unsupervised network intrusion detection with self-supervised masked context reconstruction. Computers & Security, 2023. 128: p. 103131.

28. Zeng, Y., A contrastive federated semi-supervised learning intrusion detection framework for Internet of Robotic Things. arXiv preprint arXiv:2505.06636, 2025.

29. Saha, S. and L. Xu, Vision transformers on the edge: A comprehensive survey of model compression and acceleration strategies. Neurocomputing, 2025. 643: p. 130417.

30. Jelčicová, Z. and M. Verhelst, Delta keyword transformer: Bringing transformers to the edge through dynamically pruned multi-head self-attention. arXiv preprint arXiv:2204.03479, 2022.

31. Tian, J. and J. He, FEMBNet: Toward Robust and Lightweight Federated Learning for Heterogeneous IoT Devices. Concurrency and Computation: Practice and Experience, 2026. 38(10): p. e70746.

32. Martínez, H., et al., Latency-Critical Quantized Inference With Transformer Decoders on ARM and RISC-V CPUs. IEEE Internet of Things Journal, 2025. 12(13): p. 25676–25690.

33. Ahn, H., et al., Performance characterization of using quantization for DNN inference on edge devices: Extended version. arXiv preprint arXiv:2303.05016, 2023.

34. Yang, C., et al., A Lightweight Decentralized Federated Learning Framework for the Industrial Internet of Things. Ad Hoc Networks, 2025. 178: p. 103962.

35. Mala, K. and H. Annapurna. Enhanced Attention-Based Traffic Transformer for Robust Cloud Network Classification and Intrusion Detection. in 2025 International Conference on Computing Technologies (ICOCT). 2025. IEEE.

36. Li, E., et al., Safe: Self-supervised anomaly detection framework for intrusion detection. arXiv preprint arXiv:2502.07119, 2025.

37. Li, J., R. Cotterell, and M. Sachan, Differentiable Subset Pruning of Transformer Heads. Transactions of the Association for Computational Linguistics, 2021. 9: p. 1442–1459.

38. Ullah, S., et al., Quantized Intrusion Detection for Resource-Constrained IoT: A Comparative Evaluation of Efficiency and Adversarial Robustness. Computers, Materials and Continua, 2026. 88(3).

39. Bilal, M.A., et al., Dataset-centric evaluation of federated intrusion detection models in IoT networks. Scientific Reports, 2026. 16(1): p. 2683.

40. Alayed, W., H.A. Tahir, and W.U. Hassan Application-Oriented Evaluation of Federated Learning for IoT Intrusion Detection Under Non-IID Conditions in Wireless Sensor Networks. Applied Sciences, 2026. 16, 5092 DOI: 10.3390/app16105092.

41. Yang, Z., et al., Personalized federated learning with model interpolation among client clusters and its application in smart home. World Wide Web, 2023. 26(4): p. 2175–2200.

42. Geng, G., T. Cai, and Z. Yang Better Safe Than Sorry: Constructing Byzantine-Robust Federated Learning with Synthesized Trust. Electronics, 2023. 12, 2926 DOI: 10.3390/electronics12132926.

43. Latif, N., W. Ma, and H.B. Ahmad, FedDBC: A density-based defense against collusion attacks in federated intrusion detection for IoT networks. Computer Networks, 2026. 286: p. 112497.

44. Latif, S. and D. Djenouri Adaptive Multi-Model Hierarchical Federated Learning for Robust IoT Intrusion Detection. Sensors, 2026. 26, 3198 DOI: 10.3390/s26103198.

45. Alkazaz, N.R., FedSecure: Robust and Privacy-Preserving Intrusion Detection for IoT Networks Using Federated Learning under Poisoning Attacks and Non-IID Data. International Journal of Safety & Security Engineering, 2026. 16(7): p. 1485.

46. Houichi, M., F. Jaidi, and A. Bouhoula, A trust-aware federated intrusion detection framework for privacy-preserving smart city IoT networks. Computer Networks, 2026. 288: p. 112616.

Publication Timeline

Published

Issue

Section

Articles

How to Cite

Samadzadeh, M., Moeini, A., Arasteh Rad, H., & Badi, A. (2026). Learning with Fewer Labels, Without Sharing Raw Data: A Lightweight Self-Supervised Federated Transformer for IoT Intrusion Detection at the Edge. Journal of Artificial Intelligence, Applications and Innovations. https://aiaijournal.com/index.php/aiai/article/view/114