An Intelligent Computer Vision Framework for Eye-State Detection and Adaptive Screen Control
Keywords:
Computer Vision, Machine Learning, Eye Aspect Ratio (EAR), MediaPipe Face Mesh, Adaptive Brightness Control, Human-Computer Interaction, Eye Fatigue Detection, Automatic Sleep Management, Streamlit Dashboard, Real-Time Vision SystemAbstract
The increasing use of computers and digital devices for work, education and entertainment has resulted in prolonged exposure to screens, leading to visual fatigue, excessive energy consumption and reduced user productivity. Traditional brightness adjustment mechanisms in display systems are mostly based on ambient light sensors or manual user input, ignoring the real visual situation and attention of the user. In this work, we present EyeGuard, a smart real-time computer vision framework that continuously monitors the user's eye state through a standard webcam and dynamically adjusts the screen brightness, as well as triggering system sleep mode in the event of prolonged eye closure. The proposed framework utilises MediaPipe Face Mesh for accurate facial landmark detection and calculates the Eye Aspect Ratio (EAR) to estimate the eye openness in real time. We combine Exponential Moving Average (EMA) smoothing and a debouncing mechanism to provide stable classification of the eye-state without spurious brightness changes for robustness against noise and natural blinking. The system recognises the eye conditions in four states, Fully Open, Slightly Closing, Half Closed and Fully Closed, based on the continuously analysed EAR values. The intelligent brightness controller is based on the pre-defined brightness levels of 100%, 60% and 30%. The display intensity is dynamically adjusted according to the detected state of the eye, which can improve visual comfort and reduce power consumption. Moreover, when the continuous eye closure exceeds a user-defined threshold of three seconds, the system automatically turns on the sleep mode of the operating system, which saves energy and increases the security of the workstation. A multithreaded architecture with dedicated modules for detection, graphical user interface, brightness controller, background service, and Streamlit dashboard enables real-time performance with low computational overhead. The system includes a desktop Heads-Up Display (HUD), system tray integration, and a mobile-accessible Streamlit dashboard for improved usability, enabling real-time monitoring of eye status and brightness levels, blink statistics, session duration, and intelligent alert notifications. Experimental results demonstrate the system operates stably at ~30 FPS with 5-15% CPU usage, <200 MB memory footprint, <100 ms brightness response latency, and reliable sleep activation after prolonged eye closure. The presented framework contributes to intelligent human-computer interaction by combining computer vision, machine learning, adaptive display optimisation, and automated power management in a real-time system, with applications in smart offices, learning environments, healthcare workstations, accessibility systems, and energy-efficient computing.
Downloads
References
[1] Veeraiah, V., Pandey, J. K., Das, S., Raju, D., Kumbhkar, M., Khan, H., & Gupta, A. (2023). Integrating IoT-based security with image processing. In D. Pandey, R. Anand, N. Sindhwani, B. K. Pandey, R. Sharma, & P. Dadheech (Eds.), *The impact of thrust technologies on image processing* (pp. 25–57). Nova Science Publishers. https://doi.org/10.52305/ATJL4552
[2] Zhang, Z. *Computer Vision and Human--Computer Interaction*. Springer.
[3] OpenCV Documentation. *Open Source Computer Vision Library*. 2025.
[4] Bradski, G. *The OpenCV Library*. Dr. Dobb's Journal of Software Tools.
[5] Szeliski, R. *Computer Vision: Algorithms and Applications*. Springer, 2022.
[6] Dlib Project. *Machine Learning Toolkit for Face Detection and Landmark Estimation*.
[7] Google. *MediaPipe Face Mesh Documentation*. Google AI, 2025.
[8] Google AI Research. *MediaPipe: A Framework for Building Perception Pipelines*. 2019.
[9] Soukupová, T., & Čech, J. *Real-Time Eye Blink Detection using Facial Landmarks*. Computer Vision Winter Workshop, 2016.
[10] Microsoft. *Windows Power Management API Documentation*. Microsoft Learn.
[11] Python Software Foundation. *Python 3 Documentation*. 2025.
[12] Pandey, J. K., Kotti, J., Parimita, Dhabliya, D., Sharma, V., Choudhary, S., & Anand, R. (2024). Integration of nature-inspired mechanisms to machine learning in real time sensors, controllers, and actuators for industrial automation. In N. Sindhwani, R. Anand, A. George, & D. Pandey (Eds.), *Robotics and automation in Industry 4.0* (pp. 1–25). CRC Press. https://doi.org/10.1201/9781003317456
[13] PyQt5 Documentation. *Python GUI Framework*. Riverbank Computing.
[14] Streamlit Documentation. *Build Interactive Machine Learning Applications*. 2025.
[15] C. Lord, M. Elsabbagh, G. Baird, and J. Veenstra-Vanderweele, "Autism spectrum disorder," *The Lancet*, vol. 392, no. 10146, pp. 508--520, 2018.
[16] C. Lord, T. S. Brugha, T. Charman, J. Cusack, G. Dumas, T. Frazier, E. J. Jones, R. M. Jones, A. Pickles, M. W. State *et al.*, "Autism spectrum disorder," *Nature Reviews Disease Primers*, vol. 6, no. 1, p. 5, 2020.
[17] C.-K. Ting, "Advancing medicine with computational intelligence [editor's remarks]," *IEEE Computational Intelligence Magazine*, vol. 19, no. 3, p. 2, 2024.
[18] L. Zwaigenbaum, M. L. Bauman, R. Choueiri, C. Kasari, A. Carter, D. Granpeesheh, Z. Mailloux, S. Smith Roley, S. Wagner, D. Fein *et al.*, "Early intervention for children with autism spectrum disorder under 3 years of age: recommendations for practice and research," *Pediatrics*, vol. 136, no. Supplement1, pp. S60--S81, 2015.
[19] G. Dawson, S. Rogers, J. Munson, M. Smith, J. Winter, J. Greenson, A. Donaldson, and J. Varley, "Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model," *Pediatrics*, vol. 125, no. 1, pp. e17--e23, 2010.
[20] S. Eldevik, B. Strømgren, S. Eikeseth, A. Fields, C. M. Goetz, and K. B. Titlestad, "Clinically significant outcomes of early intensive behavioral intervention for children with autism spectrum disorders: An individual participant data meta-analysis," *Autism Research*, 2026.
[21] K. Gotham, S. Risi, A. Pickles, and C. Lord, "The autism diagnostic observation schedule: revised algorithms for improved diagnostic validity," *Journal of Autism and Developmental Disorders*, vol. 37, no. 4, pp. 613--627, 2007.
[22] M. Rutter, A. Le Couteur, and C. Lord, *ADI-R: Autism Diagnostic Interview--Revised: Manual*. Los Angeles, CA, USA: Western Psychological Services, 2003.
[23] S. A. Qureshi, G. Dias, M. Hasanuzzaman, and S. Saha, "Improving depression level estimation by concurrently learning emotion intensity," *IEEE Computational Intelligence Magazine*, vol. 15, no. 3, pp. 47--59, 2020.
[24] M. A. N. Ramírez, M. M. Rodríguez, M. J. C. Salas, and E. H. H. Rincón, "Artificial intelligence in diagnosis of pediatric neurodevelopmental disorders: a scoping review," *World Journal of Pediatrics*, pp. 1--15, 2026.
[25] Y. Zhang, Y. Huang, J. Qi, S. Zhang, M. Tian, Y. Tian, F. Meng, L. Guan, and T. Chang, "Uncertainty inspired early autism spectrum disorder screening via contrastive image-viewing paradigm," *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, vol. 33, pp. 122--132, 2025.
[26] T. Falck-Ytter, S. Bölte, and G. Gredebäck, "Eye tracking in early autism research," *Journal of Neurodevelopmental Disorders*, vol. 5, no. 1, p. 28, 2013.
[27] L. Billeci, A. Narzisi, G. Campatelli, G. Crifaci, S. Calderoni, A. Gagliano, C. Calzone, C. Colombi, G. Pioggia, and F. Muratori, "Disentangling the initiation from the response in joint attention: an eye-tracking study in toddlers with autism spectrum disorders," *Translational Psychiatry*, vol. 6, no. 5, p. e808, 2016.
[28] J. N. Constantino, S. Kennon-McGill, C. Weichselbaum, N. Marrus, A. Haider, A. L. Glowinski, S. Gillespie, C. Klaiman, A. Klin, and W. Jones, "Infant viewing of social scenes is under genetic control and is atypical in autism," *Nature*, vol. 547, no. 7663, pp. 340--344, 2017.
[29] T. W. Frazier, M. Strauss, E. W. Klingemier, E. E. Zetzer, A. Y. Hardan, C. Eng, and E. A. Youngstrom, "A meta-analysis of gaze differences to social and nonsocial information between individuals with and without autism," *Journal of the American Academy of Child & Adolescent Psychiatry*, vol. 56, no. 7, pp. 546--555, 2017.
[30] Y. Xu, C. Zhang, B. Pan, Q. Yuan, and X. Zhang, "A portable and efficient dementia screening tool using eye tracking machine learning and virtual reality," *NPJ Digital Medicine*, vol. 7, no. 1, p. 219, 2024.
[31] X.-J. Kong, Z. Wei, B. Sun, Y. Tu, Y. Huang, M. Cheng, S. Yu, G. Wilson, J. Park, Z. Feng *et al.*, "Different eye tracking patterns in autism spectrum disorder in toddler and preschool children," *Frontiers in Psychiatry*, vol. 13, p. 899521, 2022.
[32] W. Jones and A. Klin, "Attention to eyes is present but in decline in 2--6-month-old infants later diagnosed with autism," *Nature*, vol. 504, no. 7480, pp. 427--431, 2013.
[33] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, "Informer: Beyond efficient transformer for long sequence time-series forecasting," *arXiv preprint arXiv:2012.07436*, 2020.
[34] H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long, "TimesNet: Temporal 2d-variation modeling for general time series analysis," in *Proceedings of the Eleventh International Conference on Learning* *Representations*, 2023.
[35] Y. Mu, M. Shahzad, and X. X. Zhu, "MPTSNet: Integrating multiscale periodic local patterns and global dependencies for multivariate time series classification," *arXiv preprint arXiv:2503.05582*, 2025.
[36] A. Stitsyuk and J. Choi, "xPatch: Dual-stream time series forecasting with exponential seasonal-trend decomposition," in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 39, no. 19, 2025, pp. 20 601--20 609.
[37] R. Engbert, A. Nuthmann, E. M. Richter, and R. Kliegl, "Swift: a dynamical model of saccade generation during reading," *Psychological Review*, vol. 112, no. 4, p. 777, 2005.
[38] E. Tjoa and C. Guan, "A survey on explainable artificial intelligence (xai): Toward medical xai," *IEEE Transactions on Neural Networks and Learning Systems*, vol. 32, no. 11, pp. 4793--4813, 2021.
[39] S. Wang, M. Jiang, X. M. Duchesne, E. A. Laugeson, D. P. Kennedy, R. Adolphs, and Q. Zhao, "Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking," *Neuron*, vol. 88, no. 3, pp. 604--616, 2015.
[40] H. Qin, H. Zhu, X. Jin, Q. Song, M. A. El-Yacoubi, and X. Gao, "EmMixformer: Mix transformer for eye movement recognition," *IEEE* *Transactions on Instrumentation and Measurement*, 2025.
[41] G. Uribarri, F. Barone, A. Ansuini, and E. Fransén, "Detach-rocket: sequential feature selection for time series classification with random convolutional kernels," *Data Mining and Knowledge Discovery*, vol. 38, no. 6, pp. 3922--3947, 2024.
[42] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, "Outrageously large neural networks: The sparsely-gated mixture-of-experts layer," *arXiv preprint arXiv:1701.06538*, 2017.
[43] P. Moridian, N. Ghassemi, M. Jafari, S. Salloum-Asfar, D. Sadeghi, M. Khodatars, A. Shoeibi, A. Khosravi, S. H. Ling, A. Subasi *et al.*, "Automatic autism spectrum disorder detection using artificial intelligence methods with MRI neuroimaging: A review," *Frontiers in Molecular Neuroscience*, vol. 15, p. 999605, 2022.
[44] X. Duan, X. Shan, L. Q. Uddin, and H. Chen, "The future of disentangling the heterogeneity of autism with neuroimaging studies," *Biological Psychiatry*, vol. 97, no. 5, pp. 428--438, 2025.
[45] S. Nisar and M. Haris, "Neuroimaging genetics approaches to identify new biomarkers for the early diagnosis of autism spectrum disorder," *Molecular Psychiatry*, vol. 28, no. 12, pp. 4995--5008, 2023.
[46] W. J. Bosl, H. Tager-Flusberg, and C. A. Nelson, "EEG analytics for early detection of autism spectrum disorder: a data-driven approach," *Scientific Reports*, vol. 8, no. 1, p. 6828, 2018.
[47] F. Alizadehziri and A. Khadem, "Deep learning for diagnosing autism spectrum disorder using EEG with upright and inverted face processing tasks," *Biomedical Signal Processing and Control*, vol. 118, p. 109655, 2026.
[48] C. DiStefano, A. Dickinson, E. Baker, and S. S. Jeste, "EEG data collection in children with ASD: The role of state in data quality and spectral power," *Research in Autism Spectrum Disorders*, vol. 57, pp. 132--144, 2019.
[49] S. Bai, J. Z. Kolter, and V. Koltun, "An empirical evaluation of generic convolutional and recurrent networks for sequence modeling," *arXiv* *preprint arXiv:1803.01271*, 2018.
[50] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," *Advances in* *Neural Information Processing Systems*, vol. 30, 2017.
[51] N. Wu, B. Green, X. Ben, and S. O'Banion, "Deep transformer models for time series forecasting: The influenza prevalence case," *arXiv* *preprint:2001.08317*, 2020.
[52] Y. Zhang and J. Yan, "Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting," in *Proceedings of the International Conference on Learning Representations*, 2023.
[53] Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, "iTransformer: Inverted transformers are effective for time series forecasting," *arXiv preprint arXiv:2310.06625*, 2023.
[54] Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, "A time series is worth 64 words: Long-term forecasting with transformers," in *Proceedings of the International Conference on Learning Representations*, 2023.
[55] J. Ye, W. Zhang, Z. Li, J. Li, and F. Tsung, "MedSpaformer: a transferable transformer with multi-granularity token sparsification for medical time series classification," in *Proceedings of the 40th AAAI* *Conference on Artificial Intelligence*, 2026.
[56] Z. Tan, X. Luo, Y. Liu, and Y. Zhang, "Mask the redundancy: Evolving masking representation learning for multivariate time-series clustering," in *Proceedings of the 40th AAAI Conference on Artificial Intelligence*, 2026.
[57] Z. Zhang, Y. Zhang, A. Zeng, D. Pan, Y. Ji, Z. Zhang, and J. Lin, "Time-series data imputation via realistic masking-guided tri-attention bi-gru," in *Proceedings of the European Conference on Artificial Intelligence*, 2023.
[58] T. Xie, Z. Tan, H. Xiao, M. Li, Y. Zhang, Y. Lu, C. Yang, and Y.-m. Cheung, "Anchormoe: Interpretable time series classification via anchor-routed moe," in *Proceedings of the ACM SIGKDD International* *Conference on Knowledge Discovery and Data Mining*, 2026.
[59] Z. Zhang, Y. Zhang, A. Zeng, D. Pan, and X. Zhang, "Learning hierarchical representations in temporal and frequency domains for time series forecasting," in *Proceedings of the Chinese Conference on Pattern* *Recognition and Computer Vision*, 2023.
[60] Z. Tan, T. Xie, B. Sun, X. Zhang, Y. Zhang, and Y.-m. Cheung, "MEET-Sepsis: Multi-endogenous-view enhanced time-series representation learning for early sepsis prediction," in *Proceedings of the Pacific* *Rim International Conference on Artificial Intelligence*, 2025.
[61] T. Xie, Z. Tan, H. Xiao, Y. Zhang, and B. Sun, "DE3S: Dual-enhanced soft-sparse-shape learning for medical early time-series classification," in *Proceedings of the 2025 International Conference on Bioinformatics* *and Biomedicine*, 2025.
[62] S. Raj and S. Masood, "Analysis and detection of autism spectrum disorder using machine learning techniques," *Procedia Computer Science*, vol. 167, pp. 994--1004, 2020.
[63] G. Uribarri, S. E. von Huth, J. Waldthaler, P. Svenningsson, and E. Fransén, "Deep learning for time series classification of parkinson's disease eye tracking data," *arXiv preprint arXiv:2311.16381*, 2023.
[64] J. Kang, X. Han, J. Song, Z. Niu, and X. Li, "The identification of children with autism spectrum disorder by SVM approach on EEG and eye-tracking data," *Computers in Biology and Medicine*, vol. 120, p. 103722, 2020.
[65] W. Zhou, M. Yang, J. Tang, J. Wang, and B. Hu, "Gaze patterns in children with autism spectrum disorder to emotional faces: Scanpath and similarity," *IEEE Transactions on Neural Systems and Rehabilitation* *Engineering*, vol. 32, pp. 865--874, 2024.
[66] M. S. Santos, J. P. Soares, P. H. Abreu, H. Araujo, and J. Santos, "Cross-validation for imbalanced datasets: Avoiding overoptimistic and overfitting approaches [research frontier]," *IEEE Computational Intelligence* *Magazine*, vol. 13, no. 4, pp. 59--76, 2018.
[67] W. Pei, B. Xue, L. Shang, and M. Zhang, "Developing interval-based cost-sensitive classifiers by genetic programming for binary high-dimensional unbalanced classification [research frontier]," *IEEE Com-**putational Intelligence Magazine*, vol. 16, no. 1, pp. 84--98, 2021.
[68] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model," *IEEE Transactions on Neural Networks*, vol. 20, no. 1, pp. 61--80, 2008.
[69] A. Klin, W. Jones, R. Schultz, F. Volkmar, and D. Cohen, "Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism," *Archives of General* *Psychiatry*, vol. 59, no. 9, pp. 809--816, 2002.
[70] K. A. Pelphrey, N. J. Sasson, J. S. Reznick, G. Paul, B. D. Goldman, and J. Piven, "Visual scanning of faces in autism," *Journal of Autism* *and Developmental Disorders*, vol. 32, no. 4, pp. 249--261, 2002.
[71] G. Dawson, S. J. Webb, and J. McPartland, "Understanding the nature of face processing impairment in autism: insights from behavioral and electrophysiological studies," *Developmental Neuropsychology*, vol. 27, no. 3, pp. 403--424, 2005.
[72] D. M. Powers, "Evaluation: from precision, recall and f-measure to ROC, informedness, markedness and correlation," *arXiv preprint* *arXiv:2010.16061*, 2020.
[73] M. Sokolova and G. Lapalme, "A systematic analysis of performance measures for classification tasks," *Information Processing & Management*, vol. 45, no. 4, pp. 427--437, 2009.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal Press
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0): anyone may copy, redistribute, remix, transform, and build upon the material in any medium or format for any purpose, including commercially, provided appropriate credit is given to the original authors and a link to the licence is provided.
