Research at Cleveland State Universityt

Smart and Connected Healthcare

Computer vision based motion tracking is a promising technology towards smart and connected healthcare. For example, the technology can be used to monitor the activities of the patients as well as caregivers for various purposes:

  • Automated assessment of the execution of rehabilitation exercises for live feedback or offline analysis. This enables a patient to carry out the prescribed exercises at home for convenience and much reduced cost.
  • Fall detection. An alert could be automatically generated and sent to caregivers on the detection of patient falls.
  • Occupational safety. Caregivers may injure themselves while providing care to patients due to incorrect movements, such as bend and turns. In fact, this is a major reason for work-related injuries for nursing assistants working in nursing homes.

For motion tracking to be useful, the tracked motion data must be analyzed to automatically recover their semantics, i.e., what they mean. This step is referred to as motion recognition. Motion recognition is an open research problem. Researchers have been investigating various methods towards more accurate and faster recognition of human gestures and activities. Most of them depend on the use of machine learning techniques. 

Over the past several years, I have been collaborating with Dr. Reinthal and Dr. Espy in School of Health Science on a project related to realtime motion assessment for rehabilitation exercises using the Microsoft Kinect sensor. In this project, we introduced a kinematic rule based approach to assessing the quality of the execution rehabilitation exercises. I developed a Kinect-based rehabilitation exercise monitoring and guidance system. The system has been used in a number of human trials and obtained moderately good result that showed that the system helped patients to adhere to the requirements better. We are currently investigating the integration of kinematic modeling with fuzzy inference so that we could catch the clinician’s requirements in greater detail and provide easily understood categorical feedback to the patient.

Recently, Dr. Zhao collaborated with Dr. Glenn Goodman and Dr. Beth Ekelman, also in School of Health Science, Dr. Joan Niederriter in School of Nursing, as well as Drs. Reinthal and Espy, on a research project to develop a Kinect-based system to enhance the workers’ compliance to best practices in nursing homes. The project was funded by the Ohio Bureau of Workers’ Compensation on February 2015. They have been working with an external partner, Jennings Center for Older Adults, Cleveland, OH, to conduct extensive human subject trials of the system.
With invaluable input from his collaborators,

Dr. Zhao and his students have developed a Kinect-based system to track the worker’s activities in compliance with best practices, which is currently under intense testing via human subject trials at CSU and at the Jennings. A big thank to Mr. Connor Gordon, an CSU Scholar Student, for creating the first research prototype of the system. Dr. Zhao and his students have demonstrated the technology at several venues, including the OneCommunity [R]IoT event on July 14, 2015, in Cleveland, Ohio,  the World Congress in Computer Science, Computer Engineering and Applied Computing on July 27-28, 2015, in Las Vegas, and the IEEE Smart World Congress on August 10, 2015, in Beijing, China.

A provisional patent has been filed by Cleveland State University where I am the sole inventor in June 2015. In June 2016, CSU has filed a utility patent.

Our projects in this area have been funded by the Ohio Bureau of Workers Compensation, two CSU FRD awards, and several Undergraduate Summer Research awards, I-Corps@Ohio, and CSU/KSU TeCK Fund.

The research so far has resulted in the following publications:

  1. Wenbing Zhao, Qing Wu, Ann Reinthal and Nian Zhang, “Design, Implementation, and Field Testing of a Privacy-Aware Compliance Tracking System for Bedside Care in Nursing Homes,” Applied System Innovation, vol.1, no.1, pp.3, 2018.
  2. Wenbing Zhao, R. Lun, C. Gordon, A. Fofana, D. Espy, A. Reinthal, B. Ekelman, G. Goodman, J. Niederriter, X. Luo, “A Human-Centered Activity Tracking System: Towards a Healthier Workplace,” IEEE Transactions on Human-Machine Systems, vol. 47, no. 3, pp. 343-355, June 2017, doi: 10.1109/THMS.2016.2611825.
  3. Wenbing Zhao, M. Ann Reinthal, Deborah D. Espy, Xiong Luo, “Rule-Based Human Motion Tracking for Rehabilitation Exercises: Realtime Assessment, Feedback, and Guidance,” IEEE Access, vol. 5, issue 1, pp. 21382-21394, October 2017, doi:10.1109/ACCESS.2017.2759801.
  4. Xiongyi Liu, Qing Wu, Wenbing Zhao, and Xiong Luo, “Technology-Facilitated Diagnosis and Treatment of Individuals with Autism Spectrum Disorder: An Engineering Perspective,” Applied Sciences, vol. 7, no. 10, 1051, doi:10.3390/app7101051, October 2017.
  5. W. Zhao, R. Lun, C. Gordon, A. Fofana, D. Espy, A. Reinthal, B. Ekelman, G. Goodman, J. Niederriter, C. Luo, X. Luo, LiftingDoneRight: A Privacy-Aware Human Motion Tracking System for Healthcare Professionals, International Journal of Handheld Computing Research, vol. 7, no. 3, pp. 1-15, July-September 2016.
  6. Q. Wu and W. Zhao, “Towards a Technology-Enabled Environment of Care for Nursing Homes,” Proceedings of the IEEE Cyber Science and Technology Congress, Orlando, FL, November 6-10, 2017, pp. 299-302.
  7. X. Liu and W. Zhao, “Buddy: A Virtual Life Coaching System for Children and Adolescents with High Functioning Autism,” Proceedings of the IEEE Cyber Science and Technology Congress, Orlando, FL, November 6-10, 2017, pp. 293-298.
  8. W. Zhao, Q. Wu, V. Padaraju, M. Bbela, A. Reinthal, D. Espy, X. Luo, and T. Qiu, “A Privacy-Aware Compliance Tracking System for Skilled Nursing Facilities,” Proceedings of the IEEE International Conference on Systems, Man, and Cybernatics, Banff, Canada, October 5-8, 2017, pp. 3568-3573.
  9. W. Zhao, Q. Wu, A. Reinthal, D. Espy, X. Luo, and T. Qiu, “Enhancing Body Mechanics Training for Bedside Care Activities with a Kinect-Based System,” Proceedings of the IEEE International Conference on Systems, Man, and Cybernatics, Banff, Canada, October 5-8, 2017, pp. 3558-3562.
  10. W. Zhao, Q. Wu, D. Espy, A. Reinthal, X. Luo, and Y. Peng, “A Feasibility Study on Using a Kinect-Based Human Motion Tracking System to Promote Safe Patient Handling,” Proceedings of the IEEE International Conference on Electro Information Technology, May 14-17, 2017, pp. 462-466. Lincoln, Nebraska.
  11. W. Zhao, R. Lun, et al., A Privacy-Aware Kinect-Based System for Healthcare Professionals, Proceedings of the IEEE International Conference on Electro Information Technology, Grand Forks, ND, USA, May 19-21, 2016.
  12. W. Zhao, On Automatic Assessment of Rehabilitation Exercises with Realtime Feedback, Proceedings of the IEEE International Conference on Electro Information Technology, Grand Forks, ND, USA, May 19-21, 2016.
  13. W. Zhao, D. Espy, A. Reinthal, B. Ekelman, G. Goodman, J Niederriter, Privacy-Aware Human Motion Tracking with Realtime Haptic Feedback, Proceedings of the 4th IEEE International Conference on Mobile Services, New York, NY, USA, June 27-July 2, 2015, pp. 446-453.
  14. W. Zhao, R. Lun, D. Espy, and M. A. Reinthal, Realtime Motion Assessment for Rehabilitation Exercises: Integration of Kinematic Modeling with Fuzzy Inference, 4(4), October 2014, pp. 267-285.
  15. R. Lun and W. Zhao, A Survey of Applications and Human Motion Recognition with Microsoft Kinect, Internal Journal of Pattern Recognition and Artificial Intelligence, 29 (5), August 2015, pp. 1555008.
  16. W. Zhao, D. Espy, M. A. Reinthal, and H. Feng, A Feasibility Study of Using a Single Kinect Sensor for Rehabilitation Exercises Monitoring: A Rule Based Approach, in Proceedings of the IEEE Symposium on Computational Intelligence in Healthcare and e-Health, Orlando, Florida, USA, December 9-12, 2014, pp. 1-8.
  17. W. Zhao, R. Lun, D. Espy, and M. A. Reinthal, Rule Based Realtime Motion Assessment for Rehabilitation Exercises, in Proceedings of the IEEE Symposium on Computational Intelligence in Healthcare and e-Health, Orlando, Florida, USA, December 9-12, 2014, pp. 133-140.
  18. W. Zhao, H. Feng, R. Lun, D. Espy, and A. Reinthal, A Kinect-Based Rehabilitation Exercise Monitoring and Guidance System, in Proceedings of the 5th IEEE International Conference on Software Engineering and Service Science, Beijing, China, June 27-29, 2014, pp. 762-765.
  19. W. Zhao, D. Espy, A. Reinthal, and H. Feng, Feasibility Study of Using Microsoft Kinect for Physical Therapy Monitoring, Encyclopedia of Information Science and Technology, Third Edition, 2014, pp. 5542-5554.

Dependable Distributed Computing

I have been doing research in the field of dependable distributed computing since 1998. Over these years, I have worked on more than a dozen projects, one of which was sponsored by the National Science Foundation (CNS-0821319, 2008-2012).  Together with my students, we investigated several key issues related to Byzantine fault tolerance for long-running, nondeterministic systems. In particular, we developed a set of mechanisms to reconcile the seemingly conflicting requirements of strong replica consistency and the independency of each individual replica, and designed a method to significantly increase the obtainable concurrency for replicated systems by using software transactional memory. Furthermore, we also proposed a migration-based proactive recovery scheme that ensures a much reduced vulnerability window.

Grown out o the NSF funded project, I am currently investigating what I call application-aware Byzantine fault tolerance. Application-aware Byzantine fault tolerance aims to increase the practicality of Byzantine fault tolerance by exploiting the application semantics. Application-aware Byzantine fault tolerance makes it possible to facilitate concurrent processing of requests, to minimize the use of Byzantine agreement, and to identify and control replica nondeterminism. We have studied several distributed systems using the application-aware Byzantine fault tolerance approach, including Web services coordination (which have led to two IEEE Transactions articles), applications with Conflict-free replicated data types (which have led to an IEEE SCC paper in 2014), and collaborating editing applications. The experiences gained enabled us to come up with a classification of various approaches to application-aware Byzantine fault tolerance with the corresponding guideline on designing mechanisms for significantly improve the system runtime performance (two journal papers).

The following are the most recent publications in this research:

  1. B. Krupp, N. Sridhar, and Wenbing Zhao, “SPE: Security and Privacy Enhancement Framework for Mobile Devices,” IEEE Transactions on Dependable and Secure Computing, vol.14, no.3, pp. 433-446, July-August, 2017.
  2. Elham Akbari, Qing Wu, Wenbing Zhao, Hamid R. Arabnia, Mary Qu Yang, “From Blockchain to Internet-Based Voting,” Proceedings of the International Conference on Computational Science and Computational Intelligence, Las Vegas, NV, December 14-16, 2017, pp. 218-221.
  3. Wenbing Zhao, Mary Yang, “Dependability enhancing mechanisms for integrated clinical environments,” The Journal of Supercomputing, doi:10.1007/s11227-017-2003-0, March 2017.
  4. Mamdouh Babi, Wenbing Zhao, “Towards trustworthy collaborative editing,” Computers, vol. 6, 13; doi:10.3390/computers6020013, 2017.
  5. W. Zhao, M. Babi, W. Yang, X. Luo, Y. Zhu, J. Yang, C. Luo, M. Yang, Byzantine fault tolerance for collaborative editing with commutative operations, in Proceedings of the IEEE International Electro-Information Technology Conference, Grand Forks, ND, pp. 246-351 , May 19-21, 2016.
  6. W. Zhao, “On Intention Preservation in Deterministic Multithreading: A Partial Solution,” IET Software, vol. 10, no. 6, pp. 155-163, 2016.
  7. W. Zhao, Performance optimization for state machine replication based on application semantics: A review, Journal of Systems and Software vol. 112, pp. 96-109, 2016.
  8. W Zhao, W Yang, H Zhang, J Yang, X Luo, Y Zhu, M Yang, C Luo, High-throughput state-machine replication using software transactional memory, The Journal of Supercomputing, vol. 72, no.11, pp. 4379-4398, November 2016.
  9. W. Zhao, Optimistic Byzantine Fault Tolerance, The International Journal of Parallel, Emergent and Distributed Systems, vol. 31, no.3, pp. 254-267, June 2016.
  10. W. Zhao,  Performance optimization for state machine replication based on application semantics: A review, Journal of Systems and Software, 112, 96-109
  11. W. Zhao, “Towards Trustworthy Integrated Clinical Environments,” The 12th IEEE International Conference on Advanced and Trusted Computing, August 10-14, 2015, Beijing, China
  12. H. Chai and W. Zhao, Byzantine Fault Tolerance for Services with Commutative Operations, in Proceedings of the 11th IEEE International Conference on Services Computing, Anchorage, Alaska, USA, June 27 – July 2, 2014, pp. 219-226.
  13. H. Chai and W. Zhao, Byzantine Fault Tolerant Event Stream Processing for Autonomic Computing, in Proceedings of the 12th IEEE International Conference on Dependable, Autonomic and Secure Computing, Dalian, China, August 24-27, 2014, pp. 109-114.
  14. W. Zhao, Application-Aware Byzantine Fault Tolerance, in Proceedings of the 12th IEEE International Conference on Dependable, Autonomic and Secure Computing, Dalian, China, August 24-27, 2014, pp. 45-50.
  15. H. Chai and W. Zhao, Towards Trustworthy Complex Event Processing, in Proceedings of the 5th IEEE International Conference on Software Engineering and Service Science, Beijing, China, June 27-29, 2014, pp. 758-761.
  16. Byzantine Fault Tolerance for Session-Oriented Multi-Tiered Applications, H. Chai and W. Zhao. International Journal of Web Services, vol.2, no.1/2, pp.113-125, 2013.
  17. Trustworthy Coordination for Web Services Atomic Transactions, H. Zhang, H. Chai, Wenbing Zhao, P. M. Melliar-Smith, and L. E. Moser. IEEE Transactions on Parallel and Distributed Systems, Vol. 23, No. 8, 2012, pp. 1551-1565.
  18. Concurrent Byzantine Fault Tolerance for Software-Transactional-Memory Based Applications, H. Zhang and Wenbing Zhao. International Journal of Future Computer and Communication, Vol. 1, No. 1, June 2012, pp. 47-50.
  19. Design and Implementation of a Byzantine Fault Tolerance Framework for Non-Deterministic Applications, H. Zhang, Wenbing Zhao, L. E. Moser and P. M. Melliar-Smith. IET Software, vol. 5, no. 3, 2011, pp. 342-356.
  20. Recovering Lagging Replicas in a Fault Tolerant System, H. Chai and Wenbing Zhao, International Journal of Performability Engineering, Short Communications, vol. 7, no. 2, March 2011, pp. 195-197.
  21. Proactive Service Migration for Long-Running Byzantine Fault Tolerant Systems, Wenbing Zhao and H. Zhang, IET Software, vol. 3, no. 2, April 2009, pp. 154-164.

Cyber Physical Systems

Recently, I started exploring cyber physical systems. I am particularly interested in addressing the safety-critical requirements of such systems, particularly for smart grids and integrated clinical environments.

The following paper has been accepted and will be presented at The 12th IEEE International Conference on Advanced and Trusted Computing:

  • W. Zhao, “Towards Trustworthy Integrated Clinical Environments,” The 12th IEEE International Conference on Advanced and Trusted Computing, August 10-14, 2015, Beijing, China

Research Monograph

Instructional materials for the book

Research Indexing

ResearchGate

Google Scholar

dblp

Course Offered (Spring 2015)

EEC484 Computer Networks

© 2024 Cleveland State University | 2121 Euclid Avenue, Cleveland, OH 44115-2214 | 216.687.2000
Cleveland State University is an equal opportunity educator and employer.