Research at Cleveland State University

Smart and Connected Healthcare 

Computer vision based motion tracking is a promising technology towards smart and connected healthcare. For example, the technology can be used to monitor the activities of the patients as well as caregivers for various purposes:
Automated assessment of the execution of rehabilitation exercises for live feedback or offline analysis. This enables a patient to carry out the prescribed exercises at home for convenience and much reduced cost.

Fall detection. An alert could be automatically generated and sent to caregivers on the detection of patient falls.

Occupational safety. Caregivers may injure themselves while providing care to patients due to incorrect movements, such as bend and turns. In fact, this is a major reason for work-related injuries for nursing assistants working in nursing homes.
For motion tracking to be useful, the tracked motion data must be analyzed to automatically recover their semantics, i.e., what they mean. This step is referred to motion recognition. Motion recognition is an open research problem. Researchers have been investigating various methods towards more accurate and faster recognition of human gestures and activities. Most of them depend on the use of machine learning techniques. 

Over the past several years, I have been collaborating with Dr. Reinthal and Dr. Espy in School of Health Science on a project related to realtime motion assessment for rehabilitation exercises using the Microsoft Kinect sensor. In this project, we introduced a kinematic rule based approach to assessing the quality of the execution rehabilitation exercises. My students and I developed a Kinect-based rehabilitation exercise monitoring and guidance system. The system has been used in a number of human trials and obtained moderately good result that showed that the system helped patients to adhere to the requirements better. We are currently investigating the integration of kinematic modeling with fuzzy inference so that we could catch the clinician's requirements in greater detail and provide easily understood categorical feedback to the patient.

Recently, I collaborated with Dr. Glenn Goodman and Dr. Beth Ekelman, also in School of Health Science, Dr. Joan Niederriter in School of Nursing, as well as Drs. Reinthal and Espy, on a research project to develop a Kinect-based system to enhance the workers' compliance to best practices in nursing homes. The project was funded by the Ohio Bureau of Workers' Compensation (BWC) on February 2015. We will be working with an external partner, Jennings Center for Older Adults, Cleveland, OH, to conduct extensive human subject trials of the system that I will be developing to track the worker's activities in compliance with best practices.

The technology foundation of our BWC grant has already been developed by me and my two undergraduate summer research students, Connor Gordon and Abou-Bakar Fofana. A provisional patent has been filed by Cleveland State University where I am the sole inventor in June 2014. We demonstrated the technology at a OneCommunity [R]IoT event on July 14, 2015. I recently demonstrated the technology again at WorldComp on July 28, 2015 in Las Vegas.

Our projects in this area have been funded by the Ohio Bureau of Workers Compensation, two CSU FRD awards, and several Undergraduate Summer Research awards.

The research so far has resulted in the following publications:

  1. W. Zhao, R. Lun, D. Espy, and M. A. Reinthal, Realtime Motion Assessment for Rehabilitation Exercises: Integration of Kinematic Modeling with Fuzzy Inference, 4(4), October 2014, pp. 267-285.
  2. R. Lun and W. Zhao, A Survey of Applications and Human Motion Recognition with Microsoft Kinect, Internal Journal of Pattern Recognition and Artificial Intelligence, 29 (5), August 2015, pp. 1555008.
  3. W. Zhao, D. Espy, A. Reinthal, B. Ekelman, G. Goodman, “Privacy-Aware Human Motion Tracking with Realtime Haptic Feedback,” The 4th IEEE International Conference on Mobile Services, New York, NY, USA, June 27-July 2, 2015, pp. 446-453.
  4. W. Zhao, D. Espy, M. A. Reinthal, and H. Feng, A Feasibility Study of Using a Single Kinect Sensor for Rehabilitation Exercises Monitoring: A Rule Based Approach, in Proceedings of the IEEE Symposium on Computational Intelligence in Healthcare and e-Health, Orlando, Florida, USA, December 9-12, 2014, pp. 1-8.
  5. W. Zhao, R. Lun, D. Espy, and M. A. Reinthal, Rule Based Realtime Motion Assessment for Rehabilitation Exercises, in Proceedings of the IEEE Symposium on Computational Intelligence in Healthcare and e-Health, Orlando, Florida, USA, December 9-12, 2014, pp. 133-140.
  6. W. Zhao, H. Feng, R. Lun, D. Espy, and A. Reinthal, A Kinect-Based Rehabilitation Exercise Monitoring and Guidance System, in Proceedings of the 5th IEEE International Conference on Software Engineering and Service Science, Beijing, China, June 27-29, 2014, pp. 762-765.
  7. W. Zhao, D. Espy, A. Reinthal, and H. Feng, Feasibility Study of Using Microsoft Kinect for Physical Therapy Monitoring, Encyclopedia of Information Science and Technology, Third Edition, 2014, pp. 5542-5554.
The following journal article has been submitted and currently under view:

  1. W. Zhao, R. Lun, D. Espy, and M. A. Reinthal, A Validity Study on Using a Single Kinect Sensor to Assess the Sit-to-Stand Exercise.

Dependable Distributed Computing

I have been doing research in the field of dependable distributed computing since 1998. Over these years, I have worked on more than a dozen projects, one of which was sponsored by the National Science Foundation (CNS-0821319, 2008-2012).  Together with my students, we investigated several key issues related to Byzantine fault tolerance for long-running, nondeterministic systems. In particular, we developed a set of mechanisms to reconcile the seemingly conflicting requirements of strong replica consistency and the independency of each individual replica, and designed a method to significantly increase the obtainable concurrency for replicated systems by using software transactional memory. Furthermore, we also proposed a migration-based proactive recovery scheme that ensures a much reduced vulnerability window.

Grown out o the NSF funded project, I am currently investigating what I call application-aware Byzantine fault tolerance. Application-aware Byzantine fault tolerance aims to increase the practicality of Byzantine fault tolerance by exploiting the application semantics. Application-aware Byzantine fault tolerance makes it possible to facilitate concurrent processing of requests, to minimize the use of Byzantine agreement, and to identify and control replica nondeterminism. We have studied several distributed systems using the application-aware Byzantine fault tolerance approach, including Web services coordination (which have led to two IEEE Transactions articles), applications with Conflict-free replicated data types (which have led to an IEEE SCC paper in 2014), and collaborating editing applications. The experiences gained enabled us to come up with a classification of various approaches to application-aware Byzantine fault tolerance with the corresponding guideline on designing mechanisms for significantly improve the system runtime performance (two journal papers).

The following are the most recent publications in this research:

  1. W. Zhao, Optimistic Byzantine Fault Tolerance, The International Journal of Parallel, Emergent and Distributed Systems, in press.
  2. W. Zhao,  Performance optimization for state machine replication based on application semantics: A review, Journal of Systems and Software, 112, 96-109
  3. W. Zhao, “Towards Trustworthy Integrated Clinical Environments,” The 12th IEEE International Conference on Advanced and Trusted Computing, August 10-14, 2015, Beijing, China
  4. H. Chai and W. Zhao, Byzantine Fault Tolerance for Services with Commutative Operations, in Proceedings of the 11th IEEE International Conference on Services Computing, Anchorage, Alaska, USA, June 27 – July 2, 2014, pp. 219-226.
  5. H. Chai and W. Zhao, Byzantine Fault Tolerant Event Stream Processing for Autonomic Computing, in Proceedings of the 12th IEEE International Conference on Dependable, Autonomic and Secure Computing, Dalian, China, August 24-27, 2014, pp. 109-114.
  6. W. Zhao, Application-Aware Byzantine Fault Tolerance, in Proceedings of the 12th IEEE International Conference on Dependable, Autonomic and Secure Computing, Dalian, China, August 24-27, 2014, pp. 45-50.
  7. H. Chai and W. Zhao, Towards Trustworthy Complex Event Processing, in Proceedings of the 5th IEEE International Conference on Software Engineering and Service Science, Beijing, China, June 27-29, 2014, pp. 758-761.
  8. Byzantine Fault Tolerance for Session-Oriented Multi-Tiered Applications, H. Chai and W. Zhao. International Journal of Web Services, vol.2, no.1/2, pp.113-125, 2013.
  9. Trustworthy Coordination for Web Services Atomic Transactions, H. Zhang, H. Chai, Wenbing Zhao, P. M. Melliar-Smith, and L. E. Moser. IEEE Transactions on Parallel and Distributed Systems, Vol. 23, No. 8, 2012, pp. 1551-1565.
  10. Concurrent Byzantine Fault Tolerance for Software-Transactional-Memory Based Applications, H. Zhang and Wenbing Zhao. International Journal of Future Computer and Communication, Vol. 1, No. 1, June 2012, pp. 47-50.
  11. Design and Implementation of a Byzantine Fault Tolerance Framework for Non-Deterministic Applications, H. Zhang, Wenbing Zhao, L. E. Moser and P. M. Melliar-Smith. IET Software, vol. 5, no. 3, 2011, pp. 342-356.
  12. Recovering Lagging Replicas in a Fault Tolerant System, H. Chai and Wenbing Zhao, International Journal of Performability Engineering, Short Communications, vol. 7, no. 2, March 2011, pp. 195-197.
  13. Proactive Service Migration for Long-Running Byzantine Fault Tolerant Systems, Wenbing Zhao and H. Zhang, IET Software, vol. 3, no. 2, April 2009, pp. 154-164.

Cyber Physical Systems

Recently, I started exploring cyber physical systems. I am particularly interested in addressing the safety-critical requirements of such systems, particularly for smart grids and integrated clinical environments.

The following paper has been accepted and will be presented at The 12th IEEE International Conference on Advanced and Trusted Computing:

  • W. Zhao, “Towards Trustworthy Integrated Clinical Environments,” The 12th IEEE International Conference on Advanced and Trusted Computing, August 10-14, 2015, Beijing, China