To capture human motion, the most common way for almost all human movement research labs is to use a marker-based motion capture system. Markers attached to the subject body allow all the cameras to know the locations of body parts to reconstruct the movement in 3D. However, the markers are preventing this kind of system from being used in many applications as it requires a lot of subject preparation time, manual post-processing time, and a few humans in the loop. Therefore, this project aims to build a human motion capture system without attaching any markers or sensors on the human subject while maintaining the tracking accuracy that is comparable to the marker-based motion capture system.

This can be done in a data-driven way using modern neural-network-based machine learning techniques. Basically, we teach a computation model to predict the 2D location of markers on the synchronized videos of a markerless subject and strategically triangulate them to get 3D position of those virtual markers. To create the training dataset, our annotations on the video are done with high precision using data from a marker-based motion capture system. In addition, the scale of our training data is already bigger than any public datasets of the same kind and it is continuously growing to enhance the accuracy and robustness of the trained model.