“Could you help me film a series of seminars and process them so that the lead figure is replaced with an imaginary 3D creature that mimics his actions exactly?” inquired the author of video seminars.
The customer needed a product for a course of psychological training sessions. When watching, one would concentrate his attention on the image bearing some special psychological meaning. For that, we were required to write motion capture software using video stream.
We instantly consented to work on the project and gathered a team of professionals. It took no more than 3 hours to consider possible solution variants, assess the necessary resources, and run the experiment to evaluate the feasibility of the idea. We offered our motion capture solution to the customer and announced the approximate costs the same day.
Marker technology, which is usually used for motion capture, costs a minimum of 10,000 Euros for a set of equipment. We would need motion sensors (minimum 200 Euros), sensors for hands, legs and other body parts, and a controller. The more sensors used, the better the accuracy, but also the higher the price. However, our solution involved markerless technology with fully programmatic image capturing from several cameras at the same time.
To improve the accuracy of the motion capture and the creature’s movements, the actor was recorded with several cameras using a homogenous background. The lighting also played an important role. It was adjusted in a certain way to make realistic shade and to add naturalness to the picture.
We placed particular emphasis on the host’s head for more precise movement capture. It was necessary to use a motion sensor to recognize the host’s head position.
The most time-consuming part was the processing of the recording. It was made entirely using video card resources. Each second of recorded video took about 20 to 30 seconds to process, which was too long. The problem was solved by installing a high-performance video card.