Automatic Digital Rotoscoping using Refined Deep Masks with Instance Segmentation

Abstract

" Quality of movie graphics and scenes is a popular topic among movie critiques. Most of the blockbuster movies are famous due to their extraordinary scenes and special effects. Therefore VFX (special effects) artists tend to use novel techniques to improve the quality of the modern movies. One memorable movie which has remarkable special effects is Jurassic Park (1993). The entire Jurassic Park movie series is the best example to observe the evolution of VFX. Creating high-end visual effects is a complex workflow where each shot is subjected to artistic operations. The key step in this workflow is the process called digital rotoscoping which is manually outlining and cutting out the characters and objects in a raw video frame via software. This is a time consuming process and the average number of frames a professional artist can rotoscope per day is 15. Due to its complexity, it has become a bottleneck of the post production pipeline. Therefore, this research project proposes a novel way to automate the manual rotoscoping process with image processing techniques like instance segmentation. The solution was able to reduce the time taken for manual rotoscoping and now the process is fully automatic. VFX pipeline will be smoother with this solution and it will directly impact the quality of future movie scenes. "