Title

Real-Time Video Matting For Mixed Reality Using Depth Generated Trimaps

Keywords

Chroma-keying; Depth sensor; Image matting; Mixed reality; Natural image matting; Real-time; Trimap generation

Abstract

In Mixed Reality scenarios, background replacement is a common way to immerse a user in a synthetic environment. Properly identifying the background pixels in an image or video is a difficult problem known as matting. Proper alpha mattes usually come from human guidance, special hardware setups, or color dependent algorithms. This is a consequence of the under-constrained nature of the per pixel alpha blending equation. While the field of natural image matting has made progress finding a least squares solution for an alpha matte, the generation of trimaps, indicating regions of known foreground and background pixels, normally requires human interaction or offline computation. We overcome these limitations by combining a low fidelity depth image that segments the original video signal with a real-time parallel natural image matting technique that favors objects with similar colors in the foreground and background. This allows us to provide real-time alpha mattes for Mixed Reality scenarios that take place in relatively controlled environments. As a consequence, while monochromatic backdrops (such as green screens or retro-reflective material) aid the algorithm's accuracy, they are not an explicit requirement.

Publication Date

6-18-2012

Publication Title

GRAPP 2012 IVAPP 2012 - Proceedings of the International Conference on Computer Graphics Theory and Applications and International Conference on Information Visualization Theory and Applications

Number of Pages

280-288

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

Socpus ID

84862235491 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84862235491

This document is currently not available here.

Share

COinS