Microsoft Research shows off their IllumiRoom, a new concept of a technology that tries to increase video game immersion by extending the environment of the game beyond the confines of your display. IllumiRoom consists of a modified Kinect motion-sensor and a projector, the system performs a depth-scan of your room and uses the projector to extend the game world beyond the edges of your TV or monitor. This enables you to experience effects like gunfire and explosions that exit the screen and travel across the room, as well as to immerse you in environmental effects like snowflakes or fire.
Full details at Bit Tech.
The result is undeniably incredible: even on a small-screen system, the game world appears to surround the user, covering the player's visual field with images that are not only there to increase immersion but which can also provide an edge during gaming - providing an in-game equivalent to real-life peripheral vision in a similar way to that promised by the Oculus Rift project, but without the need to wear a silly headset.
The basics of IllumiRoom could, of course, be replicated through a standard projector and a modified rendering engine - but the Kinect-powered depth scan provides important information for setting up the system. In its demonstration, Microsoft Research shows the system being used in a realistic living-room environment - complete with a cabinet, shelves and an entertainment unit on which the TV sits. All of these objects provide an uneven surface for the projector which would normally distort the projected image, turning straight lines into a maze of zigzags that would entirely ruin the effect.
To avoid this, IllumiRoom uses the depth data from the Kinect sensor - gathered during the 'scanning room' portion of the system's setup - to pre-distort the image in a such a way that, when projected onto the uneven surface surrounding the TV, the picture appears perfect to the player. The scanning system also detects the size and position of the TV set, ensuring that the projector doesn't attempt to wash out its image with a picture of its own while creating a perfectly-centred illusion of immersion for the player.