John Canny

This talk covers several current projects at the Berkeley Institute of Design (BID) on more natural human-machine interaction. Multiview is a video-conferencing system that preserves eye contact in group situations, and closely mimics face-to-face for certain high-stakes communication tasks. We are pursuing several projects on technology for developing regions. This work covers language learning, story writing, speech interfaces and micro-finance. In this setting, "naturalness" is particularly important and strongly tied to the context of the interaction.

The remainder of the talk will discuss a general framework for natural interaction. The key again is to expose and use context. We argue that context must be studied on 3 planes (roughly time scales). One of these, the activity plane, has been reified in a prototype called CAAD that builds models of user work activity from desktop logs. The other two planes are being explored through current and inter-related projects on natural speech interfaces and story understanding.