李开复-自然用户界面

李开复的文献,值得看看

Towards a Natural User InterfaceKaiKai-Fu Lee Corporate Vice President Natural Interactive Services Division Microsoft Corporation

李开复的文献,值得看看

Talk OutlineThe NUI revolutionThe vision. The ingredients and technologies.

5 challenges to getting NUI started 10 steps to overcome challenges Conclusion

李开复的文献,值得看看

User Interface Evolution

Search Engines Hyperlinks Multiple Windows Menus Command line1985 PC 1990 GUI

Graphical User Interface

1995 Internet

李开复的文献,值得看看

User Interface EvolutionGraphical User InterfaceGUI TechnologyDirect manipulationMouse / keyboard input Mouse input mapped to Windows, menus, etc. Keyboard input (search) mapped to text index

GUI InteractionUser is the expert and knows EnablesWhat functions exist Where to go What to click (and in what order) What to type

李开复的文献,值得看看

User Interface EvolutionNatural User InterfacePersonal Assistant Multimodal (speech, ink…) Graphical Natural Language User Interface Search Engines Hyperlinks Multiple Windows 2005 Any place, Menus Any time, 2000 Any device Command line1985 PC 1990 GUI 1995 Internet XML Web Services

李开复的文献,值得看看

NUI – “Do What I Mean” UIUsers naturally articulate what they mean, on any device, to any application or web service, and have their intention interpreted and executed accurately. Why NUI?Expressive. Natural. Scalable.

李开复的文献,值得看看

Natural User InterfaceMotivated by human-human interaction humanInteracting with a Librarian

“Information about Chicago” “Chicago the city, college or music group?” “City” “History or travel information?” “Travel”

“Who knows about traveling to Chicago?” Books, magazines, other libraries… Present choices

李开复的文献,值得看看

Natural User Interface (NUI)NUI Technology1. Hear what you sayTyped, spoken, written Enables

NUI Interaction User asks naturally says / types:“Find information “Information about Chicago”

Text 2. Know what you meanAnalyze text Consider context Clarify using dialog

about Chicago”

“I want to travel to Chicago”

3. Do what you wantBroker and execute by finding content / services

“Book my flight to Chicago using Expedia”

李开复的文献,值得看看

NUI: Hear What You SayNUI Technology1. Hear what you sayTyped, spoken, written Enables

NUI Interaction User says / types:“Informationabout Chicago” “I want to travel to Chicago”

Text 2. Know what you meanAnalyze text Consider context Clarify using dialog

3. Do what you wantBroker and execute by finding content / services

“Book my flight to Chicago using Expedia”

李开复的文献,值得看看

Hear What You SayHuman language : “Invented” for interaction“[Language is] a biological adaptation to communicate information… One of nature’s engineering marvels” – Steven Pinker “Vision evolved from the need to survive; speech evolved from the need to communicate” – Michael Dertouzos.

Speech is central…… but also typing, handwriting, gestures… Best form depends on:Habit / skills (typing to PCs) Form factor (speaking to phones) Situation (writing at meetings)

Eventually, multimodal

李开复的文献,值得看看

NUI: Kno

w What You MeanNUI Technology1. Hear what you saidTyped, spoken, written Enables

NUI Interaction User says / types:“Informationabout Chicago” “I want to travel to Chicago”

Text 2. Know what you meanAnalyze text Consider context Clarify using dialog

3. Do what you wantBroker and execute by finding content / services

“Book my flight to Chicago using Expedia”

李开复的文献,值得看看

Know What You Mean – by combining:Syntax (rules of the human’s language)Nouns, verbs, etc. and how they combine“Book about a trip to Chicago” vs. “Book a trip to Chicago”

Normalize linguistic variations .

SemanticsMeaning of the wordsBook means reserve a ticket; requires from-city, to-city, etc.

Context (additional hints)Domain knowledge :No train from Hawaii to Chicago

Statistics : Book as a noun > Book as a verb“Book Chicago”

Personal Preferences :Where you live, your calendar, how you pay…

Model of time, urgency, presence

Dialog (resolving ambiguity & determine intent)“Buy a book or book travel?” “What date would you like to travel?”

李开复的文献,值得看看

NUI: Do What You WantNUI Technology1. Hear what you saidTyped, spoken, written Enables

NUI Interaction User says / types:“Informationabout Chicago” “I want to travel to Chicago”

Text 2. Know what you meanAnalyze text Consider context Clarify using dialog

3. Do what you wantBroker and execute by finding content / services

“Book my flight to Chicago using Expedia”

李开复的文献,值得看看

Do What You Want(by brokering & combining) No system can know everything. Broker intent to experts:Take user intent and determine“Who are the possible experts for this intent?”

Each expert (web service) registers what it knowsEncarta knows about history, geography, … Expedia knows about travel (to various places…) Amazon knows about books, book reviews…

Combine all experts’ answers,Content (like a search) – Find history of Chicago. Action (like a service) – Buy a ticket to Chicago.

李开复的文献,值得看看

NUI Will Enable…Find the Bill Gates book on future When is the Britney Spears concert? How do I replace my printer cartridge? Buy the Gladiator DVD for less than $15 Send flowers to mom on her birthday Continue working on my annual report

李开复的文献,值得看看

Challenges to NUITechnology limitations User reluctance Unproven business value High development effort More than UI – infrastructure needed

So, how do we make NUI work? 10 evolutionary steps.

李开复的文献,值得看看

Challenges to NUITechnology limitations User reluctance Unproven business value High development effort More than UI – infrastructure needed

李开复的文献,值得看看

1. Change the world, one domain at a timeGeneral understanding is hard. Domain knowledge can:Reduce linguistic ambiguity: proceeds, IRA… Reduce semantic ambiguity: I want to check in. Can get benefit even for “very big domains”

Use of contextual cues helps further.

Domain Constraints Explicit Input Context = implicit input

“Reasoning Engine”

李开复的文献,值得看看

2. Don’t try to solve all the problems at once….

Probably Solvable No

w

Long-Term Research Problems

李开复的文献,值得看看

3. Be religious about solution, but be pragmatic about technologyDon’t fall in love with one technology! Take whatever technology works. If there are several approaches,Take the simplest! Rule of 80-20 could be 99-1 for hi-tech! 8099hi-

Combine them if that helps.

李开复的文献,值得看看

4. Use UI to hide technology imperfections.Push the intelligenceintelligencerequired problems back to the human. Request-andRequest-and-choose UINatural (in search). Top N precision, not top 1 precision.

Other UI tricks….

本文来源：https://www.bwwdw.com/article/cysm.html