« PreviousContinue »
(12) United States Patent ao) Patent No.: Us 7,536,650 Bi
Robertson et al. (45) Date of Patent: May 19,2009
(54) SYSTEM AND METHOD THAT FACILITATES COMPUTER DESKTOP USE VIA SCALING OF DISPLAYED OBJECTS WITH SHIFTS TO THE PERIPHERY
(76) Inventors: George G. Robertson, 1214 E. Hamlin St, #9, Seattle, WA (US) 98102; Eric J. Horvitz, 330 Waverly Way, Kirkland, WA (US) 98033; Daniel C. Robbins, 115 11th Ave. E, Seattle, WA (US) 98102; Mary P. Czerwinski, 14330 178th La. NE, Woodinville, WA (US) 98072; Patrick M. Baudisch, 732 Boylston Ave. E, #7, Seattle, WA (US) 98102; Gregory R. Smith, 13250 NE. 40th St, Bellevue, WA (US) 98005; Brian R. Meyers, 4848 194th Ave. SE, Issaquah, WA (US) 98027
( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 506 days.
(21) Appl.No.: 10/851,459
(22) Filed: May 21, 2004
Related U.S. Application Data
(63) Continuation-in-part of application No. 10/374,351, filed on Feb. 25, 2003.
(51) Int. CI.
G06F 3/048 (2006.01)
(52) U.S. CI 715/767; 715/764; 715/769;
715/768; 715/790; 715/802
(58) Field of Classification Search 715/764 769,
715/790, 802 See application file for complete search history.
(56) References Cited
U.S. PATENT DOCUMENTS
5,305,435 A * 4/1994 Bronson 715/777
FOREIGN PATENT DOCUMENTS
EP 1253510 10/2002
WO 9800787 1/1998
WO WO 01/33889 Al 5/2001
Jonathan Gradin, "Partitioning Digital Worlds: Focal and Peripheral Awareness in Multiple Monitor Use," vol. 3, No. 3, Issue No. 1, ACM 2001.*
(Continued) Primary Examiner—Tadeese Hailu
(7'4) Attorney, Agent, or Firm—Amin, Turocy & Calvin, LLP (57) ABSTRACT
The present invention relates to a system that facilitates multitasking in a computing environment. A focus area component defines a focus area within a display space—the focus area occupying a subset area of the display space area. A scaling component scales display objects as a function of proximity to the focus area, and a behavior modification component modifies respective behavior of the display objects as a function their location of the display space. Thus, and more particularly the subject invention provides for interaction technique^) and user interface(s) in connection with managing display objects on a display surface. One aspect of the invention defines a central focus area where the display objects are displayed and behave as usual, and a periphery outside the focus area where the display objects are reduced in size based on their location, getting smaller as they near an edge of the display surface so that many more objects can remain visible. In addition or alternatively, the objects can fade as they move toward an edge, fading increasing as a function of distance from the focus area and/or use of the object and/or priority of the object. Objects in the periphery can also be modified to have different interaction behavior (e.g., lower refresh rate, fading, reconfigured to display sub-objects based on relevance and/or visibility, static, etc.) as they may be too small for standard rendering. The methods can provide a flexible, scalable surface when coupled with automated policies for moving objects into the periphery, in response to the introduction of new objects or the resizing of pre-existing objects by a user or autonomous process.
25 Claims, 105 Drawing Sheets
K. S. Booth et al., "On the Parameters of Human Visual Performance an Investigation of the Benefits of Antialiasing," ACM 2001. * Leung, et al. "A Review and Taxonomy of Distortion-Oriented Presentation Techniques," ACM transaction On Computer-Human Interaction, vol. 1, No. 2, Jun. 1994, pp. 126-160 * Andy Harter, et al., A Distributed Location System for the Active Office, IEEE Network, 1994, pp. 62-70.
Guarding Chen, et al., A Survey of Context-Aware Mobile Computing Research, Dartmouth Computer Science Technical Report, 2000, 16 pages.
William Noah Schilt, A System Architecture for Context-Aware
Mobile Computing, Columbia University, 1995, 153 pages.
Mike Spreitzer, et al., Providing Location Information in a Ubiqui-
tous Computing Environment, SIGOPS '93, 1993, pp. 270-283.
Marvin Theimer, et al., Operating System Issues for PDAs, In Fourth
Workshop on Workstation Operating Systems, 1993, 7 pages.
Roy Want, Active Badges and Personal Interactive Computing
Objects, IEEE Transactions on Consumer Electronics, 1992, 11
pages, vol. 38—No. 1.
Bill N. Schilit, et al., The ParcTab Mobile Computing System, IEEE
WWOS-IV, 1993, 4 pages.
Bill Schilit, et al., Context-Aware Computing Applications, In Proceedings of the Workshop on Mobile Computing Systems and Applications, Dec. 1994. pp. 85-90.
Bill N. Schilit, et al., Customizing Mobile Applications, Proceedings USENIX Symposium on Mobile and Location Independent Computing, Aug. 1993, 9 pages.
Mike Spreitzer, et al., Architectural Considerations for Scalable. Secure, Mobile Computing with Location Information, In The 14fh International Conference on Distributed Computing Systems, Jun. 1994, pp. 29-38.
Mike Spreitzer et al., Scalable, Secure, Mobile Computing with Location Information, Communications of the ACM, Jul. 1993, 1 page, vol. 36—No. 7.
Roy Want, et al., The Active Badge Location System, ACM Transactions on Information Systems, Jan. 1992, pp. 91-102, vol. 10—No. 1.
Mark Weiser, Some Computer Science Issues in Ubiquitous Computing, Communications of the ACM, Jul. 1993, pp. 75-84, vol. 36—No. 7.
M. Billinghurst, et al., An Evaluation of Wearable Information Spaces, Proceedings of the Virtual Reality Annual International Symposium, 1998, 8 pages.
Bradley J. Rhodes, Remembrance Agent: A continuously running automated information retrieval system, The Proceedings of The First International Conference on The Practical Application Of Intelligent Agents and Multi Agent Technology, 1996, pp. 487-495. Eric Horvitz, et al., In Pursuit of Effective Handsfree Decision Support: Coupling Bayesian Inference, Speech Understanding, and User Models, 1995, 8 pages.
Bradley J. Rhodes, The Wearable Remembrance Agent: A System for
Augmented Theory, The Proceedings of The First International Sym-
posium on Wearable Computers, Oct. 1997, pp. 123-128.
Eric Horvitz, et al., Attention-Sensitive Alerting in Computing Sys-
tems, Microsoft Research, Aug. 1999.
Bill N. Schilit, et al., Disseminationg Active Map Information to
Mobile Hosts, IEEE Network, 1994, pp. 22-32, vol. 8—No. 5.
Mark Billinghurst, et al., Wearable Devices: New Ways to Manage
Information, IEEE Computer Society, Jan. 1999, pp. 57-64.
Thad Eugene Starner, Wearable Computing and Contextual Aware-
ness, Massachusetts Institute of Technology, Jun. 1999, 248 pages.
Bradley J. Rhodes, The Wearable Remembrance Agent: A System for
Augmented Memory, Personal Technologies Journal Special Issue
on Wearable Computing, 1997, 12 pages.
Workshop on Wearable Computing Systems, Aug. 19-21, 1996.
Mark Billinghurst, Research Directions in Wearable Computing,
University of Washington, May 1998, 48 pages.
Mark Weiser, The Computer for the 21st Century, Scientific Ameri-
can, Sep. 1991, 8 pages.
T. Joachims, Text categorization with support vector machines: learning with many relevant features, Machine Learning, European Conference on Machine Learning, Apr. 21, 1998, pp. 137-142. International Search Report dated Sep. 29, 2003 for PCT Application Serial No. 00/20685, 3 Pages.
Robert M. Losee, Jr., Minimizing information overload: the ranking of electronic messages, Journal of Information Science 15, Elsevier Science Publishers B.V., 1989, pp. 179-189.
Liam Bannon, et al., Evaluation and Analysis of User's Activity Organization, Proceedings of the CHI '83, Dec. 1983, pp. 54-57. Patrick Baudisch, et al., Focus Plus Context Visualization Screens: Combining Display Technology with Visualization Techniques, Proceedings of UIST '01, ACM 2002, pp. 31-40. Benjamin B. Bederson, et al., Pad++: A Zooming Graphical Interface for Exploring Alternative Interface Physics, Proceedings of UIST '94, Nov. 2-4, 1994, pp. 17-26.
Blaine A. Bell, et al., Dynamic Space Management for User Interfaces, Proceedings of UIST '00, ACM Symposium of User Interface Software and Technology, San Diego, California, Nov. 5-8, 2000, pp. 238-248.
SaraA. Bly, etal., A Comparison of Tiled and Overlapping Windows, Proceedings of CHI '86, Apr. 1986, pp. 101-106. Stuart K. Card, et al., A Multiple, Virtual-Workspace Interface to Support User Task Switching. In Proc. CHI+GI 1987, pp. 53-59. Edward Cutrell, etal., Notification, Disruption, and Memory: Effects of Messaging Interruptions on Memory and Performance, HumanComputer Interaction—Interact '01, IOS Press, 2001, pp. 263-269. Mary Czerwinski, et al., Instant Messaging and Interruption: Influence of Task Type on Performance, Proceedings of OZCHI2000, pp. 356-361.
Mary Czerwinski, et al., Instant Messaging: Effects of Relevance and Time, Proceedings of HCI 2000, British Computer Society, 2000, vol. 2, pp. 71-76.
Mary Czerwinski, et al., An Investigation of Memory for Daily Computing Events, Proceedings of HCI 2002, pp. 230-245. Mary Czerwinski, et al., A Diary Study of Task Switching and Interruptions, Proceedings of the Conference on Human Factors in Computing Systems (CHI 2004), Vienna, Austria, Apr. 24-29,2004, vol. 6, No. l,pp. 175-182.
A. Goldberg, Smalltalk-80: Chapter 17-Views, New York: AddisonWesley, 1983, pp. 292-297.
D. Austin Henderson, Jr., et al., Rooms: The Use of Multiple Virtual
Workspaces to Reduce Space Contention in a Window-Based
Graphical User Interface, ACM Transactions on Graphics, vol. 5, No.
3, Jul. 1986, pp. 211-243.
Eser Kandogan, et al., Elastic Windows: Evaluation of Multi-Win-
dow Operations, Proceedings of CHI 1997, pp. 250-257.
Victor Kaptelinin, Umea: Translating Interaction Histories into
Project Contexts, Proceedings of CHI 2003, Ft. Lauderdale, Florida,
Apr. 5-10, 2003, volume No. 5, Issue No. 1, pp. 353-360.
Blaire Maclntyre, et al., Support for Multitasking and Background
Awareness Using Interactive Peripheral Displays, Proceedings of
UIST 2001, Orlando, Florida, Nov. 11-14, 2001, ACM 2001, vol. 3,
No. 2, pp. 41-50.
Thomas W. Malone, How Do People Organize Their Desks? Implications for the Design of Office Information Systems, ACM Transactions on Office Information Systems, Jan. 1983, vol. 1, No. 1, pp. 99-112.
Brad A. Myers, Window Interfaces: A Taxonomy of Window Manager User Interfaces, IEEE Computer Graphics and Applications, Sep. 1988, vol. 8, No. 5, pp. 65-84.
Jun Rekimoto, Time-Machine Computing: A Time-centric Appraoch for the Information Environment, Proceedings of UIST '99, pp. 45-54.
George Robertson, et al., The Task Gallery: A 3D Window Manager, Proceedings of CHI '00, Apr. 1-6, 2000, vol. 2, Issue 1, pp. 494-501. Greg Smith, et al., GroupBar: The TaskBar Evolved, Proceedings of OZCHI '03, 2003, 10 pages.
Warren Teitelman, Ten Years of Window Systems—A Retrospective View, in F. Hopgood, et al. (Eds.), Methodology of Window Management, Berlin: Springer-Verlag, 1986, pp. 35-46. European Search Report for European Application No. EP04004253, dated Oct. 26, 2006. 3 pages.
XDESK Software, About Virtual Desktop Managers, Available at: http://www.virtual-desktop.info/, last accessed on May 17, 2006, 3 pages.
Tim Pattison, et al., "Information Visualization Using Component
Layouts and Visual Sets," Copyright 2001, Conference in Research
and Practice in Information Technology, vol. 9, Commonwealth of
Y. K. Leung, et al. A Review of Taxonomy of Distortion -Oriented
Presentation Techniques, ACM Transactions on Computer-Human
Interaction, vol. 1, No. 2, Jun. 1994, pp. 126-160.
"F-Secure Announces World's First Anti-virus Product for Wireless
Information Devices." pp. 1-2, XP-002220984, http://www.europe.
f-secure.com/news/20. Aug. 10, 2000.
Desney S. Tan, et al., WinCuts: Manipulating Arbitrary Window Regions for More Effective Use of Screen Space, CHI 2004, Apr. 2004, pp. 1525-1528, ACM, Vienna, Austria.
Dugald Hutching, et al., Revisiting Display Space Management:
Understanding Current Practice to Inform Next-generation Design,
Graphics Interface 2004, May 19, 2004, pp. 127-134, Canadian
Human-Computer Communications Society, Ontario.
Nicolas Roussel, Ametista: A Mini-Toolkit for Exploring New Win-
dow Management Techniques, Latin American Conference on
Human-Computer Interaction, Aug. 2003, pp. 117-124, ACM, Rio de
George Robertson, et al., The Task Gallery: A 3D Window Manager,
CHI 2000, Apr. 2000, pp. 494-501, V. 2.1, ACM, The Hague,
Eser Kandogan, et al., Elastic Windows: Evaluation of Multi-Window Operations, CHI 97, 1997, pp. 250-257, ACM, Atlanta, GA. George Robertson, et al., Scalable Fabric: Flexible Task Management, AVI '04, May 2004, pp. 85-89, ACM, Gallipoli, Italy. Francois Guimbretiere, Maureen Stone, and Terry Winograd. Fluid Interaction with High-resolution Wall-size Displays. Symposium on User Interface Software and Technology '01. Nov. 2001. 10 pages. Mary P. Czerwinski, Maarten Van Dantzich, George Robertson, and Hunter Hoffman. The Contribution of Thumbnail Image, Mouseover Text and Spatial Location Memory to Web Page Retrieval in 3D. Interact '99, Edinburgh, UK.
George Robertson, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten Van Dantzich. Data Mountain: Using Spatial Memory for Document Management. Symposium on User Interface Software, UIST '98.
Elizabeth D. Mynatt, Takeo Igarashi, W. Keith Edwards, and Anthony Lamarca. Flatland: New Dimensions in Office Whiteboards. Conference on Human Factors and Computing Systems '99. pp. 346-353. May 1999.
Mary Czerwinski, Susan Dumais, George Robertson, Susan Dziadosz, Scott Tiernan, and Maarten Van Dantzich. Visualizing Implicit Queries For Information Management Retrieval. Conference on Human Factors and Computing 1999. Robert St. Amant, Michael D. Dinardo, and Nickie Buckner. Balancing Efficiency and Interpretability in an Interactive Statistical Assistant. International Conference on Intelligent User Interfaces'03, pp. 181-188, Jan. 2003.
* cited by examiner