Systems, apparatus, methods and computer program products are described for using surround sound to audibly describe the user interface elements of a graphical user interface. The position of each audible description is based on the position user interface element in the graphical user interface. A method is provided that includes identifying one or more user interface elements that have a position within a display space. Each identified user interface element is described in surround sound, where the sound of each description is positioned based on the position of each respective user interface element relative to the display space.
Here's Apple's background and summary of the invention: "Graphical user interfaces (GUIs) typically include a variety of interface elements including, for example, windows, buttons, input boxes, list boxes, labels, tool bars and icons. Generally a GUI is visually presented on a display device (e.g., a monitor or screen). Audibly announcing or reading the elements of a GUI can be helpful to users, particularly those who are visually impaired. Some conventional GUI systems such as Mac OS X operating system from Apple Computer Inc. of Cupertino, Calif. or Microsoft Windows provide accessibility mechanisms to audibly announce the elements of a GUI currently being displayed.
[0003]Some conventional systems provide a visual cue to a user when announcing a particular user interface element. For example, an element can be highlighted (e.g., with an arrow, a border or some other visual effect) contemporaneously with the announcing of the element. The highlighting is intended to indicate where the element being announced is currently located on the display. Visually impaired users, however, may not be able to easily perceive where the element is located on the display using the visual cue.
"Some systems provide visual cues designed to enable vision-able users to identify particular user interface elements. For example, some systems include techniques that allow a user to easily identify where a mouse cursor is within a display. Such techniques are particularly useful with large displays, small cursors or under low-visibility condition. For example, the mouse cursor can be flashed, temporarily enlarged or rendered during motion with contrails (mouse tails). Although these effects can be useful, these visual effects are is still dependent on a user's visual perception.
"Systems, apparatus, methods and computer program products are described below for using surround sound to audibly describe the user interface elements of a graphical user interface. The position of each audible description is based on the position user interface element in the graphical user interface. The details of which are described below in full.
"In one aspect a method is provided that includes identifying one or more user interface elements that have a position within a display space. Each identified user interface element is described in surround sound, where the sound of each description is positioned based on the position of each respective user interface element relative to the display space.
"One or more implementations can optionally include one or more of the following features. The display space can be a currently focused window. The display space can be an entire graphical user interface. The method can include receiving user input to control the identification. Each user interface element can be associated with a description. The description of each user interface element can be visually rendered with each respective user interface element. The method can include translating the description associated with each identified user interface element into sound using text-to-speech where the description includes a character string. The method can include identifying user interface elements automatically. The method can include identifying elements from left to right at a particular vertical position. The method can include indicating that the particular vertical position has changed. The method can include indicating that the next identified element is left of the previously identified element. The method can include indicating that the element has been described. The method can include presenting new user interface elements in the display space; and automatically identifying one or more of the new user interface elements. The method can include presenting a cursor in the display space highlighting the user interface element that is currently being described. Each user interface element can have a depth, where the depth of each user interface element based on a front-to-back ordering of user interface elements of the display space. The method can include identifying audio volume for presentation of the description of each user interface element based on the depth of each respective user interface element. The method can include filtering the presentation of each user interface element's description based on the depth of each respective user interface element. Filtering the presentation of a user interface element occluded by another element can cause the presentation to sound muffled.
"In another aspect a method is provided that includes, in response to user input, identifying a user interface elements having a position within a display space. A sound is generated in surround sound, the position of the sound based on the position of the a user interface elements within the display space.
"One or more implementations can optionally include one or more of the following features. The user interface element can be a mouse cursor. The user input can include moving the mouse cursor.
"In another aspect a method is provided that includes, identifying a first portion of a graphical user interface. The identified portion of the graphical user interface contains a plurality of user interface elements. For each user interface element in the plurality of user interface elements: 1) a position of the user interface element is identified within a second portion of the graphical user interface, 2) a description for the user interface element is identified, and 3) a presentation of the description is positioned in surround sound, where the positioning is based on the identified position.
"One or more implementations can optionally include one or more of the following features. The second portion of the graphical user interface can be the window containing the user interface element. The second portion of the graphical user interface can be the entire graphical user interface. Identifying a description can include converting a textual description associated with the user interface element into audio using text-to-speech.
"Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. Announcing elements using positional audio allows users who are vision impaired to quickly receive position information during each user interface element announcement. A visual cursor highlighting an element that is being announced helps able-sighted users determine which element is being announced. Additional auditory cues indicate the order and progress of user interface element announcements. A positional audio cue that reflects the position of the mouse cursor allows users to easily perceive the location of the mouse cursor even in low visibility conditions."
The inventors are Eric Taylor Seymour, Richard W. Fabrick, Patti Pei-Chin Hoa and Anthony E. Morales. The graphic below is an illustration of an exemplary graphical user interface.
