This FAQ explains how makers of Linux- or UNIX- based screen readers, voice dictation packages, onscreen keyboards, magnification software and other assitive technologies can support Gecko-based software. The base of our support for these products is AT-SPI (Assistive Technology Service Provider Interface) and the keyboard API/user interface.
For Firefox and all other Gecko-based products: this documentation only applies to up-to-date builds of Firefox -- currently not an official releases. Grab the current build of Firefox which supports these features
Here are some basic definitions that you'll need for this document to make sense:
The AT-SPI tree and the DOM tree are parallel structures, although the AT-SPI tree is
a subset of the DOM tree. QueryInterface()
can be used to switch between
the interfaces (Accessible, AccessibleText, AccessibleValue, etc.).
What is exposed? Any DOM node that ...
Gecko is a rendering engine that Firefox, SeaMonkey, Netscape and yelp use. Gecko can render a variety of content, not just HTML and supports key web standards such as Cascading Style Sheets, Javascript and the W3C DOM. Gecko also handles the users keystrokes and mouse clicks. Gecko is the core architecture that we are adding accessibility to, in order to support basic accessibility in all applications that are based on it.
The Mozilla Gecko engine must be version 1.9 or later to have high quality AT-SPI support, as documented here.
Embedded clients use Gecko only in the content window, at the moment for HTML and generic XML only. They typically use standard Windows controls for their user interface -- the area outside of the client content window, plus the context menu.
XUL-based clients make full use of the Gecko architecture, not only for HTML content, as well as for menus, dialogs and the entire user interface via an XML language called XUL (eXtensible User-interface Language). None of the user interface contains standard Windows controls -- not even the menus! This is done to ensure a common look and feel across all supported platforms, and to allow for different skins (appearances).
See the about:accessibilityenabled Firefox extension.
In XUL-based clients, screen readers may need to find the content window so that they know where to start grabbing the AT-SPI tree, in order to load the current document into a buffer in their own process.
Use RELATION_EMBEDS on the ROLE_FRAME, which is at the root of each top level window. This will point to the root content accessible(s) for that XUL window.
When you see the content window receive focus, first check the role. If it is a ROLE_DOCUMENT then this should be treated as a document for the default modality of the screen reader. If it is a ROLE_EMBEDDED, ROLE_DIALOG or ROLE_ALERT then stay in focus tracking mode -- there is no need to parse the document. In addition, if it is a ROLE_ALERT, a screen reader should treat it as a message box -- that is, to read the entire contents. These roles can occur on content because of the new Accessible DHTML technology which allows the author to specify the type of document or container.
Gecko also helps determine when to load a new window by firing the following document events: document:load-complete, document:reload and document:load-stopped
Interface | Supported | Notes |
---|---|---|
Accessible | Yes | See supported object attributes below |
Action | Yes | All methods |
Application | Yes | All methods |
Component | Yes | All methods |
Desktop | No | |
Document | Yes | All methods? |
EditableText | Yes | All methods |
Event | Yes | See supported events below |
Hyperlink | Yes | Any object embedded in text is considered a hyperlink! |
Hypertext | Yes | Any text with objects embedded in it is considered a hypertext! |
Image | No | ROLE_IMAGE is supported, and is enough |
LoginHelper | No | |
Registry | No? | |
Relation | Yes | See supported relations below |
Role | Yes | See supported roles below |
Selection | Yes | Not yet supported for text |
Selector | No | |
State | Yes | See supported states below |
StreamableContent | No | Could apply to plugins, but probably not objects rendered by Gecko |
Table | Yes | In addition, the object attribute layout-guess=true when Gecko thinks the table is for layout, not for data |
Text | Yes | See supported text attributes below |
Value | Yes |
The following roles are currently supported:
** XXX Needs updating -- this list is copied from MSAA document ***
Role | Supported? | Special features |
---|---|---|
ROLE_INVALID | Not a role that can be supported | |
ROLE_ACCELERATOR_LABEL | No | |
ROLE_ALERT |
XUL: <browsermessage> DHTML: xhtml2:role="wairole:alert" |
|
ROLE_ANIMATION | No. Animated images use ROLE_GRAPHIC with STATE_ANIMATED. | |
ROLE_ARROW | No | |
ROLE_CALENDAR | No | |
ROLE_CANVAS |
No *** XXX File bug to support for <canvas> *** |
|
ROLE_CHECK_BOX |
XUL: <checkbox> HTML: <input type="checkbox"> DHTML: role="wairole:checkbox" |
Fires Fires object:state-changed when radiobutton is set/unset when checkbox is toggled |
ROLE_CHECK_MENU_ITEM |
XUL: <menuitem type="checkbox"> DHTML: *** XXX file bug to support via role="wairole:menuitemcheckbox" *** |
Supports STATE_CHECKED |
ROLE_COLOR_CHOOSER | No | |
ROLE_COLUMN_HEADER |
XUL: tree column headers HTML: <th> DHTML: role="wairole:columnheader" |
|
ROLE_COMBO_BOX |
XUL: <menulist> HTML: <select size="1"> DHTML: role="wairole:combobox" |
What event do we fire for changes in closed combo box? |
ROLE_DATE_EDITOR | No | |
ROLE_DESKTOP_ICON | No | |
ROLE_DESKTOP_FRAME | No | |
Role | Supported? | Special features |
ROLE_DIAL | No | |
ROLE_DIALOG |
XUL: <dialog> |
accessible name exposes the <title> of the current dialog |
ROLE_DIRECTORY_PANE | No | |
ROLE_DRAWING_AREA | No | |
ROLE_FILE_CHOOSER | Supported for native filepicker | |
ROLE_FILLER | No | |
ROLE_FONT_CHOOSER | No | |
ROLE_FRAME | Top level window | |
ROLE_GLASS_PANE | No | |
ROLE_HTML_CONTAINER | No | |
ROLE_ICON | No | |
ROLE_IMAGE |
XUL: <image> HTML: <img> |
XUL: <image> |
ROLE_INTERNAL_FRAME | No | |
ROLE_LABEL |
XUL: <label> HTML: <label> DHTML: role="wairole:label" |
Supports RELATION_LABEL_FOR (although technically anthing can) |
ROLE_LAYERED_PANE | No | |
ROLE_LIST | XUL: <listbox> | |
ROLE_LIST_ITEM |
XUL: <listitem> HTML: <li>, <option> or <optgroup> DHTML: role="wairole:listitem" |
Sets STATE_SELECTED if the current listitem is selected. *** XXX Perhaps should support object attribute "level" when in ROLE_TREE or ROLE_TREE_TABLE, see object attributes section *** |
ROLE_MENU |
XUL: <menuitem> with children DHTML: role="wairole:menuitem" with child menuitems *** XXX does this work? *** |
|
ROLE_MENU_BAR |
XUL: <menubar> DHTML: role="wairole:menubar" |
|
Role | Supported? | Special features |
ROLE_MENU_ITEM | ||
ROLE_OPTION_PANE | No | |
ROLE_PAGE_TAB |
XUL: <tab> DHTML: role="wairole:tab" |
|
ROLE_PAGE_TAB_LIST | ||
ROLE_PANEL |
XUL: <radiogroup>, <groupbox>, <iframe> HTML: <fieldset>, <frame>, <iframe> DHTML: role="wairole:group", role="wairole:radiogroup" |
|
ROLE_PASSWORD_TEXT |
XUL: <textbox type="password"> HTML: <input type="password"> DHTML: role="wairole:secret" |
|
ROLE_POPUP_MENU | No, apparently we use ROLE_MENU even for the context menu. XXX *** Is this right? *** | Supports Text and EditableText interfaces |
ROLE_PROGRESS_BAR |
XUL: <progressmeter> DHTML: role="wairole:progressbar" |
Fires "object:property-change:accessible-value" when progressbar moves by at least 3% Supports Value interface |
ROLE_PUSH_BUTTON | ||
ROLE_RADIO_BUTTON |
XUL: <radio> HTML: <input type="radio"> DHTML: role="wairole:radio" |
|
ROLE_RADIO_MENU_ITEM |
XUL: <menuitem type="radio"> DHTML: *** XXX file bug to support via role="wairole:menuitemradio" * |
Supports STATE_CHECKED |
ROLE_ROOT_PANE | No | |
ROLE_ROW_HEADER | DHTML: role="wairole:columnheader" | |
ROLE_SCROLL_BAR | Not yet, bug 285167 | |
ROLE_SCROLL_PANE | No | |
Role | Supported? | Special features |
ROLE_SEPARATOR | XUL: <separator> | |
ROLE_SLIDER |
XUL:slider should be supported soon via
bug 109215 DHTML: role="wairole:slider" |
Fires "object:property-change:accessible-value" when slider thumb is moved Supports Value interface |
ROLE_SPIN_BUTTON | DHTML: role="wairole:spinbutton" |
Fires "object:property-change:accessible-value" when changed Supports Value interface |
ROLE_SPLIT_PANE | No | |
ROLE_STATUS_BAR | XUL: <statusbar> | |
ROLE_TABLE |
HTML: <table> DHTML: role="wairole:grid" |
Supports Table interface Supports object attribute "layout-guess", see object attributes section |
ROLE_TABLE_CELL |
HTML: <td> DHTML: role="wairole:gridcell" |
*** XXX Perhaps should support object attribute "level" when in ROLE_TREE or ROLE_TREE_TABLE, see object attributes section *** |
ROLE_TABLE_COLUMN_HEADER |
HTML: <th> DHTML: role="wairole:columnheader" *** XXX Actually, do we use this one or ROLE_COLUMN_HEADER? *** |
|
ROLE_TABLE_ROW_HEADER | DHTML: role="wairole:rowheader" *** XXX Actually, do we use this one or ROLE_ROW_HEADER? *** | |
ROLE_TEAROFF_MENU_ITEM | No | |
ROLE_TERMINAL | No | |
ROLE_TEXT |
HTML: fallback for display-block styled elements with no known semantic role for
the tag DHTML: role="wairole:description" XXX *** We are using ROLE_TEXT for DHTML descriptions but ROLE_LABEL for XUL ones, that's inconsistent XXX |
Supports Text interface |
ROLE_TOGGLE_BUTTON | No | |
ROLE_TOOL_BAR |
XUL: <toolbar> DHTML: role="wairole:toolbar" |
|
ROLE_TOOL_TIP | XUL: <tooltip> or tooltiptext attribute | |
ROLE_TREE |
No, XXX *** We should use this when there is only 1 column in the tree *** DHTML: ? |
|
ROLE_TREE_TABLE |
XUL: <tree> DHTML: role="wairole:tree" ? |
|
ROLE_UNKNOWN | No | |
ROLE_VIEWPORT | No | |
ROLE_WINDOW | No | |
ROLE_HEADER | No | |
Role | Supported? | Special features |
ROLE_FOOTER | No | |
ROLE_PARAGRAPH | HTML: <p> | |
ROLE_RULER | No | |
ROLE_APPLICATION | Root accessible object, parent of top level ROLE_FRAME's | |
ROLE_AUTOCOMPLETE | XUL: <textbox type="autocomplete"> | |
ROLE_EDITBAR | No | |
ROLE_EMBEDDED | DHTML: role="wairole:application" | |
ROLE_ENTRY |
XUL: <textbox> HTML: <input type="text"> or <textarea> DHTML: role="wairole:textfield" or role="wairole:textarea" |
Supports Text interface |
ROLE_CHART | No | |
ROLE_CAPTION | HTML: <caption> | |
ROLE_DOCUMENT_FRAME |
HTML: <body> without a role on the <html> or <body> element (unless
that is wairole:document) DHTML: role="wairole:document" |
Supports Document interface |
ROLE_HEADING | HTML: <h1>, <h2>, <h3>, <h4>, <h5>, <h6> |
Supports Text interface Supports object attribute "level", see object attributes section *** |
ROLE_PAGE | No | |
ROLE_SECTION | HTML: <div> | Supports Text interface |
ROLE_FORM | HTML: <form> | Supports Text interface |
ROLE_REDUNDANT_OBJECT | No | |
ROLE_LINK |
XUL: <label class="text-link"> HTML: <a>, <area> DHTML: role="wairole:link" |
|
ROLE_INPUT_METHOD_WINDOW | No | |
Role | Supported? | Special features |
The following states are currently supported:
State | Supported? | Where used |
---|---|---|
STATE_INVALID | No | Not a usable state -- should not be confused with STATE_INVALID_ENTRY for form controls, which is supported |
STATE_ACTIVE | Yes | Top level window |
STATE_ARMED | No | *** XXX Where should this be supported? Same as MSAA's STATE_HASPOPUP? *** |
STATE_BUSY | Yes | Documents which are loading |
STATE_CHECKED | Yes | Checkboxes as well as checkable menuitems and treeitems |
STATE_COLLAPSED | Yes | Outline items which have children but are not expanded -- always used with STATE_EXPANDABLE |
STATE_DEFUNCT | Yes | Old objects which are no longer available |
STATE_EDITABLE | Yes | Anything that supports the EditableText interface |
STATE_ENABLED | Yes | Form controls |
STATE_EXPANDABLE | Yes | Outline items which have children |
STATE_EXPANDED | Yes | Outline items which are expandable and the children are shown -- always used with STATE_EXPANDABLE |
STATE_FOCUSABLE | Yes | Nearly any element can be focusable |
STATE_FOCUSED | Yes | Only 1 item at a time is ever focused -- always used with STATE_FOCUSABLE |
STATE_HAS_TOOLTIP | No | *** XXX Need to file bug *** |
STATE_HORIZONTAL | No | *** XXX need to file a bug for this and STATE_VERTICAL -- check XUL CSS *** |
STATE_ICONIFIED | No | |
STATE_MODAL | Yes | Modal dialog boxes |
STATE_MULTI_LINE | Yes | ROLE_ENTRY -- always used with STATE_EDITABLE |
STATE_MULTISELECTABLE | Yes | ROLE_TREE, ROLE_LIST, ROLE_TREE_TABLE. Indicates that Enter will insert a new line. |
STATE_OPAQUE | No | |
STATE_PRESSED | Yes | ROLE_BUTTON |
STATE_RESIZABLE | No | |
STATE_SELECTABLE | Yes | ROLE_TREEITEM, ROLE_TABLE_CELL, ROLE_LIST_ITEM |
STATE_SELECTED | Yes | ROLE_TREEITEM, ROLE_TABLE_CELL, ROLE_LIST_ITEM -- always used with STATE_SELETABLE |
STATE_SENSITIVE | No | Where should this be used? |
STATE_SHOWING | Yes | Anything |
STATE_SINGLE_LINE | Yes | ROLE_ENTRY -- always used with STATE_EDITABLE. Indicates that Enter will submit the default button. |
STATE_STALE | No | Where should this be used? |
STATE_TRANSIENT | Yes | Where should this be used? |
STATE_VERTICAL | No | *** XXX need to file a bug for this and STATE_HORIZONTAL -- check XUL CSS *** |
STATE_VISIBLE | Yes | Absence of this state is important on documents, popups and tree items |
STATE_MANAGES_DESCENDANTS | No | Not needed, although potentially useful for trees |
STATE_INDETERMINATE | No | *** XXX File bug to support this based on nsIAccessible::STATE_MIXED. Should work on ROLE_CHECK_BOX, ROLE_TREE_ITEM, ROLE_LIST_ITEM, ROLE_PROGRESS_BAR *** |
STATE_REQUIRED | Yes | XForms form controls, and any element with aaa:required="true" |
STATE_TRUNCATED | No | Where should we use this? |
STATE_ANIMATED | Yes | Animated images. |
STATE_INVALID_ENTRY | Yes | XForms form controls, and any element with aaa:invalid="true" |
STATE_SUPPORTS_AUTOCOMPLETION | No | *** XXX File bug to support this *** |
STATE_SELECTABLE_TEXT | Yes | Anything that supports the Text interface |
STATE_IS_DEFAULT | Yes | Default ROLE_BUTTON's, such as OK in XUL dialogs or Submit in HTML |
STATE_VISITED | Yes | ROLE_LINK |
The following relations are currently supported:
enum {RELATION_LABEL_FOR = 0x1002 };
enum {RELATION_DESCRIPTION_FOR = 0x100f };
These two relations can be used on object to determine what form control is being labelled or desribed.
enum {RELATION_LABELLED_BY = 0x1003 };
enum {RELATION_DESCRIBED_BY = 0x100e };
These two relations are they inverse; they can be used on form controls. If the form control has an accName, you can get the IAccessible that it was labelled by in order to get more formatting information. It is also useful to check for a description.
Note that the label and description relations may be used to prevent redundant information from being presented by the screen reader, since the label and description can occur both on their own, and in the name or description fields of an IAccessible.
enum {RELATION_EMBEDS = 0x1009 };
This relation is used on the root accessible object for a top level Mozilla window, corresponding to what's returned for OBJID_CLIENT for that window. It points to the accessible object corresponding to the root of content in that window. This relation is very useful for finding the content quickly, and will be the proper method for finding content in Firefox 3 and beyond.
enum {RELATION_CONTROLLED_BY = 0x1000 };
enum {RELATION_CONTROLLER_FOR = 0x1001 };
These two relations show what form controls may dynamically change areas of the document, in response to user changes in the form controls themselves. Both controlled_by and controller_for are set in markup from the single dynamic content accessibility dynamic content aaa:controls attribute. The inverse controlled_by relation is automatically calculated.
enum {RELATION_FLOWS_TO = 0x1006 };
enum {RELATION_FLOWS_FROM = 0x1007 };
These two relations allow the reading flow to break out of the normal DOM flow. Both flows_to and flows_from are set in markup from the single dynamic content accessibility aaa:flowsto relation -- the inverse flow_from relation is automatically calculated.
The following object attributes are currently supported:
Object attribute | Applies to (content/UI/any/[role]) | Possible values |
---|---|---|
tag | any | The actual markup tag used to create this element (also used in XUL) |
xml-roles | any | If a dynamic content accessibility role string is used, it is exposed here. This may provide more information than the AT-SPI role, which is best-fit. In the future, this may be a space or comma delimited list of roles |
id | any | Any value, defined by UI/content developers. Used to indicate a persistant identifier for any object, useful for scripting |
layout-guess | ROLE_TABLE | "true" when Gecko's heuristic determines that it is a table that is probably used for layout, not for table. Value not set means it is probably a data table. |
level | ROLE_HEADING | The heading level |
The following text attributes are currently supported:
Text attribute | Possible values |
---|---|
static | "true" for list bullet/numbering text or layout-inserted text (such as via CSS pseudo styles :before or :after) |
The following events are currently supported:
The document: events are not supported yet but should be soon.
The event object:link-selected is purposely not supported, but focus events are fired on ROLE_LINK objects instead.
As just stated, the event object::link-selected is purposely not supported, but focus events are fired on ROLE_LINK objects instead.
See Firefox 3 dependencies, and in particular the New ATK blocking bugs.
*** XXX To Be Done ***
It is the assistive technology's responsibility to watch for events that indicate when windows or content subtrees are being destroyed, and to release all Accessible objects related to that window. In addition, STATE_DEFUNCT is set on objects that should be released by the assistive technology. XXX *** Indicate what events, I think we need to add object:children-changed:foo support ***
To help developers in that regard, there is Memory Leak monitor, a Firefox extension.
Fortunately, Gecko uses the standard keyboard API's for each supported platform.
The Mozilla keyboard shortcuts for content are similar to what is used in other browsers. Here is a list of Firefox keyboard shortcuts.
HTML content | HTML source | AT-SPI representation |
---|---|---|
This is a headingThis is a paragraph with an image in it. This is another heading |
<h1>This is a heading</h1> |
{parent Text, role=ROLE_HEADING, objattr="xhtml:tag=h1", |
Hey! |
<p>Hey!<br>Tell me something.</p> |
{parent Text, role=ROLE_PARAGRAPH, attr="xhtml:tag=p"} |
Hey! Tell me something. |
<a href="http://www.google.com">Hey!
|
{parent Hyperlink, Text, role=ROLE_LINK |
Hey Tell me something |
<p>Hey</p><hr/>
|
{parent1 Text, role=ROLE_PARAGRAPH, text="Hey"} {parent2 role=ROLE_SEPARATOR, [note: State doesn't include STATE_VERTICAL, to distinguish from vsep]} {parent3 Text, role=ROLE_PARAGRAPH, text="Tell me something"} |
You are a nice person. |
<p> |
{parent Text,
role=ROLE_PARAGRAPH, attr="html:tag=p", text="You are a nice person", attribute run for "are", with objattr="role = html:em", textattr="css:text-style=oblique"} |
Here is a bartending site. |
<p> |
{parent Hypertext,
role=ROLE_PARAGRAPH, attr="html:tag=p" text="Here is a *.", attribute run for "bartending site." with textattr="link=true, css:text-decoration=underline, css:color=(0,0,255)"} {child Text, Hyperlink, role=ROLE_LINK, attr="html:tag=a, link-type:anchor", text="bartending site", hyperlink-indices=[10,11] hyperlink-URI="http://foo.bar.com"} |
Here is a bartending site. |
<p> |
{parent Hypertext,
role=ROLE_PARAGRAPH, attr="html:tag=p" text="Here is a *." attribute run for "bartending site." with textattr="link=true, css:text-decoration=underline, css:color=(0,0,255)"} {child Hypertext, Hyperlink, role=ROLE_LINK, text="*bartending site" hypertext-indices=[10,11], [not sure if we need to dup textattrs here, or add them to defaulttextattrs] , hypertext-URI="http://foo.bar.com"} {grandchild Image, Hyperlink role=ROLE_IMAGE, attr="html:tag=img, link-type=image" AccName/ImageDescription="beer glass", hyperlink-indices=[0,1] URI="beerglass.GIF"} [don't know if the URIs should always be fully specified, or if omitting the base URI is OK At the moment, not planning to do this, instead plan to expose repair text in the name if no alt/title exists ] |
Here is a bartending site . |
<p> |
{parent Hypertext,
role=ROLE_PARAGRAPH, attr="html:tag=p", text="Here is a **."} {child Image, Hyperlink, role=ROLE_IMAGE, attr="html:tag=img, link-type=image" AccName/ImageDescription="beer glass", hyperlink-indices=[10,11] hyperlink-URI="beerglass.GIF"} {child Text, Hyperlink, Action, Hypertext, role=ROLE_LINK, action-names="activate", [others?] attr="html:tag=a, link-type=anchor" text="bartending site", textattr=[as in above examples] hypertext-indices=[11,22], hypertext-URI="http://foo.bar.com"} |
<p> |
{parent Hypertext, role=ROLE_PARAGRAPH, |
|
|
<ul> |
[Bill: we should be able to support list-style=image, and "list-style-image=URL()", etc. this way. In the above example, it's not clear whether the bullet should be a unicode char or just omitted and implied by the list style.. my guess is the latter (i.e. bullets don't appear in the text)] |
|
<ol> |
{parent Object, role=ROLE_LIST, attr="html:tag=ol, css:list-style-type:decimal"} |
|
<ul> |
|
<form> |
{parent Object, role=ROLE_FORM, text=""} |
|
<form aaa:describedby="checkhelp"> |
{parent Object, |
|
<form> |
{parent role=ROLE_FORM?} |
|
<form> |
{parent role=ROLE_FORM?} |
|
To do: HTML table example | ||
To do: DHTML role example |
You may have heard of some content types beyond HTML, and want to know if Gecko based products will support them:
Please discuss accessibility issues on the Mozilla Accessibility mailing list or on the Mozilla Accessibility IRC channel.