Accessibility Issues in Two Specifications for E-Learning Tests: IMS QTI 1.2 and IMS QTI 2.0

Introduction

Web accessibility concentrates on issues such as

• accessibility of content (and guidelines, e.g. WCAG),
• accessibility of user agents (and guidelines, e.g. UAAG),
• accessibility of authoring tools (and guidelies, e.g. ATAG),
• assistive technologies and adapted devices.

Another issue is the accessibility of content formats:
Accessibility features in user agents and authoring tools build on accessibility features in content format.

Accessibility of Content Formats

Reviewing specifications for content formats and add accessibility features:

• to enable the creation of accessible content,
• to make the job of user agents developers easier.

Reviewing can take several forms:

• implement the format and document issues (e.g. QTI 1.2 in VISUAL project),
• read the specification and find (missing) accessibility features and related ambiguities.

Work on W3C's XML Accessibility Guidelines (XAG) has stopped.

Example: Question and Test Interoperability (QTI) by the IMS Global Learning Consortium.

IMS in Ten Seconds

IMS mission statement:

To promote the development of global distributed learning through the cooperative and collaborative efforts of universities, non-profit organizations, agencies of governments, companies and other organizations that make a definitive, substantial and continuing commitment to the development, evolution and/or use of advanced distributed learning environments.

Global distributed learning requires interoperability in many areas. XML-based specifications.

IMS Specifications

• IMS Guidelines for Developing Accessible Learning Applications
• IMS Question & Test Interoperability [QTI]
• IMS Learner Information Package [LIP]
  • IMS Learner Information Package Accessibility for LIP [ACCLIP]
    for storing learner's accessibility preferences and coupling them to any aspect of an e-learning environment
• IMS Access For All Metadata Specification

Question and Test Interoperability (QTI)

XML-based format for

• question data (“items”)
• test data (“assessments”)
• their corresponding result reports

Items can be used for

• assessments, evaluations
• interactive content

QTI Item Types

Items are categorized according to type of answer (identifier, string, number, order).

• 20 basic item types in IMS QTI specification
• 9 types can be handled with pure HTML:
  True/False, Multiple Choice, Multiple Response, Fill-in-Blank
• 11 types require extra or embedded interface:
  Multiple Choice with audio, item types with hot spot, sliders or drag and drop
• composite types combine two or more basic item types

Rendering QTI Items

• For each item, the specification always specifies one and only one rendering.
• IMS QTI only knows visual/graphical renderings,
  in spite of
  • emergence of voice user interfaces,
  • research on multimodal interfaces,
  • the growing awareness of accessibility and inclusive design.
• For items that would need applets or other embeded interfaces, the specification is silent on accessibility issues.

Example Issue: Alternative Text

Item type “standard multiple choice (images)”:

Code sample:
<response_label ident="A"> <material> <matimage imagtype="image/gif" uri="mchc_ir_002_image1.gif"/> <altmaterial> <mattext> Round sign with white background, a red border… </mattext> </altmaterial> </material> </response_label>

Problems:

• there can be any number of images and altmaterial elements;
• altmaterial element is not required (unlike the alt attribute in HTML);
• QTI 1.2 does not support explicit associations between images and text alternatives.

The same problems also apply to item types with other media (audio, video, …).

Accessibility and Test Validity

Equivalent versus Alternative Access

equivalent access

provides the disabled user with content identical to that used by the nondisabled user

alternative access

provides the disabled user with a learning activity that differs from the activity used by the non-disabled user

(IMS Guidelines for Developing Accessible Learning Applications)

Equivalent versus Alternative Assessment

• Challenge in alternative assessments: assessing the same learning outcomes as with the original method.
• Equivalent assessment should be provided whenever possible, but …
  accessibility features might conflict with validity constraints.
• Validity: are you testing what you claim to be testing?
• In some cases, accessibility features overcome threats to validity;
• in other cases, accessibility features can pose threats to validity.
• You need to know the “objectives” of the test (QTI's objectives element).

QTI and Voice Interaction

In VISUAL: efforts to adapt several interaction types to a voice user interface.

Interaction with a voice user interface (VUI) is different from the interaction of sighted users with a Web interface.

• HTML: two-dimensional layout; VUI: time dimension;
• whole-page units versus smaller units (steps,forms, prompts, …);
• HTML: may present many options at once; VUI: limit options to ease the burden on the user's memory.

QTI only considers visual renderings.

• Do voice-based renderings conform to the specification?
• Rendering can be relevant to the objectives; voice interface may be inappropriate or even impossible.
• Need for alternative, voice-based assessment. No guidance in QTI specification.

QTI and Voice Interaction: Example: Order Objects

• straightforward for sighted users (order is visible at any time),
  but not in a voice user interface
• burden on memory during the test:
  • order built up so far
  • remaining (unordered) items
• Process:
  1. The learner is first presented with the complete set of objects to be ordered and asked to identify the first object in the sequence.
  2.  
     • If the learner's answer is correct, the system confirms that the answer is correct (e.g. “You answered that X is the 2nd item in the sequence. Your answer is correct”) and enumerates the objects that remain to be ordered.
     • If the leaner's answer is incorrect, the system confirms that the answer is incorrect and enumerates the objects that remain to be ordered.
  3. The system prompts the learner to add the next object to the sequence, and goes to step 2.

QTI 2.0

• Thorough overhaul of the language.
• Uses many elements from HTML, a language with well-understood accessibility features.
• Alternative text for images: HTML's object element.
• Only visual renderings are considered; names of question types suggest a rendering rather than a learning outcome (hotspotInteraction, drawingInteraction).
• Equivalent tests would often make the outcome invalid.

Conclusions

• QTI 1.2 and 2.0 have left certain aspects about the allowed renderings and the intent of questions undefined. This leads to ambiguity about adaptations for people with disabilities.
• More guidance is needed on the relation between interaction types and the intent of the tests.
• Reviewing accessibility features of technical specifications is important! (And IMS does have Guidelines for Developing Accessible Learning Applications…)