|
Dybuster
A multimedia based writing training software for dyslexics
Word graph and color code of the word "Dy-bus-ter" in Dybuster
Content
The concept of the learning-software Dybuster
Prof. Dr. Markus Gross and MSc Christian Vögeli developed a multi-media learningsoftware
for dyslexic children at the Computer Science Department at ETH Zurich.
The central idea of the learning software, called Dybuster, is to recode a sequential
textual input string into a multimodal representation using a set of novel codes.
These codes reroute textual information through multiple undistorted perceptual
cues. There are topological, color and shape as well as auditory representations.
The entire framework is based on information theory and utilizes a Markovian language
model derived from linguistic analysis of language corpora for English,
French, and German. The spatio-topological code is computed by recursive parsing
tree of the string and decomposes it into words, syllables, and individual letters.
The appearance code assigns appearance attributes (colors and shapes) to each
symbol. An additional auditory code assigns Midi events to each symbol and thus
generates a melody for each input string.
Conceptual components of the method and framework.
Dybuster includes three different games. In the first game, namely the color game,
the users have to learn the association between a letter and a color. Based on the
information theory model of Dybuster, eight different colors are used. The mapping
of letters to colors is the result of a multi-objective optimization, taking into
account that e.g. letters, which dyslexics easily confuse, such as “t” and “d”, map to
different colors. Other aspects considered include a uniform distribution of the colors
over the alphabet, maximizing the information content of the letter to color assignment
measured as entropy. The idea is that associating colors and letters will
allow eliminating mistakes.
In the second game, namely the graph game, the users have to segment the word
into its syllables and letters graphically. In the third game, the actual learning
game, Dybuster presents all alternative representations of a word before the users
enter the word itself using the keyboard: the graph appears on screen, and the colors
and the shapes (spheres for small letters, cylinders for capital letters, and pyramids
for umlauts) are displayed for all letters. A voice dictates the word and the users
hear a melody computed from the involved letters and the lengths of the syllables.
The words are organized in modules, where each module consists of 100
words. The words are ordered according to their frequency and difficulty for dyslexics,
so that the users start learning the most frequent and most easy words. For effective
training word selection adapts to the user’s state and optimizes the learning
process by estimating and minimizing error entropy.
Videos
Dybuster in action
This video shows the tree games of Dybuster. First the color game is played, then Dybuster switches automatically to the graph game and finally to the learn voc game. In the last one, several errors are made and different options are used.
The video is available in a higher and a lower resolution.
Windows Media File (ca. 2.2 MB)
Windows Media File (ca. 8 MB)
Children working with Dybuster
This video clip shows children working with Dybuster during a training session of the user study. The children are generally working on their own. However, they are advised of the Dybuster concept if necessary.
The video is available in two different formats:
Windows Media File
Quicktime Movie
|
Webmaster