Investigation on Effect of Prosody in Finger Braille Chiba University, Japan Manabi Miyagi, Masafumi Nishida, Yasuo Horiuchi, Akira Ichikawa Topics * Communication of gDeafblindh * Features of gFinger brailleh * Prosody in time structure * Data recording and analysis * Experiment and result * Future work gDeafblindh * People who are both deaf and blind * About One in 6000 people * Deafblind with serious impairments use tactile media * Ways of communication of the deafblind * A translator is needed to talk to others * Hard to get information from mass media Finger Braille How to Type Finger Braille Features of Finger Braille * Easy to learn * Easy to process on digital devices * Accurate and fast * 350 syllables/minute can be transmitted by skilled translators => Prosody information is also transmitted What is Prosody Information? Do you understand the meaning of sentences at once? "thecourseoftimehasleftthissmallcountriesasthecentreofapowerpowerfulempirethisisnoticeablefirstofallinthecitiesamongthegreat buildingsthecapitalviennaisashowpieceinitseifwithitssplendidplaceslikethehofburgandschonbrunnandthemedievalststephenscathedral" How about this? I believe you understand better. "The course of time has left this small countries as the centre of a power powerful empire. This is noticeable first of all in the cities, among the great buildings. The capital Vienna, is a showpiece in itself, with its splendid places like the Hofburg and Schonbrunn, and the medieval St.Stephenfs Cathedral." (From gWELCOME TO AUSTRIAh) Because space, comma, period are used in the second paragraphs, it is much easier to understand the structure of sentences. Same thing happens in speech. Prosody of Speech Characters@@{ Prosody In speech, characters are transmitted as the linguistic information. At the same time, prosody information such as power, accent, pause , are transmitted. The meaning of the sentence is without prosody information. Prosody of Finger Braille Braille code@@{@Prosody Purpose * To detect the prosody in time structure of finger braille @ Data Recording * Purpose: To analyze prosody in time structure of finger braille * Subject: Three finger braille translator * Equipment: Force sensitive sensors Each interpreters typed 10 sentences by finger braille Our Point of View Time Structure of Finger Braille Simulation Experiment * Purpose: To confirm effect of prosody information in time structure * Method: Simulation experiment * Visual output simulating the characteristics of finger braille * Sighted-hearing Subjects The Characteristics of Finger Braille * Volatility * The code appears for a moment * One-dimensional media * Only one-dimensional information is being transmitted Interface Used * Font size: 72pt * Only one kana character is indicated * Starts from 500ms duration Experiment Conditions * Subjects: 12 college students * Output: monotonous and prosody * Sentences: 3 scientific essays for each output * Procedures * Determine the speed of output * Subject reads characters to understand essay * Subject writes the answer for questions for each essay Durations of Two Types of Output Output Duration * Output duration: 60 ms to 250 ms (80 % of the determined output duration) * Example: output duration for Subject A Types of codes Duration Determined sufficient time duration 160ms Monotonous output 128ms Prosody output End of phrases and sentences 204ms Others 102ms Results Percentages of correct answers * Prosody output 79% * Monotonous output 65 % Conclusion * Analyzed the duration of finger braille * Conducted the cognition experiment with a simulated finger braille output * Confirming the effect of prosody in the time structure * Possibility that the prosody information of finger braille can be applied to the AT Future Work * Experiments with deafblind * Analysis of the strength of finger braille * Implementation of the prosody information into the interpreting system