1
Development of a Communication Support Device Controlled by Eye Movements and Voluntary Blink
J. Hori, K. Sakano, M. Miyakawa, Y. Saitoh
Dept. of Biocybernetics, Niigata University

2
Developmentally disabled individuals with motor paralysis such as ALS, or brain-stem infarction have difficulty conveying their intentions because the motor neurons influencing voluntary muscles are affected. 
Various assistive technologies that support individual communication have been developed for disabled people. 
Some have facilitated communication with others by supplementing their impaired functions with surviving functions. 
In cases of terminal ALS patients, the eye movement muscles are not typically affected.

3
In order to detect eye movements, several practical devices have been developed.  
The videooculogram detects eye movements from pictorial images of the eyeball.
Infrared reflectance method of the cornea has been also proposed to detect eye-gazing. 
However, in these methods, the head should be fixed to a specific position.  
And a part of look is interrupted by the devices. 
Moreover, the accuracy is insufficient for practical situations. 
On the other hand, electrooculogram, EOG, can easily detect corneo-retinal potential using attached surface electrodes. 

4
EOG-based communication support devices have been studied be several researchers. 
A simplified switching system using 3 electrodes was proposed.
If this system is applied to a computer interface, an automatic scanning selection must be used for input. 
And, it needs timing and the speed is relatively slow. 
Eye-gazing system was developed for pointing device like a mouse. 
However, this system required the head is fixed. 
These EOG systems have a disadvantage that the detected signals are easily contaminated with drift in long-term measurements. 
As an intermediate system, eye movement detection system in four directions has been developed for step scan system. 
In the present study, I paid attention to the eye movement system.

5
The objective of this study is to develop a real time EOG-based communication support system that offers improved operation and simple composition with high performance. 
The present method realizes the quick and simplified selection of a menu could by cursor movements without an automatic scanning selection. 
We evaluated our proposed system through communication experiments using a screen keyboard. 

6
To accomplish this aim, we proposed a device that outputs five kinds of intentions, such as four directions and one selection.
For this purpose, two-channel EOG signals were measured. 
Eye movements operate cursor movements in four directions and a voluntary eye blink indicates the selection. 

7
First of all, we determined the optimum electrode position with a minimum number of electrodes . 
i) Eye movements of the 4 directions must be detected separately to generate 4 directional inputs. 
Thereby, 2 channels are required to clearly distinguish vertical and horizontal eye movements. 
ii) Voluntary wink should be distinguishable from involuntary eye blinks artifacts and electromyograms. 
iii) Moreover, reproducibility and usability should be considered for user.
Considering these conditions and preliminary human experiments, the electrode positions are: monopole derivations of vertical and horizontal channels.
Two electrodes are attached on the forehead upper part and temple on the dominant eye side, respectively. 
A reference electrode are put on an earlobe on the dominant eye side. 

8
This figure shows a block diagram of our EOG communication system. 
Three electrodes are put on the dominant side of eye according to the optimum positions. 
The detected EOG signals are amplified with AC-coupling and are band-pass filtered to eliminate unnecessary DC drift and high-frequency noise. 
The two EOG signals are combined logically according to the threshold levels. 
Finally, they are converted to five kinds of intention. 

9
In traditional EOG system, detected signals are usually amplified by DC coupling to specify the gazing point. 
However, the offset voltage were observed against the base lines in DC coupling as show in black lines. 
This drift phenomenon is especially seen in long-term measurements.
Fortunately, in our system, the eye movements, that is, the change of eye gazing are used for communication. 
Thus, the detected EOG signals can amplified by AC instead of DC coupling. 
As shown in blue line, the drift was reduced by AC coupling.  

10
This table shows the relationship between eye movements (input) and the detected EOG signals (output). 
V1, V2  and H1, H2 are the upper and lower thresholds of vertical and horizontal channels, respectively. 
These thresholds should be determined in advance. 

For example, when Ch. V exceeds V1 followed by excess over V2, whereas Ch. H remain OFF, output gUph.
For voluntary wink, another threshold V3 is prepared. 

11
Parameters such as the amplitude thresholds and the time interval between eye movements should be used for distinguishing the intentional signals from artifacts such as eye blink and ordinary saccade eye movements.
In our primary experiment, the subject tries to move his/her eye input according to pre-programmed instructions shown on the computer's display. 
The instructions are contained with four movements and wink. Involuntary blinks are also detected. 
By changing the parameters, the optimum parameters are searched by maximizing the accuracy. 

12
These plots show the average amplitude of first and second peaks in EOG signals detected during each eye movement. 
Healthy male subject tried to move his eyes. The view angle was set to 30 degree according to the targets. 
Upper and lower plots show the vertical and horizontal EOG signals, respectively. 
1 and 2 correspond to first and second peaks, respectively. 
The dashed lines show estimated upper and lower threshold levels for each channel. 
Though crosstalks such as U1 and U2 in the amplitude of Ch. H were observed, five kinds of intentional movements were easily distinguishable from each other because of the low fluctuation of amplitude. 

13
These figure shows the vertical EOG signals detected during looking up and involuntary eye blink. 
Voluntary wink was easy to distinguishable from others because the amplitude and duration were clearly larger than others.  
Involuntary blink artifact tended to be misconstrued as gUph output. 
However, the signals of involuntary blink are sharper than that of intentional movement guph.
By considering this characteristics, blink artifact can be eliminated. 

14
This figure shows the EOG signal with voluntary winks. 
The amplitude of voluntary wink is very large and it takes a time for returning to the original levels. 
The delayed  return causes detection error. 
To avoid this error, the Input is prohibited until the signal return to a certain level after the wink. 

15
Experiments were performed with five male healthy subjects, comprising three persons using their naked eyes and two person wearing contact lenses and grasses. 
The subject sat in front of a screen at a distance of 120cm. 
The screen displayed a virtual screen keyboard. 
 As a target for eye movements, markers were put on the screen. 

16
A screen keyboard was used in our experiments to examine the usability of the proposed EOG system. 
A cursor on the letter boxes moved step by step according to subjectfs intention. 
The subject was instructed to move his eyes as extremely as possible, but he felt no uncomfortableness. 
Input was done by his own pace. 
The subject practiced for about 20 min. in advance.
A 12-letter word and 78-letter Alphabets were attempted for each subject. 

17
To evaluate the proposed system, the accuracy and error rate were defined by following equations.  
Moreover, processing speed that is defined by input letters per minutes was measured. 
Fatigue of each subject was also evaluated by a questionnaire subjectively and a flicker test objectively. 

18
These plots show one example of detected EOG signals. 
The upper plot shows vertical EOG and lower plot is horizontal EOG. 
Red, blue, pink, and green colors show the times where each channel exceed upper or lower thresholds. 
We confirmed that the information was inputted correctly according to the subjectfs intentions on the screen keyboard. 

19
These plots shows the experimental results of virtual letter input. 
The average accuracy over 5 subjects were 94.1% for 12 letter input and 87.2% for 78 letters input. 
In every subjects, the accuracy of long input experiments became worse than short input experiments since the number of times of an input increased.

On the other hand, the processing speed was different for each subject, since input operations were done at their own pace. 
Fastest letter input speed of 10.9 letters/min. was obtained for well-trained subject in this case.  
In subjects A, B, and D, input speed of long term experiments becomes quick because of their practices.

20
These plots shows fatigue indices evaluated by questionnaires and flicker test. 
Both fatigue indices increase in 78 letter input experiments for every subjects. 
Especially in subject D, fatigue indices were much larger than others because he wore contact lenses, and frequently made mistakes. 

21
We developed an EOG-based communication support system that outputs five kinds of intention. 
Horizontal and vertical EOGs were measured using two electrodes attached above and beside the dominant eye and amplified with AC-coupling in order to reduce the unnecessary drift. 
Features of eye movements and a voluntary wink were extracted by threshold setting for two EOG signals and their mutual logical operations. 
We achieved an accuracy of 94.1% and an input speed of 8.3 letters/min. 
We believe that the proposed method has potential use in practical situations.

22
The authors thank Mr. Yamagishi (Niigata University) for his experimental help. 

This work was partly supported by Grant-in-Aid for Scientific Research, the Japanese Ministry of Education, Culture, Sports, Science and Technology, Japan and Grant for Promotion of Niigata University Research Projects.