Herramienta para la generación de texto basada en una interfaz cerebro-computador

Abstract

The purpose of the present work is about the development of a tool that allows people to communicate, only through their voluntary blinks. This tool provides a communication link mainly for people with motor disabilities, who cannot communicate through voice or text. This work takes the electroencephalogram (EEG) as the source of information to solve the problem of detecting the voluntary blinks. In this case, the EEG is recorded by the Mindwave Mobile 2 headset (from Neurosky company), which counts on one EEG channel, located in the frontal lobe of the scalp. In order to perform the digital processing of the EEG signal, a recurrent neural network (RNN) was implemented, more specifically a Long-Short Term Memory (LSTM), as these types of networks are effective for time series applications, for instance, EEG signals. The neural network implemented in this work, classifies the EEG signal in one of 5 possible classes, named: No blink, One blink, Two blinks, Three blinks, Other. The results of the trained model were an average accuracy percentage of 92%. Finally, the neural network was embedded in a native Android mobile application, that connects via Bluetooth to the Mindwave Mobile 2, and shows a virtual keyboard consisting of the 27 letters of the spanish alphabet, plus three characters are the commands “delete”, “space”, and “enter”. Each character can be selected by the user only through a determined number of voluntary blinks executed at certain times. When the user types a word and selects the “enter” command, the word is presented audio visually by the application. The mobile application was developed in Java and XML languages in the Android Studio IDE (integrated development environment). In order to verify its performance, an experiment with eight people was executed, that achieved an average spelling precision of 91,26%. On the other hand, the neural network model was designed and implemented in Python language using the TensorFlow and Keras libraries (machine learning libraries), and it was trained in the Google Colab software development environment.