New AI Model May Improve Use of Touchscreens by Patients with Parkinson’s, Other Disabilities

Finnish and Japanese researchers have developed a new algorithmic approach to user interface optimization that takes individual differences into account. This approach could be beneficial for patients with Parkinson’s disease or other disabilities.

The research, “Ability-Based Optimization of Touchscreen Interactions,” was published in the journal IEEE Pervasive Computing.

Among the limitations presented by users with disabilities are essential tremors, characterized by involuntary and rhythmic shaking, most often when using the hands for simple tasks, dyslexia, which impairs the ability to read words of the interface, and dementia.

Strategies to overcome these limitations could involve increasing the size of user interface elements and grouping functions together, reducing the amount of text in the screen and making sure it is correct,  creating designs that require as little previous knowledge as possible, and prioritizing frequent tasks.

“The majority of available user interfaces are targeted at average users. This ‘one size fits all’ thinking does not consider individual differences in abilities — the aging and disabled users have a lot of problems with daily technology use, and often these are very specific to their abilities and the circumstances,” Jussi Jokinen, one of the study’s co-authors, said in a press release.

Approaches to improve the user interface require an accurate model of the user, Jokinen observed. “Previously, designers did not have detailed models that are based on psychological research and can be used to predict how different individuals perform in interactive tasks,” he said.

The scientists developed a new model of interaction, which combines psychological research on finger pointing and eye movements to predict limitations in text entry speed, typing, and proofreading.

By simulating a user with essential tremors, the researchers predicted that using the common Qwerty keyboard is almost impossible, because more than half of all typed keys are typos. “We chose to simulate and optimize for essential tremor, because it makes text entry very difficult,” Jokinen said.

“We connected the text entry model to an optimizer, which iterates through thousands of different user interface designs. No real user could of course try out all these designs. For this reason it is important that we could automatize the evaluation with our computational model,” he added.

This resulted in an interface predicted to be superior for individuals with essential tremors; the simulated user did not make any entry errors. The optimized layout was then tested with a person with essential tremors, who was able to type almost error-free messages.

“This is of course just a prototype interface, and not intended for consumer market,” Jokinen said. “I hope that designers pick up from here and with the help of our model and optimizer create individually targeted, polished interfaces.”

“While more empirical work is needed to evaluate the results, the first evidence acquired in this paper is promising,” the researchers wrote. Future work should test the design for dyslexics, they said.

Beyond the confirmed validity of the model in essential tremor and text entry, scientists also may use it for other disabilities and tasks. “For example, we have models for simulating how being a novice or an expert with an interface impacts users’ performance,” Jokinen said.

The effects of memory impairment in learning and everyday use of interfaces also may be addressed, he added.

“The important point is that no matter the ability or disability, there must be a psychologically valid theory behind modeling it. This makes the predictions of the model believable, and the optimization is targeted correctly,” Jokinen said.