Sheldon Cooper, the theoretical physicist on the TV sitcom The Big Bang Theory , could never have imagined meeting his idol Stephen Hawking if the latter, arguably the greatest physicist alive, did not have access to the assistive technology that enables him to communicate, despite his almost total paralysis. Nor would the world have been able to experience the genius of Prof. Hawking firsthand.
Intel has been designing and sponsoring the computers that run Prof. Hawking’s assistive gadget since 1997.
Owing to motor neuron disease, which he was diagnosed with when he was 21, Prof. Hawking has steadily been paralysed from face below and has very little voluntary control, even on his face.
Today, at the age of 71, Prof. Hawking is entirely dependent on his assistive gadget for communication.
In the 1990s, Prof. Hawking was able to use his thumbs to control a joystick-like module to command the voice synthesiser and move his electronic wheelchair, but even that has eroded in the last four to five years.
Currently, he twitches his cheek to select characters on the onscreen keyboard to synthesise sentences. But even this technique has become very ineffective, and currently he can convey his thoughts at the rate of just about one word per minute.
Assistive wheelchair rig
With his limited muscle control, restricted to his face alone, and unable to speak since his tracheotomy in 1985, Prof. Hawking uses computer-based assistive technology to “speak”. He frames words and sentences by moving cursors on the onscreen keyboard running a special program, EZ keys, on a Lenovo Thinkpad powered by the Intel i7 series processor. Once a sentence has been framed, Prof. Hawking sends another gesture-based command that triggers the voice synthesiser that reads out the sentence. Of course, Prof. Hawking is not altogether impressed with the voice synthesiser, especially its accent.
“This synthesiser is by far the best I have heard, because it varies the intonation and doesn’t speak like a Dalek (the extraterrestrial villains of the Doctor Who universe). “The only trouble is that it gives me an American accent,” the British physicist blogged.
As Prof. Hawking retains no control over his limbs, he twitches his cheeks to move and select the cursor. The infrared switch mounted on his spectacles capture these movements for the computer to process them. When he twitches his cheek, the infrared switch senses the action and inputs the signal to the computer which accepts it as a simple ‘enter’ action. The EZ keys program also allows Prof. Hawking to move the cursor on his Windows 7 operating system, using which he sends email, browses using Firefox and jots notes on the Notepad application. But this is a tedious process, having to put together sentences word by word, and words, character by character.
Prof. Hawking’s ability to communicate is thus way behind the rate at which the great physicist thinks.
This is a pity, especially when there are technologies that can at least bridge the gap between what he is thinking and what he can tell us.
Word prediction
Prof. Hawking’s burden is greatly reduced by the use of word prediction algorithms, similar to the technology in mobile phones that enable us to compose text messages much faster. Of course, the version of software that helps Prof. Hawking is more complex and is bundled with a contextual analyser that tries to predict words based on the context, depending, for instance, on whether Prof. Hawking is at a lecture, or is engaged in a conversation with his secretary. This eases the burden on Prof. Hawking, who, otherwise, might have to gesture out entire spellings, painstakingly.
The weakening of Prof. Hawking’s control of his cheek muscles has impaired his ability to generate speech.
Developing a better gadget
Having struggled with the problem, he wrote last year to Intel co-founder Gordon Moore, seeking an enhancement. Moore, who is famous for his known prediction that the number of transistors on a chip will continue to double every 18 months, which has been dubbed as Moore’s Law, asked his team at Intel to develop a better gadget for Hawking.
At the Consumer Electronic Shows 2013 held in January, Chief Technology Officer, Intel, Justin Rattner presented the possibility of improving the speed of Prof. Hawking’s assistive technology by the end of the year.
With these upgrades, Intel hopes to capture more gestures from the voluntary portion of Prof. Hawking’s face, such as those from his eyebrows and mouth.
This would increase his ability to communicate; it might even enhance the speed at which he communicates to at least five to six words a minute, a substantial improvement over his current rate of one word per minute.
Apart from improvements in mechanical gesture recognition, the new upgrade will hopefully have better software for word prediction and context analysis. Also, facial recognition programs are being embedded to enable better prediction of Prof. Hawking’s intentions.
Instead of using only the infrared switch, a camera will capture and process gestures on Prof. Hawking’s face. These improvements, which might appear trivial to ordinary people, would be a giant leap for mankind in its quest to learn more from this extraordinary mind.
