When a special project now housed at Rhodes is completed, hearing and deaf people will be able to converse using a special translation app on their cellphones.
When a special project now housed at Rhodes is completed, hearing and deaf people will be able to converse using a special translation app on their cellphones.
In the age where mobiles can do everything, if not more, than computers can do, it is little wonder that phone applications – more commonly known as simply apps – have become an intrinsic part of people’s everyday lives.
Apps can help your phone help you do everything: get directions, cook a meal, track your fitness regime, learn a new language, organise your shopping list and wile away hours of boredom.
Rhodes University Computer Sciences lecturer, James Connan, has been part of the South African Sign Language (SASL)-to-English translation app since its conception in 2001.
The idea behind the app is to create a seamless translation from English to SASL and back again using only the microphone, video camera, screen and wireless connectivity of a mobile phone.
Connan is working toward a time when pointing a mobile at a hearing person while they speak will almost instantly create a 3D avatar signing what was said for the deaf person to see.
The app will also work the other way around: a deaf person can sign a response into the mobile’s camera, and the app will translate the signs into English for the hearing person to understand.
To date, the SASL project has combined the computer skills of Connan and over 40 students nationwide, to create a preliminary app called iSign.
iSign is able to recognise hand shapes, gestures and the basic facial expressions that make up the SASL language and is able to render around 50 common phrases in SASL and English.
This means iSign can be used as a phrasebook to carry out a rudimentary conversation between a hearing person and a deaf person.
The aim now is to transform iSign into a working translator of any and every sentence that could be conjured up in either language.
For the linguistic side of the project to be as flawless as possible, Connan has collaborated with the Rhodes Linguistics Department, which runs an annual course on SASL for second-year students.
Mikhaela Kohlo, a Linguistics Masters student at Rhodes, is one of the students looking to make that translation a reality. This year, Kohlo’s task is to study current theories about how Sign Languages can be coded in writing to see if they can assist in the project.
This will determine how to assign entire gestures to corresponding English phones, which is the term linguists use to define the representations of all the possible sounds in English.
“I am intensely excited in participating towards adding to technology [for the deaf]in South Africa,” Kohlo said.
There are challenges to overcome, however; SASL has a number of different dialects, not all of which can be accounted for in such a system, especially since SASL has a number of grammatical eccentricities that make finding a notation system that much harder.
And even when the translation system is finally coded there is always room for error, as anybody who has ever used Google Translate will know. “You’ll never get it perfect. But we’re aiming to get it as close as possible,” said Connan.
“I always relate projects to the development of glasses: you had the monocle, then spectacles and now there’s laser eye surgery. At the moment we’re still in monocle phase,” said Connan, who is firmly optimistic about the future and necessity of the project.
“We (can) use computers to solve other people’s problems. We learn from different areas so that we can apply other skills to our skills to make things that are necessary for those who need them, to function in life better.”
