[App Fridays] A Made in India app recognised by MIT that helps the deaf ‘hear’ and ‘speak’

Living with a disability is not easy, but most people who have some form of disability generally develop other heightened senses and skills, and are able to live an almost normal life and contribute to society.

People who have lost their sight have a heightened sense of hearing and touch, while the deaf and mute are able to read lips and communicate using sign language. But most of the general population do not know sign language and are unable to communicate effectively with the deaf.

So the deaf find it difficult to converse with people on a day to day basis. A doctor from Kerala aims to solve this issue through a smartphone application.

What is the app about?

‘Let Me Hear Again’ helps differently abled users, who are hard of hearing or deaf, communicate with others who do not know the American Sign Language. The app was ideated by Dr. Sharon Baisil and developed through the Google App Inventor platform. The app has been recognised by the Massachusetts Institute of Technology (MIT) and was also awarded the title of ‘App of the Month’ by MIT App Inventor.

About the Founder

Dr Sharon Baisil is a 29-year-old medical doctor, who is an alumnus of Government Medical College in Thrissur, Kerala. He is currently working at SPM Clinics in Cochin.

He founded Let Me Hear Again Project for deaf and late deafened adults with the aim to provide rehabilitation for the hearing impaired at the lowest cost by providing innovative technologies. Dr Baisil developed the app with the aims to provide deaf individuals with an arsenal of tools, which could help them save money from buying expensive videophones or vibrating alarms or hiring interpreters.He learnt the basics of programming and android development from scratch through online tutorial videos,ranging from those uploaded by amateurs to those uploaded by MIT professors. At present, over 500 doctors are supporting this cause by recommending the app to their patients.

The future objectives of the project are to create an effective group chat for the deaf, which could enable them to participate in real time group discussions with ‘hearing’ individuals. Another objective is to add sign language support and real-time sign language recognition. Dr. Baisil informed YourStory that he is also working on an improved Graphical User Interface with flat icons, which will be available by the second week of December as part of a major update.

Features

The app consists of five main tools, which have been integrated into one app

Face to face chat

Though most deaf individuals are adept at reading lips, there is a margin for error, so the app recognizes conversations in 41 languages and displays the words on screen and can provide live translation as well.

It has a Talk-Back feature, where the person can type in his or her reply and the phone will speak it out aloud. It includes a full screen speech to text chat with re-sizable fonts and auto speech recognition mode with custom timer.

Virtual notes

This feature can be used to capture speeches and long conversations and save it as words in the phone memory after speech recognition.

The intention is to enable the deaf to save class lectures, to do lists, key points from group discussions and meetings. These saved notes can be edited later if needed, and can even be translated to any language of choice. Notes can also be shared through social media.

Call companion

This tool enables the deaf to make and receive phone calls. It can transcribe phone calls to words in real time and can display it on the screen.

Quake awake alarm

As normal alarms are not effective for the hearing impaired, this custom alarm produces sounds above 100dB (most deaf persons have hearing loss to sounds below 90dB) and vibrates the phone for two minutes. The combination of loud sounds and vibration will make it easier for the deaf to wake up in the mornings.

Guardian angel

This feature was inspired by a real incident in the US where a pitbull saved a deaf boy from a house fire. In case of emergencies it is difficult for a deaf person to be aware of what is going on in his or her surroundings.

The last tool is a personal watchdog for the deaf person and in the app it’s a cat called ‘Angel’. This cat can be turned on by setting the timer and the person can continue with their normal daily routine. It will automatically search for loud sounds in the surroundings at set intervals and if and when it detects something, it will alert the user with high intensity sounds and vibrations until the person picks it up.

Price

There are two versions: Free and Pro: INR 50.

While the free version has access to most features, some of the features like quake awake, guardian angel, the night mode theme, edit and share tool in the virtual notes section are only available in the Pro version. The free version supports only four languages.

What works?

The overall idea and concept works well and the app solves some of the pain points a deaf person faces on a day to day basis. The face to face chat feature worked well and the Auto mode where the app checks for sounds every six or user defined number of seconds is a useful feature. As stated, the app also blocks many offensive words and does not translate them.

Virtual Notes also works well and I was able to translate short notes into four languages, including English, French, Italian and Spanish.Quake alarm works as stated, provided the user follows the instructions and does not close the app.

What does not work?

The UI and UX is average with room for improvement. For example, even after using the app for a while, I could not find a way to disable the popup ‘Help Notification’ that appears everytime one clicks on different features. The Virtual Notes feature worked well for short notes, but I was unable to find the custom timer in this section to try it for longer conversations.

As stated in the app, the call companion feature has a sharp learning curve and is not easy to implement. Simplifying this further could enhance the UI. Users currently have to connect the phone to an external loud speaker to amplify phone calls. Otherwise, the phone's speaker will not be able to detect the voice in a phone call and thus voice recognition does not work.

YS verdict

With a population of about 250-300 million deaf people in this world, there is a big potential market for such an initiative. This Made in India app is aiming to solve a relevant problem and has succeeded to a certain extent.

Download it here: Let Me Hear Again