Google DeepMind AI to Lipread Better Than a Human Professional

Artificial  Intelligence has gone much far than we had ever thought of it to be. Now, Google has come up with an AI technology that has already been tested and been proved much better in lip-reading than any pro. It has already outperformed the best professional human lip-readers available on the planet by a big margin. Also, it can lip-read any TV programme or a commentary running live on a TV channel with a lot of ease and accuracy. It is proposed to be helping the lives of the hearing-impaired people in many ways than a normal person could ever imagine. Machine Learning and Data Mining has been taken into deep consideration to develop this innovation.

This AI System has been developed in collaboration with Deepmind which is owned by Google’s parent company Alphabet and University of Oxford. Initially, tested upon news clips primarily from BBC aired in between 2010 and 2015 with runtime of more than 5000 hours and over 118,000 sentences as reported by New Scientist, now is also successfully deciphering TV programmes. The Artificial Intelligence System has also been verified by making it to lip read programmes which were aired in between March and September of 2016. The new AI system got trained by lip-reading TV programmes like Newsnight, BBC Breakfast and Question Time.

In the testing phase, the system deciphered this phrase 100% accurately - “We know that there will be hundreds of journalists here as well” by mere watching the lip movement of the reader and was even tested on other difficult phrases to understand. A professional lip-reader annotated up to 12.4% correctly out of the randomly selected 200 clips, in which the Google’s AI system registered correctness of 46.8%. It has been considered as a big step towards developing fully automatic lip-reading systems.

The Technology Involved

The video clips of BBC over which the said software was tested has to be first prepared using machine learning. The problem faced by the technologists was that the audio and video streams were not getting synced by about a second. This made AI unaccustomed to the sync of the words said and the way the speaker moved his lips. Therefore, a computer system was made to learn the tuning between sounds and mouth shapes making the audio and video aligned together with perfect accuracy. Now the AI technology was supposed to automatically process all 5000 hours of audio and video together with exact accuracy and surprisingly, it did the same with no lag detected.

Google translate, the company’s translation tool which was earlier working on Neural Machine Translation to depict whole phrases in place of single words will now be using Multilingual Neural Machine Translation, the company stated. The organization will be using the new translation system but will not be completely overlooking the old one as it will be taken as the base for the new one. Machine Learning and Data Mining is the base for this technological invention.

The words of the search giant in a blog post says -

“Our proposed architecture requires no change in the base GNMT system, but instead uses an additional "token" at the beginning of the input sentence to specify the required target language to translate to. In addition to improving translation quality, our method also enables "Zero-Shot Translation" - translation between language pairs never seen explicitly by the system”.

Google Translate is now going to use one system for all kind of translations than using individual systems for each one of the language pair translation. Translation between different language pairs will now be possible with the new translation mechanism in place which has never been achieved earlier. Though it's too early to comment on the proposed success of the new AI technology, but it's confirmed that it has impressed the world by just the bold claims it has already made regarding its Speech Recognition System. Machine Learning has transformed itself into a new development with this technological marvel.

Proposed Practical Applications in Near Future

• Improved Hearing Aids.

• Silent Dictation in Public Spaces.(Siri does not need to hear your voice from now onwards).

• Speech Recognition in Noisy Environments.

Also Read : An Industrial Revolution Sparked By Machine Learning

DeepMind is also in a position to provide an AI based app(for phones) namingly “Streams” which will help the doctors in finding the patients at risk. It looks like the inclusion of  Artificial Intelligence in the daily life of people will be a new normal in the upcoming age. May be 10 years from now or might be just five. Watch out for that.