Eight interns from the Microsoft Company have developed a new tool for language learning that utilizes the camera of a smartphone in order to help people who are looking to improve their English literacy by learning the words for the things that surrounds them. The app is called Read My World, and it will let you take a picture with your smartphone so that you can learn from a library that hosts more than 1,500 words. The photo taken by your smartphone can be a text found inside a document or just of a real-world object.
Read My World is aimed to act as a supplementary tool for users who are looking to learn words outside the environment of formal classroom training. Instead of receiving English lessons, users of this app are encouraged to take photos of their surrounding environment and increase their English vocabulary through that exchange.
Read My World uses a combination of Computer Vision APIs and Microsoft Cognitive Services in order to identify the real-world objects or texts in photos. It will then be able to display the word’s spelling and speak the phonetic pronunciation of the identified vocabulary words in the pictures. Users can also save their photos that correspond to the identified words inside a personal library.
Read My World also encourages its users to practice their newly discovered words by way of three built-in vocabulary games.
The apps library of 1,500 words may appear to be tiny, but it is actually close to the number of words foreign language learners are able to pick up through traditional study. BBC states in a report that many language learners routinely struggle to learn more than 2,000 words or even 3,000 words after spending many years of studying the English language. Another study in Taiwan has found that after spending 9 years of studying a foreign language, students have failed to learn the most frequently-used 1,000 words, corroborating the report from BBC.
Because the app primarily focuses on the things that you witness on a daily basis, it cannot be used as a tool that replaces formal learning methods. After gathering feedback from teachers and students who had an opportunity to test an early version of this app, the team has rolled out a new feature that can detect words in documents as well. It doesn’t offer the same experience as Google Lens, where written words are translated into your own language, instead, the app can identify select words which are highlighted so that you can hear how they sound, and also see a picture of it so that you know what that word means. For example, if you point the app at a school supply list of a student, the app may pick out words like scissors, binders, pencils, and notebooks.
Read My World is a project from Microsoft Garage, which is an in-house incubator of Microsoft. The app will initially be made available for testing and feedback for select organizations. Organizations that work with low literacy communities can request an invitation from Microsoft to join the experiment by filling out a form.