Google recently integrated the technology in their translate app, so now with a single app you can perform a variety of translation tasks. With the World Lens app, additional languages required an in-app purchase. The Google product will include 36 languages for free.
Here are some resources to learn more about the technology.
Real time translator systems have been developed for spoken language, but this is the first portable device that is designed to assist with sign language communication. Here are some of the challenges the developers will face:
Hands Free Operation. In some of the company videos and literature, the device user is shown holding the device in their left hand (for example) and using sign language with their right hand. However, sign language typically requires both hands for many signs.
Wide Area Analysis. The UNI is based on LEAP Motion technology which analyzes hand gestures in a small area above a viewer. Sign language often utilizes a wider space around a person.
Body Analysis. Sign language uses the body. The LEAP Motion technology only tracks hand movement.
Placement. The placement in 3D space of a sign relative to the signer also has meaning. The LEAP Motion technology doesn’t seem to account for this.
Facial Expressions. Often a facial expression can change the meaning of a sign significantly. For example, if I point, and purse my lips (as if whistling) this means something is close. However, in describing something far away, a person will open their mouth and use a different facial expression.
Similar Hand Shapes. Some sings use the same hand shape, but the hand location and orientation change, and this changes the meaning of the sign. The LEAP Motion technology will have a difficult time identifying these nuances.
Hidden Signs. In some signs, the fingers are hidden from the viewer, but because of a familiarity with sign language and the context of the conversation, one knows what’s being signed. For example, when a person is hungry, they will take their open hand as if holding an invisible cup, touch their chest in the middle, and move their hand down. This is the sign for hungry. This is done quickly, and any software designed to analyze finger placement won’t be able to visually see the fingers for such signs.
Despite these limitations, the technology does show promise. As long as the signer limits their language to vocabulary recognizable by the device, it should serve in a simple capacity. Presumably, future versions of this technology, many years from now, will place a camera on the floor or at a sufficient distance to see the upper torso and surrounding area of the signer. Perhaps some kind of x-ray technology could be used for ‘seeing’ hands regardless of any obstruction.
Below are videos about the product, the team, and their development process.
Below is a video about the technology incubator center called the LEAP-AXLR8R where MotionSavvy is based.
“The LEAP Axlr8r is a unique program designed for developers, designers and founders interested in reinventing industries through gesture based technology. We will provide design guidance, access to industry expertise, access to LEAP engineers for software and hardware development and business design expertise to help turn your product into a massively scalable business that changes the way people interact with the world forever.” (source)
LEAP Motion is a device that tracks hand motion for use in many different applications as shown below.
Walter Mossberg of The Wall Street Journal has stated: “Rosetta Stone may be the next best thing to living in a country.” You’ll find other praise for the product used heavily in their marketing materials. The Rosetta Stone System is used by the State Department, NASA, and 9,000 other public and nonprofit agencies. It’s used by over 8,000 corporations and more than 20,000 educational institutions.
Yet Rosetta Stone has also drawn criticism from people, including higher education academics and professors who specialize in computer assisted language learning. This leaves language learners at a loss to know who to believe.
One concern about Rosetta Stone is that a common set of images and base vocabulary are shared across different languages. For example, in the Spanish lesson below, one would expect to see culturally relevant images of people and places where Spanish is predominantly spoken. The man shown in the lower left picture has the Hindi “Om” written on the building behind him (perhaps at a temple). This image would be helpful for those trying to understand the context of the Hindi language and associated culture. However, in a lesson about Spanish it is out of place and disorients the learner.
There’s also been a concern that the program oversimplifies language learning by focusing only on vocabulary and simple dialogs without providing a substantive introduction to the grammar and phonetics of a language.
It may be that those who praise the product and those who criticize it are both wrong:
The supporters of Rosetta Stone make lofty and glorified claims about what it can do – stating that the product is all you need to become fluent in a language.
The critics of Rosetta Stone seem to be arguing that the product is completely worthless.
In reality, most language learners will rely on many tools and resources to learn a language. The more tools and experiences you have, the more deep and rich your understanding will be of a language and its culture(s). A mix of classroom learning, interactive software, and immersion in a foreign country are all helpful for anyone learning a language. Mobile apps, handwritten flash cards, YouTube videos, Skype sessions, and audio recordings are all examples of tools to consider.
The best way to evaluate Rosetta Stone is to try out the free online demo. The real question a language learner needs to ask is whether or not this or any learning tool is worth the cost for what it offers.
Here are a few apps and online resources to use as a comparison: