New technology is being created by researchers that will enable hand motions to control computers.
The “Typealike” prototype’s webcam is simply a conventional laptop webcam with an attached mirror. The software detects the user’s hands next to or close to the keyboard and suggests actions based on various hand positions.
For instance, if a user placed their right hand next to the keyboard with their thumb pointing up, the program would interpret this as a command to turn up the volume. Numerous operations can be performed by programming various gestures and gesture combinations.
Human-computer interaction innovations attempt to improve user experience by making it quicker and more fluid, with less reliance on keyboard shortcuts or using a mouse and trackpad.
“It started with a simple idea about new ways to use a webcam,” said Nalin Chhibber, a recent master’s graduate from the University of Waterloo’s Cheriton School of Computer Science. “The webcam is pointed at your face, but the most interaction happening on a computer is around your hands. So we thought, what could we do if the webcam could pick up hand gestures?”
A little mechanical attachment that directs the webcam downward and toward the hands was created as a result of the initial epiphany. The group then developed a piece of software that can recognize various hand motions under various lighting situations and for various users. The Typealike algorithm was trained by the team using machine learning techniques.
People look at something like Typealike, or other new tech in the field of human-computer interaction, and they say it just makes sense. That’s what we want. We want to make technology that’s intuitive and straightforward, but sometimes to do that takes a lot of complex research and sophisticated software.
Daniel Vogel
“It’s a neural network, so you need to show the algorithm examples of what you’re trying to detect,” said Fabrice Matulic, senior researcher at Preferred Networks Inc. and a former postdoctoral researcher at Waterloo. “Some people will make gestures a little bit differently, and hands vary in size, so you have to collect a lot of data from different people with different lighting conditions.”
A database of hand gestures was compiled by the researchers using a large number of research participants. To help the team understand how to make the program as practical and adaptable as possible, they also asked the volunteers to participate in tests and surveys.
“We’re always setting out to make things people can easily use,” said Daniel Vogel, an associate professor of computer science at Waterloo. “People look at something like Typealike, or other new tech in the field of human-computer interaction, and they say it just makes sense. That’s what we want. We want to make technology that’s intuitive and straightforward, but sometimes to do that takes a lot of complex research and sophisticated software.”
The Typealike program has additional uses, according to the researchers, including potential replacement of handheld devices in virtual reality.
