Optical sensing tasks have become more difficult over the past ten years. To enable mobile applications in smart phones, autonomous vehicles, robots, and drones, it has become crucial to develop inexpensive, miniaturized sensors that can be integrated on-chip. Additionally, machine-learning algorithms have been utilized in numerous recent developments, and their importance in sensing is growing.
A brand-new idea known as geometric optical deep sensing is presented by researchers in the Electrical Engineering department of Prof. Fengnian Xia in a new Science paper. The idea, which makes use of advances in device technology, condensed matter physics, and deep learning, has the potential to switch from approaches that focus on hardware to ones that focus on software.
With help from colleagues at the University of Texas, Bar-Ilan University in Israel, and Vienna University of Technology in Austria, the paper was written. “Geometric” denotes that the sensor outputs are multi-element data that can be viewed as points in high-dimensional vector spaces in this new concept. “Deep” emphasizes how important deep neural networks are to this sensing scheme.
“Innovations in device architectures, demonstrations of developing optical and optoelectronic phenomena, and development in machine-learning algorithms will assist this subject.”
Shaofan Yuan, a former Ph.D. student in Xia’s lab.
Shaofan Yuan, a co-lead author of the paper and a former Ph.D. student in Xia’s lab, said that conventional optical sensing needs multiple optical devices to fully capture the light beam’s unknown properties. These include a variety of instruments for determining the light’s intensity, polarization, wavelength, and spatial distribution. The total number of these devices adds up to a large and costly system.
According to Yuan, “much effort has been made in the past to make optical sensing devices compact and multifunctional,” and “advanced machine learning algorithms have accelerated optical sensing solutions using miniaturized devices.” Yuan also stated that the field of optical sensing technologies in the foreseeable future will be extremely interdisciplinary. Advances in machine-learning algorithms, demonstrations of new optical and optoelectronic phenomena, and innovations in device structures will all be beneficial to this field.”
Chao Ma, another co-lead author of the paper and a Ph.D. student in Xia’s lab, stated that device reconfigurability is essential for achieving complex optical sensing with a single device.
“A solitary reconfigurable gadget that can be worked at various states is fundamental to creating multi-component photoresponse information that catches numerous obscure properties of light once in a while in a verifiable way, and afterward AI calculations can be utilized to decipher the information,” Mama said.
For the encoding and decoding of information, the plan makes use of deep neural networks and reconfigurable sensors. That is, the networks have been trained with known light properties and are able to extract the appropriate data from the reconfigurable sensors’ multi-element outputs. According to Xia, it interprets the multi-element photoresponses in a manner similar to that of image recognition software.
He stated, “If you want it to recognize an image, whether it’s a dog, a cat, a person, or a car, you train it by collecting a lot of pictures with known information.” The neural network will then tell you by simply providing an unidentified number. The comparable thought is utilized here.”
The scheme’s fundamental idea can be applied to a variety of fields, including magnetic field sensing, according to the researchers. According to Xia, he and his coworkers are currently investigating potential applications. Utilizing such integrated sensing devices to improve the safety of autonomous vehicles is one possibility.
More information: Shaofan Yuan et al, Geometric deep optical sensing, Science (2023). DOI: 10.1126/science.ade1220
