An international study team, which includes three QUT researchers, has found innovative ways for retailers to employ artificial intelligence (AI) in conjunction with in-store cameras to better understand consumer behavior and design shop layouts that will increase sales.
In a study that was published in Artificial Intelligence Review, the team suggests a framework for AI-powered store layout design that retailers can use to take full advantage of the most recent developments in AI techniques, as well as its subfields of computer vision and deep learning, to monitor their customers’ actual shopping behaviors.
An effective store layout presents its merchandise to both attract customer attention to items they had not intended to buy, increase browsing time, and easily find related or viable alternative products grouped together. Anyone who has ever had to retrieve milk from the furthest corner of a store knows well how this works.
It has been demonstrated that a thoughtful layout favorably correlates with rising sales and customer happiness. One of the best in-store marketing strategies, it can directly affect consumer choices to increase sales.
QUT researchers Dr. Kien Nguyen and Professor Clinton Fookes from the School of Electrical Engineering & Robotics and Professor Brett Martin, QUT Business Schoolteamed up with researchers Dr. Minh Le, from the University of Economics, Ho Chi Minh City, Vietnam, and Professor Ibrahim Cil from Sakarya University, Serdivan, Turkey, to conduct a comprehensive review on existing approaches to in-store layout design.
Dr. Nguyen says improving supermarket layout design through understanding and prediction is a vital tactic to improve customer satisfaction and increase sales.
Stores like Woolworths and Coles already routinely use AI empowered algorithms to better serve customer interests and wants, and to provide personalised recommendations. This is particularly true at the point-of-sale system and through loyalty programs. This is simply another example of using AI to provide better data-driven store layouts and design, and to better understand customer behaviour in physical spaces.
Professor Clinton Fookes
“Most importantly this paper proposes a comprehensive and novel framework to apply new AI techniques on top of the existing CCTV camera data to interpret and better understand customers and their behavior in store,” Dr. Nguyen said.
“CCTV offers insights into how shoppers travel through the store; the route they take, and sections where they spend more time. This research proposes drilling down further, noting that people express emotion through observable facial expressions such as raising an eyebrow, eyes opening, or smiling.”
Understanding customer emotions as they browse could provide marketers and managers with a valuable tool to understand customer reactions to the products they sell.
“Emotion recognition algorithms work by employing computer vision techniques to locate the face, and identify key landmarks on the face, such as corners of the eyebrows, tip of the nose, and corners of the mouth,” Dr. Nguyen said.
“Understanding customer behaviors is the ultimate goal for business intelligence. Obvious actions like picking up products, putting products into the trolley, and returning products back to the shelf have attracted great interest for the smart retailers.”
“Other behaviors like staring at a product and reading the box of a product are a gold mine for marketing to understand the interest of customers in a product,” Dr. Nguyen said.
Layout managers can use tools like heatmap analytics, human trajectory tracking, and customer action recognition to help them make decisions in addition to using face signals to identify moods and customer characterization.
Without having to know about specific names, this kind of knowledge may be evaluated immediately from the video and can be useful for understanding customer behavior at the store level.
Professor Clinton Fookes said the team had proposed the Sense-Think-Act-Learn (STAL) framework for retailers.
“Firstly, ‘Sense’ is to collect raw data, say from video footage from a store’s CCTV cameras for processing and analysis. Store managers routinely do this with their own eyes; however, new approaches allow us to automate this aspect of sensing, and to perform this across the entire store,” Professor Fookes said.
“Secondly, ‘Think’ is to process the data collected through advanced AI, data analytics, and deep machine learning techniques, like how humans use their brains to process the incoming data.”
“Thirdly, ‘Act’ is to use the knowledge and insights from the second phase to improve and optimize the supermarket layout. The process operates as a continuous learning cycle.”
“An advantage of this framework is that it allows retailers to evaluate store design predictions such as the traffic flow and behavior when customers enter a store, or the popularity of store displays placed in different areas of the store,” Professor Fookes said.
“Stores like Woolworths and Coles already routinely use AI empowered algorithms to better serve customer interests and wants, and to provide personalised recommendations. This is particularly true at the point-of-sale system and through loyalty programs. This is simply another example of using AI to provide better data-driven store layouts and design, and to better understand customer behaviour in physical spaces.”
Dr. Nguyen said data could be filtered and cleaned to improve quality and privacy and transformed into a structural form. As privacy was a key concern for customers, data could be de-identified or made anonymous, for example, by examining customers at an aggregate level.
“Since there is an intense data flow from the CCTV cameras, a cloud-based system can be considered as a suitable approach for supermarket layout analysis in processing and storing video data,” he said. “The intelligent video analytic layer in the THINK phase plays the key role in interpreting the content of images and videos.”
Dr. Nguyen said, “layout managers could consider store design variables (for example space design, point-of-purchase displays, product placement, placement of cashiers), employees (for example: number, placement) and customers (for example: crowding, visit duration, impulse purchases, use of furniture, waiting queue formation, receptivity to product displays).”
