Explore seven marketing use cases of computer vision to see why the technology holds promise and unlocks new capabilities.
Computer vision has grown by leaps and bounds, enabling exciting capabilities in the marketing field. The technology uses AI and machine learning to scan images and accurately identify objects and components within them.
Through computer vision, digital systems are able to recognize and make sense of the contents within images, similar to how humans use their eyes and brains to see and process the world around them. In fact, you have likely already interacted with various components of computer vision on social media. For instance, when Facebook recognizes a friend’s face in your photo and asks, “Do you want to tag [name]?” – that’s computer vision doing its thing. Snapchat and Instagram also use the technology to apply filters, leveraging computer vision to recognize parts of the face like eyes, lips, jawline, and nose, and augmented reality (AR) to modify features in real time, like adding a puppy face or branded image.
More broadly, computer vision algorithms can break down and translate image contents into metadata, which can then be stored, organized, and analyzed just like any other dataset. With these capabilities, computer vision can not only recognize and identify image components, but also detect patterns, respond with triggers like personalized suggestions, overlay virtual images, enable searchable image sets, and so on.
The capability to consume and translate images into a dataset has become vital as online communication evolves. Visual content is poised to overtake text in its importance in social media, publishing platforms, and online marketing campaigns. “The camera is replacing the keyboard,” said Richard Lee, the CEO of a visual technology company. “Images are clearly the overwhelming data piece of social. And now, it is possible for brands to extract data and insights from the plethora of images available online at scale.”
To illustrate just how useful and versatile computer vision is becoming to marketers, here are seven of the most exciting applications of computer vision we expect to see in the near future.
1. Helping Customers Discover Products Based on Visual Traits
When browsing a website but looking for a specific item, customers generally rely on the search or filter function to help find the category of products they’re looking for. Similarly, sites often provide recommended or related items based on a selected product. On the back end, this typically requires an extensive set of “tags” that are manually assigned to products and subjective to the retailer. For instance, one brand’s “culottes” can be another’s nearly identical “gaucho pants,” and details can get even more confusing when proprietary terms are used for specific clothing lines or styles.
Thus, visual search can help customers browse, compare, and narrow their choices through image-generated similarities vs. manually attributed classifications. A customer can use a shoe they like, for example, to find more options in a similar style without the need for specific terminology or from the potential bias of a tagged category. Features like these minimize the need for customers to know brand jargon and simplifies their product hunt.
Data generated from browsing using visual cue modifiers can be just as useful for retailers, who can use retail content management systems (CMS) to trace patterns not visible through language-based tagging alone. For example, two seemingly unrelated products might be purchased together because they have complementary style elements, enabling the system to suggest similar pairings moving forward.
2. Using the Camera over the Keyboard to Search for Products
Computer vision technology enables the use of photo inputs to begin a search query. For instance, if someone sees a great hat or backpack, they can snap a photo of the item and submit it for product information.
Think of it like a reverse Google image search. Rather than entering text information to find images, you can submit a photo for analysis to return information on it. And because computer vision can not only identify the contents of an image, but also the context, it knows that the item is not just a pair of sunglasses, but Ray-Ban Clubmaster Classics.
This can shorten the customer journey by directing the searcher to an item’s product page, minimizing the steps to purchase, or a missed opportunity altogether. For this reason, platforms like Pinterest and eBay have been exploring ways to use photos instead of text within search. Shoppers can also use photos of items they already own to get complementary suggestions, such as submitting a photo of their car to find floor mats that fit.
3. Scraping Data from Social and Video Channels for Discovery
Computer vision can scrape images and videos for metadata to be used for image-based discovery. For instance, Instagram could use computer vision to recognize people and products in photos that are then searchable within their Explore function. Instead of searching for specific hashtags (that would’ve had to have been manually added by the uploader), users could use general terms like “beach bag” or specific product names to return user images or videos with those contents identified in them. This would aid potential customers in product research and help brands identify influencers.
The technology can also track fashion trends by drawing patterns from popular photos and videos. This information can supply insights and feedback to optimize creative and improve targeting through image-based performance analytics.
4. Serving Relevant and Personalized Creative
With computer vision’s ability to ascribe detailed attributes and text descriptors to images, this metadata can then be used in algorithms to guide machine learning selection of creatives within ad or marketing campaigns.
Someone who regularly browses fitness sites, for instance, can be served creative imagery that has been tagged with descriptors that correspond to an active person’s lifestyle. The process can also work in reverse, such as when a brand of soy-based milk substitute places ads on sites showing images of people engaged in fitness activities or making healthy meals. Similarly, contextual ads for a specific cosmetic brand can be served as an in-video overlay on a makeup tutorial featuring their products.
5. People Tracking for Optimization and Non-Digital Ad Attribution
Modern computer vision technology has progressed to tracking human behaviors in real-time through a live video feed, like how autonomous cars can sense pedestrians. Cameras in a retail location, for instance, can draw conclusions about a store layout and shelf arrangement based on customer traffic, how they move throughout a space, and where their gaze falls using facial recognition. For example, the system can accurately track the first area that a majority of customers go to when they enter a store and which products are drawing the most attention.
This technology can also serve as a form of attribution for ad formats such as outdoor signage and potentially TV viewership. Just as how digital ads can track impressions versus clicks, an outdoor sign could track how many people walked past the ad versus how many actually looked at it, how long they looked at it, and even estimate individual demographics like age and gender.
6. Gathering Data for Emotional Analytics and Tracking Consumer Attention
Similar to how social listening technology can gauge sentiment within written content, computer vision systems can track and measure emotional reactions to ad creative. This data is important because self-reported emotions can be inaccurate, especially if the subject’s face is telling a different story.
Annalect’s Moodometer experiment, for instance, revealed that Super Bowl watchers had the most positive reaction to an ad that they had ranked 55 out of 63. The experiment demonstrated that the creative had an impact despite its lower ranking and that consumer surveys do not always provide a complete picture of a campaign’s effectiveness.
7. Using Visual Data for Customer Personalization
Using computer vision, companies can gather real-time visual data on customers to personalize experiences and inform marketing strategy. Select McDonald’s locations have implemented camera-equipped kiosks that suggest menu items based on the customer’s perceived age and gender.
In another example, some analysts theorize that the addition of a camera-equipped smart speaker to the Amazon Echo lineup could give Amazon the ability to gather customer data for more effective cross-sells. By observing what people wear and what they bring into their homes, the company can learn which products to restock or suggest for purchase.
Computer Vision Brings Powerful Data to Images
While computer vision systems have a ways to go in terms of accuracy, reliability, adoption, and potential privacy concerns, the progress they have made over the past few years is remarkable enough to warrant the attention of marketers. Through this technology, marketers have the ability to explore the benefits and versatile use cases listed above, and much more. As the technology matures and society’s reliance on visual communication deepens, we can anticipate many new and exciting creative uses of computer vision to come.