AI is being used in retail to power a new wave of customer-centric decisions

(Image credit: Image Credit: Geralt / Pixabay)

Retailers are increasingly looking to harness the rewards available from social media. Social platforms are already widely used by retailers in an effort to connect with customers in an engaging and authentic manner.

Sentiment analysis is used widely, but studies have shown that its accuracy can be as low as 58 per cent. Furthermore, it typically misses detailed signals such as specific nuances within customer concerns, plus emotional intent. Without intent it is hard to take action.

These traditional methods for social listening pick up on specific hashtags and keywords and whether the messages are positive, negative or neutral. They struggle with analysing sentiment due to the vagaries of language and are unable to determine customer intent. For example, they are able to determine basic feedback, ‘this is good’ or ‘this is bad’ but the lessons that can be learned can lack a depth of understanding and important nuances can be missed. Messages therefore need to be manually filtered to learn what customers really think about a new product, a store, or an offer.

This way of listening to consumers is inevitably superficial at scale and the rise in the sheer amount of data will only exacerbate the issue. According to Brandwatch, as of May 2019 there were 3.499 billion active social media users sending 500 million tweets every single day on Twitter. With so much data it has become harder to realise a fraction of the value that could be derived and it is even harder to take action when insight is limited to sentiment. For example, if customer intent is negative but does not really explain why, it is hard to act.

Understanding concepts

Things are changing though. Using sophisticated computer algorithms developed at the University of Warwick over a decade of academic research, machine learning specialist Warwick Analytics is able to apply highly accurate models to textual data including social media, but also other customer feedback from complaints, calls, CRM notes and chat. The model is continuously optimised by bringing in a ‘human in the loop’, which can be a non-data scientist, to validate anything it might not be sure about and then trains and updates itself with the input – but the machine learning model ensures this remains efficient at scale.

With minimal manual intervention, root causes of churn and loyalty, as well as emotional causes of purchasing decisions can be identified. This in turn enables retailers to really understand customer feedback and where necessary make changes to recipes, packaging, store layouts or even identify new opportunities to introduce new products.

For example, if customers tweeted feedback about how a product’s recipe did not correlate to the messaging on the packaging, traditional social media insights would flag this as a product fault and, if enough negative feedback was recorded, the product could be liable to be discontinued. However, by harnessing detailed insight beyond sentiment and into customer intent, retailers are able to see that the error was related to the packaging – a relatively straightforward artwork ‘fix’ compared to undoing the whole product development cycle.

Human behaviours, like sarcasm, can also lead to the intent of a comment being misconstrued with most tools. Based on human analysis of a small section of data, this new modelling technology understands concepts rather than keywords and therefore identifies intent to a far higher accuracy than has been possible until now.

Interpretation and understanding

For example, consider the Tweet “why have u added milk to a perfectly nice soup??”. Many tools would misinterpret this as positive sentiment (potentially wrongly about milk) rather than correctly attribute it as negative sentiment about soup.

Consider these following recent customer-service related tweets. If you applied a generic model, you might get a sentiment score for each one, and pick out some topics or keywords. However, if you were to stop and dream up what the perfect, granular, actionable signals were from each one, you might come up with a much richer understanding as exemplified.

Tweet: @tesco @LidlGB hey both! We shop in your supermarkets, do you have lists online of products you sell which don’t contain palm oil?

Potential AI interpretation and understanding: Neutral sentiment but “environmental concerns – avoiding palm oil”, and “easy information on products containing palm oil”. In other words, there’s a CSR opportunity here to capitalise on any trends in this space

Tweet: @asda When are you going to start stocking “Marmite peanut butter”?

Potential AI interpretation and understanding: “New product suggestion”. If more people are asking for this, or the new product teams are alerted, then this can provide valuable insight into trends and opportunities.

Tweet:@Morrisons So annoyed with the countless empty shelves in the Burton store today. It was impossible to do a full shop so I just abandoned my trolley, walked out and spent my £100 at Tesco who don’t seem to have the same ridiculous stock issues!!

Potential AI interpretation and understanding: “Many items out of stock”, “shopped elsewhere”. This is obviously a concern that a customer is churning from this particular shop. They have not indicated they are abandoning the brand (indeed there is a hint that they are loyal customers providing feedback) but clearly there is an issue at that branch.

Rather than just determining the sentiment of the comment (positive versus negative), new generation machine learning tools can determine much more information and deliver real intelligence for retailers to act on. Information which no longer should just be the domain of the ‘marketing’ department, but which can add value an insight for brand managers, product managers and developers.

Social media provides a window into a consumer’s world. Both positive and negative comments are inevitable, and until now even the most sophisticated tools would simply focus on tallying these up.

It’s an exciting time in retail with AI making it possible to learn not just sentiment, but intent. Intent avoids false triggers but ensures more actionable insight for brands and retailers. This is giving brands a new edge as they identify issues and opportunities earlier than ever and engage with customers in an even more powerful and meaningful way. Brands are already fighting it out to grab attention on social media, but by harnessing more intelligent AI driven insights, perhaps they’ll deliver what we want before they vast majority of us even realise we want it.

James Butcher, CEO, Solutions for Retail Brands