How Reckitt Leverages AI and Machine Learning in Display Campaign Optimization

This webinar is presented by CGT and presented in partnership with Sigmoid. My name is Maia Jenkins, I'm an editor at CGT, and looking forward to a great conversation today.

According to recent data, $6 out of every $10 in media spending in the United States is allocated to digital. This means, of course, that brands are competing for attention more than ever before, but also that a lot of this attention is directed to the digital screen share, making it all the harder to stand out from the crowd.

Of course, there are immense challenges in this area, including how to maximize return on spend, but also how to gather the right data, make sure you're asking the right questions, and targeting the right audience, as well as how to ensure these efforts are being rewarded and you're getting a return on investment.

There is also a huge opportunity to tap into a massive audience with the right investment, which means investing in the right technologies. One brand that is driving growth and making progress in this area, is Reckitt, who is investing in machine learning and AI capabilities, and fueling e-commerce success using these tools.

We'll look into that in more detail today, but more broadly, we'll look at how leading brands are using these technologies to influence audience selection, decide on bid frequency, campaign budget, and context setting. We won’t simply focus on the campaigns themselves, but how to approach these campaigns and where to start.

We'll also discuss how companies like Reckitt and others are developing frameworks or a greater understanding of consumer data using these CDPs, and leveraging the insights therein, to drive value, growth, and ultimately, lead to greater sales. 

Hopefully you'll leave with a better understanding of the state of advertising on retail e-commerce platforms, as well as practical tips for engineering and more strategic advantage in this growing area. There's lots of opportunity, it's an exciting area, and we are looking forward to a great conversation today. 

With us today to explore these ideas are Imteaz Ahamed and Sundeep Kumar. Can you each introduce yourself, and share a bit about your role?

Imteaz Ahamed: Thank you for having me, I'm Imteaz, and I've been with Reckitt for 14 and a half years now. I've worked across the Australian, UK, Dutch, and now U.S. markets. The first six years of my career were on the sales side of the organization, and then I fell into e-commerce because I was qualified in PowerPoint and Excel.

Over the last eight years, I've done everything from setting up direct-to-consumer websites and setting up supply chains for direct-to-consumer websites, to learning media, CRM, data science, etc. I did a course at Harvard to learn data science as well, because I was hiring data scientists who work under me to crunch crazy numbers of data.

It's been an interesting journey because I didn't think I'd be where I am today when I started, and I don't know where I'm going to be in 10 years time either. This journey using technology to unlock growth is something I'm passionate about. 

Sundeep Kumar: Hi, my name is Sundeep. I’m a senior strategy consultant at Sigmoid with close to a decade of experience in managing data science projects, data engineering projects, as well as providing technology and strategy consulting. With Reckitt, we’ve had a strategic partnership for close to five years across a variety of business units in supply chain, finance, and e-commerce marketing.

This has been an exciting initiative that I’m looking forward  to talking about today.

Jenkins: We’re going to get started with a poll question: What do you currently leverage to optimize ad campaigns on e-commerce platforms? Is it recommendations provided by the ad platform, machine learning-driven recommendations or historical insights from campaign managers? 

The most popular choice is historical insights from campaign managers with 67%, and followed by recommendations, then machine learning-driven recommendations. Imteaz, do these results surprise you?

Ahamed: This is what I'd expected in terms of using an agency or performance media manager to look at historical insights of what has happened previously, and use that as proxies for predicting the future – it’s probably the most common way of doing it, currently.

The ad recommendations from the platform specifically, are always in the platform's interest to optimize performance on the platform. It’s not surprising to see these answers, but this is where the industry is at today. I'm not surprised to see machine learning being the lowest out of the three right now, but hopefully we can change people's minds towards the end of this presentation today.

Jenkins: Sundeep, do you have any thoughts on this as well?

Kumar: Condition-based decision-making is what everyone goes through today. After our discussion today, I'm not sure how many would change to data-based decision-making, but we’ll try our best.

Jenkins: With that in mind, we've just seen the more popular choices of how people are leveraging advertising on e-commerce platforms. Imteaz, can you give us an overview of the current state of advertising on retail platforms? What are some of the current challenges of running media with retail media partners?

Ahamed: First, there's a plethora of retail media platforms launching at the moment. Retail media is having a bit of a moment, in terms of volume of growth from the sheer number of networks that are launching. As an advertiser, that is a struggle because you have to understand which ones are a priority for the brand or category, versus which ones don't fit. 

Second, not all retail media platforms are created equal. By that, I mean the KPIs are different from platform to platform. Attribution windows and how you attribute sales – from both an online and offline point of view – is different by platform. Again, another challenge there. There are requirements for you to do custom integrations with platforms to see how data will flow through from your side of the equation into the retail media equation, as well.

There's a lot of inconsistency in terms of the go-to-market, which requires a specialized skill set to do all the things that you would want to do, at the scale that you want to do it. 

Finally, costs on these platforms are rising. A lot of brands are heavily investing in this area, and sometimes they're investing ahead of what they could or should be, which is driving up cost. From an advertiser point of view, I want to drive efficiency and drive performance. Sometimes, when people get too excited about something and throw too much money at the opportunity, it throws the category out of whack. We need to be cautious in terms of how we invest, and ensure that the results and KPIs we want to drive for the overall business and category, are actually being delivered.

You need to have a category-specific and business-specific approach to this because you can spend a lot of money very quickly, and that's not necessarily going to drive performance.

Jenkins: It's interesting what you're saying about the nature of it potentially being fragmented. How do we consolidate that investment in technology – that people are getting over-excited and maybe overextending themselves in this way. 

Touching on the specific skill sets needed – because while this is a tech focus – there is still a people element to this as well, and a specific type of capabilities required of the people who are coming to use these technologies. What were the drivers or pain points, specifically, that prompted Reckitt to partner with Sigmoid? 

Ahamed: I started my role just over a year ago. At the time, Amazon Ads was discussing the launch of its audience upload feature within Amazon Marketing Cloud, which is Amazon's data clean room solution. Having my background in direct-to-consumer, e-commerce, and CRM marketing, my eyes lit up as soon as I heard the audience upload into a data clean room, I was just like, "this is crazy."

For the first time, the challenge of seeing direct attribution of media to sales at scale for a CPG could actually be realized. I had a deep understanding of CDP because I worked in CRM marketing before. I also had an understanding of media capabilities because I've done performance media and direct-to-consumer, and worked in e-commerce over the last eight years.

Connecting those dots, I thought we could come to something amazing if we could do this properly. Then, I embarked on this journey of finding the right partner to help us. That meant detailing the opportunity or business problem to about seven vendors that could crunch the data, understand ad media buying, understand CDP, etc.

What it came down to – and the reason we went with Sigmoid for this use case – is the deep understanding of CDP, and how our internal architecture works, specifically for exporting this data into a different platform. Having worked with us before, and having seen good results from other media executions that we'd previously done, made the decision easy to proceed with Sigmoid.

Having a partner that understands the business infrastructure, as well as the internal business process in order to move forward on new projects, makes it easier. That's why we had a timeline that was under a year. If I had to onboard someone new to do this, it would take a significantly longer time, and we wouldn’t get a pilot up as quickly as we did.

Jenkins: That deep understanding of the infrastructure and CDP is invaluable to hit the ground running, and implement a pilot in under a year. Sundeep, do you have any comments on what Imteaz has said about the partnership?

Kumar: For sure, based on our long relationship with Reckitt, we understand the business problems and the navigation skills that are needed to get things done. Reckitt is just amazing. We have had many barriers of experience, which allowed us to know the technology stack that needed to be used to bring this solution. 

For example, our understanding of Amazon's system, and how marketing in Amazon happens in Reckitt? We've got multi-touch solutions deployed in Reckitt so we understand the marketing behavior, the consumer mindset that they follow, how the brand reached the audience – all of that understanding helped us and acted as a catalyst in terms of taking the solution to the market as early as possible. That was definitely an edge.

Jenkins: You mentioned earlier, Imteaz, about being specific in how you set up the KPIs before embarking on an initiative like this. Obviously, specific KPIs of different efforts will depend on the goals, the expectations of a particular campaign, but also the category you're working within. Could you talk more about how you go about setting up those appropriate KPIs for your business?

Ahamed: I previously worked in consumer healthcare, and also in household cleaning. Now, I work in the baby category, which is very unique because you only have this consumer of your product for a certain amount of time. For baby formula, they're only drinking baby formula for under a year typically. You only have about 300 days to realize your lifetime value with that consumer. 

Given that it's such a short period of time, the dollars involved could be anywhere between $600-1,200 worth of lifetime value. The marketing efforts you put in to acquire the consumer, retain the consumer, and extend the consumer are mission-critical. 

Some of the KPIs that we are looking for are, in particular, like new to the brand.How many of those consumers coming to our brand are net new? In the U.S. in particular, there are 10,000 babies being born every day, which also means there's 10,000 babies leaving the category every day. If we don't acquire new babies every single day, we don't have a business in X-amount of time. For this category, lifetime value is mission-critical, followed by how much you are spending on acquiring that individual consumer.

In my previous career, we’d look at profitability by category, business unit, or profitability at a case-level. Here, we aim to look at profitability at a user-level for a certain record in CDP. That's the vision of where we're heading with this. For this particular execution, given the levels of investment that we make within paid media, it's understanding how to shoot a bullseye with every dollar that is spent, or how to get better at targeting so that I only display my ads to people that are in the market for my product. There's no point in showing baby formula products to someone if they don't have a baby, or aren’t interested in the baby category. It was focused on how to reduce reach, improve efficiency, and improve ROAS over time, while still growing new to the brand.

Next, in terms of driving profitability, there's a metric called PPUPY (profit per user per year). If you can drive business to a user-level and understand lifetime value, how much money is spent to acquire that customer? What are the COGS for the products that they're buying? What are the other costs of serving that consumer to drive overall profitability at a user-level?

That provides clarity in terms of what the levers are that you need to push within your business to drive that profit. All these tools help us get there. We're not there yet, it's a journey, but using tools like a clean room to refine targeting and optimize spend helps you get there.

Jenkins: I have a 19-month-old, and I remember the day he turned one. Suddenly, I was passing by the formula aisle and thinking, "Not my problem anymore," because it is really that line there. It's interesting how that customization is necessary. Sundeep, do you have anything to add about how Sigmoid supports in this effort to set appropriate KPIs and drive the PPUPY?

Kumar: Imteaz shared a lot of business KPIs that we have to achieve. Our job was mainly to understand the business, and then understand what business KPIs need to be impacted, then convert those into analytical KPIs. We know that we have to increase sales, but to increase the sales we have to know who to sell. In this, the KPI for the consumer was the audience.

There are cases where we have the first-party data – in this case we have the first-party of the CDP data, as well. We know that those audiences are the analytical KPI that we had. Then, what audience to target and how many impressions the purchase has driven. How many impressions became the analytical KPI. Find the right drivers (impressions), then the consumer (the audience), and then the budget.

We had a marketing budget to spend to achieve that purchase. How do you convert budget into an analytical KPI? For that, it was the bid price – the price we need to deliver impressions and to what audience. It became a bunch of analytical KPIs that we had to target, and that increased the overall sales. It's mainly the analytical KPI, but the way you calculate is the differentiator that we had to add in.

Ahamed: Building on that, the point here is to spend the actual time to understand what you actually want to drive. Everyone is happy to buy new technology, you can buy all the shiny tools in the world, but if you don't spend the time to unpack what the most important things are to drive and how to drive them within the business – you could get lost quickly with the volume of things available to you.

Jenkins: Thinking about who's driving these initiatives, what are the skill sets needed to think about approaching or solving this problem? We're not there yet, but what is needed from a skills' perspective? 

Ahamed: In terms of using a data clean room, the skill sets required would be audience management, media management, understanding of e-commerce delivery, and customer journey mapping. I mention customer journey mapping because within CPG, there are two things that you're trying to drive: First, acquisition – how to get new users into the brand. Second, how to get them to buy again. From a media point of view and a commercial point of view, understanding those are the key skill sets required. 

Then, how to translate those business needs into something a technology partner can solve. Having a connection with an IT or tech partner that can translate those into specific requirements that need to be built, is the basis of how we did this. Otherwise, there are too many stakeholders trying to discern what all the things I want to do, then changing that into specific IT requirements without understanding the business context. It’s very easy to get lost and not build what you need.

Jenkins: Sundeep, what could you add here about the tech capabilities that are needed for this?

Kumar: In general – the way Reckitt stands out from others – the business folks have a good sense of analytical data sets, which reduces the overall team structure needed to drive this. It makes the team extremely lean. While we hired the business folks under Imteaz's team, who had a bit of analytical abilities, we had to have great mathematicians who could understand the business, convert those and drive the models, then get the MVP delivered.

We started with a very lean team – a bunch of principal data scientists and data scientists to build the model and the solutions. Once the solution was deployed and successful, we built out the team to ensure we deployed to production and then scale this. After that, the operational expenses to run this across geographies and brands was low. It was very lean to start, proof, build, take to production, and then have a small, fully-automated setup to run it smoothly.

Ahamed: Building that meant starting with the end in mind, but also getting a pilot that does all the things you want it to do, well enough so that you can validate the output.

Sometimes, when we do pilots, they're small and off the mark. Then, you have no trust in the scaling of it. Whereas here, we did something end-to-end at a rapid pace, we believed the numbers, and were happy to scale.

Jenkins: The knowledge that you mentioned of the infrastructure would help in that end-to-end building, and end-to-end visibility. Imteaz, you mentioned data clean rooms earlier, can you define exactly what a data clean room is, so that everybody understands.

It's a secure, controlled environment for companies to share and analyze sensitive data, but you're not falling foul of any compliance data breaches. It allows companies to compare data sets in a safe, controlled environment and it's becoming a more widely adopted practice. It's a popular tool.

Among companies that use privacy-preserving technologies, we’ve seen that two-thirds of them are now investing in data clean rooms. Could talk a little bit more about data clean rooms and some of the opportunities that you see with this tool?

Ahamed: I love explaining a data clean room through an analogy of a middle school dance. Imagine you have two sides of the room, the boys on one side and the girls on the other. Say the boys are brands or CPG, and the girls are the media publishers. Then, what happens in between, is a highly-chaperoned experience of people talking to each other, doing something, and then going back to the wall. That's a clean room. Basically, you put your data with somebody else's data in a highly compliant, secure place.

There's an interaction and an exchange that happens. It's a learning experience, a doing experience, and then you go back to your corner. A data clean room enables us to do things at scale with compliance and security. It's the way that we have to go in the future because of the cookieless world we’re going into, as well as privacy concerns, etc. It's an amazing piece of technology that addresses a lot of the concerns we have within marketing technology.

Jenkins: Sundeep, do you have any thoughts on data clean rooms? We’re seeing a lot of it in our reporting and research of brands adopting it. What excites you about the opportunities of data clean rooms?

Kumar: Reckitt is almost there in the modernized data architecture system, following the medallion architecture where each of the data sets are exposed as an API. You have the CDP data, which is the first party, which is exposed as an API endpoint for us. Then, the EMC data is exposed as an API endpoint.

We were able to extract this data cleanly, merge it together and feed it into the model. Obviously, if the source is corrupted, then it would be a “garbage in, garbage out" solution, but that was not the case. Reckitt has been managing CDP for many years now, so the data was very clean.

We were able to get the CDP and the audience data, and refine those to get the outputs. Data clean room concept in Reckitt has been there for years, and that helped us spend days fixing data issues. Those issues were not even there. That's a great thing that increased our go-to-market significantly.

Jenkins: Talking about data, you mentioned Reckitt had first-party data to leverage. What would your advice be, Imteaz, for companies that don't? What does it mean for companies that don't have that access for that resource?

Kumar: I’ll take this one. The solution that we have designed, can work for having Amazon audience data, as well. Having first-party data enhances the quality of the solution dramatically, but it is safe to assume that having only the Amazon audience data and some audience data, from systems like back view, would also enable you to improvise and bring a solution.

It is more about how quickly you are able to make decisions, and that can be done by machines much better than humans – if you're able to make decisions on a weekly or  biweekly basis. Let machines take those decisions of where the ad needs to be spent, rather than humans because the factors machines can take care of at a single instance of time would be more than humans, if you're able to feed all the inputs inside that.

Jenkins: Sundeep, can you give us a quick, high-level look at how Sigmoid's solution in this instance works?

Kumar: We built two major solutions. The entire use case was solved by two machine learning solutions that we built. The first one was aimed at driving the purchases, which was a combination of figuring out the right audience, and then the right audience at the right budget. That was a statistical solution we had built.

The other focused on optimizing the cost, because obviously there is no end to the number of dollars that you want to spend to achieve the audience you want. How do you optimize that? That was a combination of the right budget and right settings. When we talk about settings, it's mainly around frequency, the amount of bid price you have to put, and the viewability parameters.

These two solutions, when working in tandem, were able to deliver what was needed. Again, we are talking about a 10,000-foot view. We have the CDP data from Reckitt, which is an optional data set that everyone here should know. Then, we have the audience and campaign data from Amazon. Next, we have the media platform from where we were getting some bit of audience intelligence.

This all gets fed into the optimization model setup. We were able to figure out what should be the right audience and right money to invest. This was configured into the Amazon DSP platform where the building used to happen. The feedback used to get into the campaign monitoring system, where we are seeing if this campaign is something new and the audience chosen was appropriate or not. 

After one week, the system is intelligent enough to realign the new set of audience, and ensure we are not wasting money. This is an improvement from every three or four weeks when case campaign managers have to do it manually. The combination of optimization models set up and campaign monitoring in totality, allowed us to achieve results of a campaign by increasing the ROAS within three weeks of its launch, which is completely unheard of.

Jenkins: You said this implementation took three weeks to put in motion and see results?

Kumar: The creation of this was a 12-week exercise, again, because we had some business context from before, but the scaling of it, yes. 

Any new brand and geo combination can be integrated into this and provide results in three weeks of launch.

Jenkins: Thank you for the overview of how this all works, and some of the impacts it's having. We've seen this done for one retailer and talked here about Reckitt. How could this be applied elsewhere to some of the others?

How are we measuring success, Imteaz, some of the qualitative or the anecdotes that you have?

Ahamed: Over the course of this pilot, we improved purchase per dollar by 400%, versus the benchmark period that we compared against. For context, the benchmark was three set periods over the last two and a half years. We didn't just compare one period. For the ad sets involved, or the campaign line items, we had ROAS improve by more than 10X against the benchmark, and the purchase rate improved by more than 160%.

From an efficiency point of view – and when I say efficiency, I mean team point of view – one person is running this internally from a commercial point of view. My media manager is now able to scale multiple display campaigns using this technology, rather than having to employ another three or four people to run these campaigns, with all of these iterations, and all of these tests. We’re using the Sigmoid solution to do that with one person. The efficiency unlock is amazing, but the results are even better.

We are looking to actively scale this for the rest of our portfolio, over and above the test that we've run on one particular product set and category.

Jenkins: Those are incredible, game-changing numbers. For you, Sundeep, you've done this now with Reckitt. How could this be applied elsewhere in e-commerce with retailers or even in other industries as well?

Kumar: The concept that we did here works like a charm for Amazon. Even if you do not have first-party data – obviously we could not guarantee these numbers – there would still be a dramatic improvement in purchase per dollar, ROAS, and purchase rate. For Amazon, we are good. We know that we are not able to get the same amount of data for other e-commerce retailers. This is for all the e-commerce retailers to know that for us to be competitive, we have to expose our data sets as well.

That would be my honest request. We are trying to build something like ChatGPT for e-commerce marketing. If everyone tries to share data sets, we could build holistic solutions that can work for everyone. This solution, as of today, works very well for Amazon, but we are trying to build this for the Walmarts and Targets of the world, as well. This is industry agnostic, we just need to change some configurations.

If you're doing it for manufacturing – for Reckitt and other brands as well – there are business-level configurations that need to be changed. But for Amazon, we should not be worried and have a very quick go-to-market on all of this.

Jenkins: It's industry agnostic, but what I'm hearing is that it still requires customization. There’s not a one-size-fits-all solution, rather it’s dependent on a lot of different factors depending on the category, market, etc. 

Sundeep, what are some of the tips or best practices for companies starting out on day zero of the journey to optimize their marketing on e-commerce?

Kumar: That's a hard one. What we would request is to first figure out the data sets. You need to have the relevant data set centralized. If not, our first request would be to build a centralized data warehouse, so that you can bring all the data sets together and do baselining as to where you currently stand.

Then, move into AI or ML-related solutions because the solution is only as good as the data sets you have. If you are ahead in a maturity curve where the relevant data sets are there for you, we could just plug the solution, make those changes to the configurations, and move ahead. 

Jenkins: In terms of data clean room testing, when starting out, are there specific partners that are more willing to work in this way, for example, Amazon, Target, Kroger, Walmart? Any additional guidance on budget requirements or considerations for the test phase?

Ahamed: In terms of connecting your data set to a data clean room, we were previously using a third-party service to connect our data to Amazon's data set, but we lost more than 50% of our data through the exchange. Directly connecting the data set from our CDP to Amazon significantly improved that performance or match-rate.

There are third-party services out there that do that for you, and many of the other retailers are pushing people to use the third-party service to connect the data, but there is a match-rate loss. That's one of the significant considerations of doing this: When and if the retailers do allow us to do direct connections, we're going to see better match-rates versus the current solutions today. 

Jenkins: The second part was if you have any recommendations on setting a budget during the test phase for data clean rooms?

Ahamed: I can't give you exact numbers, but from a pilot point of view, you can start under $100,000 in terms of this exercise. However, if your total ad spend per month is less than $10,000, this is not going to work for you. 

Jenkins: Sundeep, how long did it take for Reckitt to start seeing results post-implementation?

Kumar: I'll give the end-to-end timelines: 

• Building a solution was a 12-week exercise. 
• Once deployed, we saw results in three weeks. 
• Then, sustained those results for another three weeks.
• Six weeks was when we were able to prove to the business that the solution works and started scaling it to other brands. 
• Each brand scaling, once developed, was a three-week exercise.

Jenkins: Quite a quick rollout. We are just about at time, Imteaz and Sundeep, do you have any final words of advice, or just final thoughts to leave us with?

Kumar: It's never too late to think about having a ChatGPT-like solution for e-commerce marketing. Feel free to reach out to me, and we'll be able to provide a helping hand.

Ahamed: This is only going to get bigger, you need to get your toes in the water and get started on this now, more than anything else.

Jenkins: The parting thoughts are not to get left behind in this, right? It's an exciting time and area of development. Thank you both for sharing your thoughts with us today and to Sigmoid for sponsoring. Also, thank you to the audience for joining us. We hope you've gained some insights and wish you a great rest of your day. Thank you