3 Major Recommendation Algorithm Mistakes Fortune 500 Companies Make

Several recommendation algorithms power email-marketing campaigns as well as on-site product recommendations. With Amazon’s success in driving revenue and engagement from product recommendations, several companies leverage these algorithms to cross-sell/up-sell products to users.

The data science team at Retention Science has helped power onsite/app and email recommendations for more than 75 e-commerce companies. With over 20 different flavors of recommendation algorithms running in production, we’ve seen high variability in data and businesses. These algorithms are inherently designed to maximize click through rates or conversion rates.

We’ll be discussing 3 problems that most recommendation algorithms overlook and some solutions to fix them.

Problem 1:

“Why am I being shown duplicate recommendations or the same over and over again?”

On average, about 60% of items in an e-commerce site are a potential duplicate of another item. Duplication occurs due to different configurations of the same product such as color, size, length, volume, etc or sometimes extremely similar products. There could be temporal duplication as well, where users are shown the same impression over and over again.

De-duplication of product recommendations is a tough problem. Several Fortune 500 companies are guilty of not de-duplicating their recommendations (see Fig 1.1).


  • Start out by annotating duplicate items by measuring similarities of product name and descriptions.
  • Work with vendors in deciding on rules on how to select the parents of these duplicates.
  • Once items are annotated with their parent duplicates, ensure a filter system (look at Bloom Filters) is de-duping duplicates in real-time.
  • Ensure temporal de-duplication by keeping track of what products were sent out earlier and filter them in consecutive sends (if it’s seen by users).
  • Add reinforcement learning or an exploration parameter to ensure temporal duplication is not taking place.


Fig 1.1. Four weekly email recommendations of a prominent brand.
1st and 2nd emails contain almost the same items.. Duplicates milk item shown on 3rd email and most items are repeated week after week. Kroger/Ralphs example)


Fig 1.2. Three weekly email recommendations of a prominent brand.
All 3 emails contain the same recommendations with different headers and orders. (Walmart Labs example)

Problem 2:

“I already bought/seen this, why am I being shown this?”

Most recommendation systems collect several data points on a user, and their transaction and on-site behavior are commonly tracked in real time. Recommendation algorithms do not have an explicit rule built in that prevents recommending items users have already seen or bought. This is commonly known as the banana problem.

The primary goal of recommendation systems is to make users discover and buy new products rather than show you products you already know or have purchased.

Putting a filter system in place for this kind of item serves 3 purposes:

  1. Keeping users engaged and encouraging exploration of other products.
  2. It helps the recommendation engine not get false attribution for repeatedly bought products.
  3. With limited data on users, algorithms get stuck in what’s called “local minima.” Users might experience the same recommendations over and over again.


  • In lifecycle marketing, ensure that you’re not recommending the same products on different touch points.
  • Several dictionary based methods help you keep a running tab on the recommendations already shown to users which you should use to create filters.

Problem 3:

“I’m a female, why am I being recommended male products?”

Recommendation algorithms usually pick up gender as a latent feature or as a user context. However, it’s not completely guaranteed that male users won’t receive products that are primarily “feminine” in nature or vice versa.

This usually leads to users not clicking through and developing a disinterest at the recommendations, as it doesn’t seem tailored to them. It’s highly prevalent in businesses which have a mix of male and female users and have different product inventories tailored to each gender.


  • If the gender of a user is not known, annotate by using their first and middle names.
  • The gender identification for an item can be annotated by looking at the dominating gender buying the product, however some products could be neutral.
  • In cases where the buying patterns are not obvious, Retention Science deploys natural language processes to determine the gender for the item.
  • Once these 2 unknowns are annotated, a simple filter system can start removing gender specific items.


Fig 1.3. Product Recommendation in an email for a male user, with products primarily bought by only female users.


Our filter system Redicto applies these post processing rules as well as some other business domain rules. See Fig 1.4 for the flow.


Fig 1.4. Redicto incorporating these rules to enhance the recommendations from algorithms.


These filters are implemented as post processing steps. Different variations of these filters are tried on multiple algorithms, then post processed recommendations are used in our A/B test platform. Table 1 shows the performance of these filters in terms of click rates. Each filter individually brings in a statistically significant increase in click rates, and when combined together in series we observe a lift of ~4% for two of our algorithms.

The validation results are encouraging especially for e-commerce businesses with high product diversity.

Screen Shot 2017-01-26 at 2.53.19 PM

Fig 1.5 shows how Click rates and conversion rates increase on using each filter by Redicto

About The Author

Vedant Dhandhania is a Machine Learning Engineer at Retention Science. He helps predict customer behavior using advanced machine learning algorithms. His passion lies in the intersection of Signal Processing and Deep Learning.


Scaling Recommendation Engine: 15,000 to 130M Users in 24 Months


Interested in learning the right way to retain your customers? Get some important insights from us.