Right now, Wayfair has about 86,000 rugs available for purchase. On the first page of our rug category page, however, we only have room for the top 48. You see even fewer if you don’t happen to scroll or swipe down. When it comes to our customer’s experience, what we choose to place at the top makes a big difference! At present, we don’t explicitly showcase a diverse array of products. On browse pages for categories such as rugs, couches, or coffee tables, we instead combine personalized recommendations with the general popularity of products.
How diverse are the choices that we display to customers at the top of a category page? For example, are we showing too many of the same kind of rugs in a row? We were curious to quantify the diversity of the pages our algorithm naturally produces, and to see whether it made a difference to customer interest. Armed with a way of measuring diversity, we could rearrange these items in a new order to maximize the variety of rugs, couches, or coffee tables shown, and see whether we’re still emphasizing popular products. Our goal was to expose more choices to meet the delightfully different needs of our customers, and to make our category pages more colorful and fun to visit.
Quantifying the diversity of Product Category pages
How can we quantify the overall diversity of a Product Category page? First, we can consider that a page is as diverse as the items displayed on it are “distant” from one another. This underlines a definition of “distant” that corresponds somewhat to our customer’s perception of this experience. For the page as a whole, we can take an average of the distances between each pair that makes up our list of 48 top items.
Second, it’s handy to note that distance is just the flipside of similarity. Too many similar items next to each other may look redundant to the customer. What we would like to compute is the inverse of how similar every item is to every other item on the page.
One way to measure similarity could be to look at which products were clicked by the same person during their visit to our website. The problem with this approach is that the most popular items, at the top of the page, tend to get clicked on together quite often. They would be registered as similar even when they aren’t - they’re just at the same level of popularity.
A way to avoid this problem is to instead measure similarity based on which products were added to the same Idea Board.
Idea Boards are private lists of products built by customers by adding items they find inspiring. It captures something about their tastes and preferences, and often groups products further by room or theme. These boards are very popular, and cover a surprising percentage of our catalog. On a typical day, Wayfair customers create tens of thousands of Idea Boards, and add hundreds of thousands of items to them. Since these lists are carefully curated, items are less likely to be affected by placement, for example, by customers casually clicking on the top few items.
Computing a Jaccard index is a simple way to quantify which items were often found on the same Idea Board, adjusting for how popular those items are:
[latexpage]
\begin{equation}
\frac {A \cap B}{A \cup B}=\frac{number~of~idea~boards~containing~a~and~b}{idea~boards~containing~a~+~idea~boards~containing~b}
\end{equation}
Where a and b are two products, and A and B are the sets of Idea Boards containing those products.
The validity of this metric was easy to verify for specific categories, simply by taking popular products and putting them side-by-side with the closest match according to this score. The closest match was both conceptually and visually similar in every case. Let’s take rugs to provide a concrete example.
By using the above metric, the distance between two products could then be defined as the inverse of these Jaccard scores. This allows for the diversity of the page as a whole to be based on the collective distances between each pair of products. With 48 products, there are 1,128 pairwise combinations, and since Idea Boards are so popular, almost every one of these combinations has been added to an Idea Board. We can use the median as the summary score to avoid the effects of outliers, including pairs of products that have never been added to the same board, and are thus quite distant, but we don’t know exactly how much.
Does increased product diversity boost customer interest?
With a pagewide metric showcasing the diversity on Product Category pages, we’re now equipped to ask the following: Does the customer click rate increase on more diverse pages? Do they buy more?
To answer this question, we can look at what products we actually showed customers on Product Category pages and how the pagewide diversity varied. A random sample of 1,200 category pages shown to customers over the last 10 months was a substantial data set, of which we were able to compute the pagewide diversity of each.
As you can see in the following figure, there was considerable variation in the diversity of Product Category pages, both due to gradual trends in our regular sort, and sharper variations due to experiments and changes in our algorithms. This differed across other top categories, too: Kitchen and Dining Tables had the most consistently high level of diversity, while Beds had the most variation in the diversity of pages that were shown to customers.
Knowing that customers saw more and less diverse pages, we can dive a little deeper once more. Did this make a difference to customer behavior?
On grouping diversity scores into quintiles, we found that both the probability of customer clicking on at least one product on the page and them adding at least one product to their cart increased with the diversity quintiles.
This was very promising! The data above shows that increasing diversity could make customers more interested in our Product Category pages, and more likely to find the products they want to buy.
Building a diverse page of rugs with a greedy algorithm
So how would be go about creating this diversity? We chose the greedy algorithm approach, which would build a diverse page by choosing products one at a time, maximizing the distance from the set of products that have already been chosen. An advantage of this method is that we achieve more diversity at the top of the page, which is seen by the most customers. Furthermore, the overall page generated by a greedy algorithm should come close to maximizing the pagewide diversity metric.
Let’s start with a seed, such as the most popular rug of the moment, and add rugs one at a time from the pool of the first eight pages of the usual category page assortment. Conceptually, this re-arranges already-popular rugs but increases the diversity of our grid of recommendations. Each choice is the one that had the highest average distance (our inverse Jaccard score for Idea Boards).
Did we manage to successfully produce a more diverse Product Category page? The scores on the page-wide diversity metric were higher than almost any category pages that were shown to customers. For example, our generated rugs page had a diversity score of 97.8, 4 standard deviations above the mean of real area rug page loads.
Balancing popularity and diversity
Here’s a visual representation of one of these maximally diverse pages of rugs.
This definitely has some product diversity, alright – perhaps too much! Many of these rugs only sold a handful of times in the last month. Maybe some would perform better if we gave them a chance in the spotlight, near the top of the page. However, not every product will be suitable for everyone, and probably shouldn’t be so high up our list.
Let’s add popularity as a weighting factor during the greedy building process, incorporated into the equation when deciding which product to pick next. The nice part about this approach is that the relative importance of product diversity and popularity can be tuned, like a knob that goes from “all popularity” to “all diversity”. The result from setting the knob to “50%” shows how we can mix up the top selections without putting odd or unpopular items at the top.
What really counts is how valuable our customers find these pages. At Wayfair, we’re always testing improvements to our algorithms and incorporating user feedback to provide a seamless and delightful user experience. These efforts enable Wayfair to march towards a best-in-class showcase for our “zillion things home”.
Interested in further insights from our Data Science team? Find more information via the Data Science category on the blog.
