Hover over the dots to explore related posts. Closer dots are more semantically related, and the red dot marks the current page.
I was invited to give a talk at the SIGIR Workshop On eCommerce 2019 but unfortunately, I am not attending SIGIR this year. Instead I wrote down some thoughts on interesting problems, ideas and challenges in the e-commerce domain. Here they are:
The vocabulary gap is still an open problem.¶
People refer to products in different ways that products are described in the catalogue. Neural networks, learning to rank, query intent engines are all important. Our experience at 904Labs shown that a query intent engine that boosts specific product categories given a query, boosts our learning to rank system by more than 16% in additional revenue. These results suggest that effective initial ranking is very important for effective learning to rank. To this end, we foresee an increasing interest in methods that can re-rank the entire collection and not only the top-N documents. This is particularly important for larger shops with large inventories (more than hundred of thousands of items) where a query can return hundreds of items, and only the top few are re-ranked.
E-commerce search is as much as about exploration as it is about finding the best match.¶
From our experience at 904Labs, we see a large fraction of queries to revolve around categories, or combinations of categories, e.g., red shoes, kitchen tables, dvd players, ebooks for 12 years old. This type of queries go beyond our typical search and require understanding the query and generating a list of recommended relevant items. This proposition is supported by the surprising effectiveness of sorting by popularity; the most popular items for a query are potential good candidates for this “recommendation list” that the user is looking for. Back to query understanding of this type of exploratory queries, one would think that natural language processing can help here but in the e-commerce setting, queries are very short and any language analysis falls short.
An open question here is how can we go from these broad queries to a good set of recommendations?
A natural way is to devise a hybrid system of search and recommendations: We fire the query to a search system and then we take the first few items as seed to a recommender system for getting similar items. Or a system that transforms a query to an image (think AttnGAN or similar) and each product to an image and then rank documents by their image similarity to the query’s–the image representation may constraint the latent space, abstract the language of the query and that of the document and be able to capture semantics that are otherwise difficult to encode in textual form. The image representation of queries and documents also offers explainability when it comes to explaining the rankings of the system; which is becoming increasingly important in machine learning-based systems.
Evaluation metrics in e-commerce.¶
There are several directions here that we need do more work. First, the e-commerce setting is a conjunction of exploratory search, typical search, and recommendations. Using the standard IR precision and recall measures for e-commerce may not tell us the entire story for how happy makes its users. We need to discover the aspects of a system that makes users happy for designing one or more metrics for evaluating search and recommendation systems. These metrics should also correlate well with revenue but also with customer loyalty (measured in returning customers and in shortening the time between returns). Such a metric (or a multiple of metrics) will then allow us to run offline experiments and make predictions on revenue, which is the main KPI that systems are evaluated in production for most e-commerce business.
Last, an extra tip: we found that boolean scoring may be on par or outperform tf.idf or BM25 scoring in the e-commerce domain, it’s worth checking its effectiveness on your own data ;)
Related Posts
-
As eCommerce is becoming increasingly important, we recruited a small team of researchers from major and not so major players in the field to share views and experiences on theoretical and practical challenges and future directions on eCommerce search and recommendations. The result of our effort, Challenges and Research Opportunities in eCommerce Search and Recommendations is published in June 2020 issue of SIGIR Forum.
This work is related to thoughts that I shared earlier on product search, and to 904Labs’ Query Intent engine and its magnificent results (examples).
-
In our talk, I described the importance of search in e-commerce by giving examples of failed searches in a number of settings, from finding advertized items using onsite search to mobile search. I followed by with why people choose Amazon to start their product search and highlighted that 54% of them choose Amazon because of their great search functionality–that is the reason number five for people to go to Amazon. Then, I explained why optimizing the ranking manually is close to impossible for humans by laying out the insane amount of options available and enumerating the search space (which can be at millions of millions of choices). With this as foundation, I talked about machine learning and the particular type of machine learning that we use at 904Labs for optimizing search rankings in real-time. I followed up with describing our query intent engine, which is powered by 904Sense, and our automatic synonym extraction engine. In my conclusions, I re-iterated that A.I. for Search
-
In the last couple of weeks, our team has been busy tweaking our engine and pushing it even further. Our Query Intent/Understanding engine was doing already a good job, mapping query free text to product attributes such as category, or brands. However, we’ve never tested going deeper into particular product attributes, such the product weight or even, wattage. Our engineers managed to extend the logic and the scalability of the algorithm to support very fine grained suggestions, if such fine-grained information exists in the index. It is quite a feat and I’m very proud of our team!
-
In the past year, at 904Labs A.I. for Search, we’ve put a lot of effort to optimize and push further the technology behind query intent/understanding engines. If you are in the search engine/information retrieval community, you know the term and how challenging the problem is. But if you are not familiar with the problem, it is hard to grasp its importance and challenges. Here’s my attempt to explain it in simple words.
A Query Intent/Understanding engine aims at identifying the intent of a user’s query and ultimately at translating it to a set of search directives.
Consider a user coming to your e-shop and typing in the search box: “red shoes”. The query may look pretty straightforward to you when it comes to what products to show to the user but, for a machine, it’s pretty hard to figure out what the person meant.
-
I’m happy to share some great results with regards to the utility of self-learning search and revenue on e-commerce sites. 904Labs self-learning search improves revenue by 36% when compared to a highly, but manually, tuned Apache Solr search engine. Good job 904Labs team!
Read the full blog post at 904Labs, here: https://www.904labs.com/en/self-learning-search-improves-revenue-for-e-commerce
-
This year I talked at the industry track where I shared my experiences from running a search startup in the Netherlands. I received positive feedback, and it was good to catch up with old friends and meet new people. If you want a short recap from the latest and greatest trends in search technology, read on my summary on 904Labs blog.
-
Starting my talk at ECIR 2018 Industry track. Courtesy of Gabriela Kazai. European Conference on Information Retrieval (ECIR) is a annual European scientific conference around bhe topics of search engines, recommender systems, text analytics, user modeling, and evaluation. This year ECIR was held in Grenoble, France. With more than 250 attendants and 4 days packed with tutorials, workshops, research, and industry talks, it was a great place to be to get updated with the latest and greatest about search engines.
The theme of this year’s Industry Day was to bring lessons learned from industry to academia. What are differences when developing a research algorithm and when we bring to practice? These lessons could inspire and inform our fellow academics for the challenges practitioners face when bring these algorithms to production.
