ACM RecSys Conference 2021

The Preamble team was in Amsterdam for the 15th ACM conference on recommender systems. The Association for Computing Machinery (ACM) is the world’s largest scientific and educational computing society and seeks to advance computing as a science and a profession. Their Recommender Systems¹ (RecSys) conference brings together research groups and companies from all over the world each year to present and discuss recommender systems research.  

No alt text provided for this image

The team had a great time meeting many talented researchers and industry professionals. We had many intriguing conversations, learned a lot, and saw a few common themes being discussed: regulating recommenders, key problems with recommendation systems, ensuring fairness, reducing bias, and how to better manage large recommendation systems.

Recommendation Systems

Recommendation systems are a hidden but very powerful force in our digital society. These algorithms are typically designed to suggest items of interest to users based on their engagement with the platform. Recommendation systems are deployed in many sectors such as ecommerce, travel, fashion, insurance, media, human resources, finance, restaurants, and many more.

They decide on the flow of information we see, manage our attention, and can alter a company's visibility for better or worse. Companies that have more users will essentially have more power and influence over markets and society as they can optimize their algorithms for user engagement and profitability. Regulators and users have become wary of these companies because their algorithms are difficult to understand and commonly have an underlying bias built in that is not optimizing for the users best interests.

Below we have highlighted three sessions we really enjoyed.

Session: Regulating Recommenders

Natali Helberger², is a Distinguished University Professor of Law and Digital Technology at the University of Amsterdam, her research has focused on how AI and automated decision making are transforming society and their implications for law. She regularly advises national and European law makers on AI and freedom of expression.  

Natalie explained how the European Union is proposing the Digital Services Act (DSA) as   legislation that would establish a framework for online platforms.  The DSA takes a risk based approach to systemic risks. Within the areas of regulating recommender systems, she highlighted the language that requires Very Large Online Platforms (VLOPs) to be transparent and include an accessible interface for consumers. She also discussed how the DSA could kickstart debate for incentivizing recommender systems to promote pluralism, privacy, and freedom of expression.

The DSA has three primary goals³:

  1. Better protect consumers and their fundamental rights online
  2. Establish a powerful transparency and a clear accountability framework for online platforms
  3. Foster innovation, growth and competitiveness within the single market

Regulating these recommender systems is not going to be an easy task, but a modern legal framework is needed to ensure safety of users is a priority while maintaining a fair and open online platform.

Session: Conversational Recommendation Systems (CRS)

The presenters of this tutorial went into great detail about highlighting the two key problems for recommenders⁴:

  1. Information asymmetry — the system can only make recommendations based on historical data
  2. Intrinsic limitation — the users preferences drift over time and can be difficult to identify why the preferences have changed

They defined a CRS as “a recommender system that can elicit the dynamic preferences of users and take actions based on their current needs through real time interactions.” The biggest advantage is the system can ask the user questions. CRS's can be based on interactive dialogue or based on interactive button-clicking. CRS's can bridge the gap between search and recommender systems. CRS's do have their own set of challenges, but have the potential to use user feedback to make better recommendations.

Session: Best Practices for Operating a Large-Scale Recommender System

Mohammad Saberian⁵ from Netflix discussed the very challenging problem of maintaining a healthy large-scale recommendation system. Large-scale recommendation systems are very complex as their environment is dynamic and changing every second. To ensure the recommendation system is working properly, it is imperative to focus on these four components:

  1. Issue Detection — it is the most challenging part and is essential that issues are identified quickly. It is imperative to implement best practices such as: unit tests, MLOps, CICD, and regular training. It is essential to make sure the entire pipeline is working end to end.
  2. Prediction — ideally we want to predict issues before they happen. We can train models that predict the production model’s behavior. We want to flag any unexpected predictions in advance and investigate them.
  3. Diagnosis — need to reproduce the issue in isolation. It is imperative to have advanced logging. To determine if it is a data or model issue, tools like SHAP and LIME can be used to help.
  4. Resolution — similar to software engineering there are hot fixes and long-term solutions. Hot fixes in ML are challenging because models are highly optimized and hot fix modifications will lead to sub optimality.

Focusing on these four components will help identify the issues and help resolve them in large-scale ML recommender systems.


It was great to meet so many talented researchers and industry professionals who are working on building safer and more accurate recommender systems. With everything that is going on currently, and with the Facebook whistleblower explaining how algorithms can be optimized for user engagement and profitability. It is reassuring to see that many researchers and industry professionals are tackling issues surrounding algorithms to ensure they are working towards increasing the safety and fairness for consumers.

We had a great time and are looking forward to being at ACM RecSys 2022 in Seattle!



Background shape

Get our updates.

We’ll notify our community of new libraries and AI tools released.

Start taking control of your enterprise AI systems today