3bij3 – Developing a framework for researching recommender systems and their effects

Abstract

Today’s online news environment is increasingly characterized by personali-zed news selections, relying on algorithmic solutions for extracting relevant articles and composing an individual’s news diet. Yet, the impact of such re-commendation algorithms on how we consume and perceive news is still understudied. We therefore developed one of the first software solutions to conduct studies on effects of news recommender systems in a realistic set-ting. The web app of our framework (called 3bij3) displays real-time news ar-ticles selected by different mechanisms. 3bij3 can be used to conduct large-scale field experiments, in which participants’ use of the site can be tracked over extended periods of time. Compared to previous work, 3bij3 gives re-searchers control over the recommendation system under study and creates a realistic environment for the participants. It integrates web scraping, diffe-rent methods to compare and classify news articles, different recommender systems, a web interface for participants, gamification elements, and a user survey to enrich the behavioural measures obtained.