2.1 Methodology overview

In line with the spirit of improvisation and iteration recommended by de Goede et al. (2019), we consider the eclectic mix of approaches and methods that follows not as a fixed or complete recipe for the investigation of how normativity informs complex technical objects. The different ingredients have been encountered and chosen inductively, from extensive literature review and our own situated experiences with studying AI systems from our disciplinary backgrounds in media studies, law, and computer science. As Straube argues in his attempt to adapt social scientific research to the study of digital devices such as algorithms, “the most helpful methodological toolkit depends heavily on the precise context of the research and the affordances presenting themselves in the field” (2019, p. 188). In our specific case, and for corporate owned- and operated AI systems in general, the toolkit is heavily predicated towards methods that do not require access to actors working at the companies in question. While we do discuss ethnographic approaches in chapter four, this is not an attempt to adapt traditions like multi-sited ethnography (Marcus, 1995), as Bosma (2019) does in her work on digital security technologies, which still rely mainly on interviewing and observing the (human) actors assembling around these systems. Instead, we take inspiration from projects like Eriksson et al.’s (2019) in-depth study of Spotify, which faced considerable resistance from the company, to think about how to complement “front-end inquiries (such as interviewing) with experimental back-end studies of digital media infrastructure, metadata generation, and aggregation practices” (p. 2). Their opportunity-driven approach, which embraced unpredictability and combined a variety of methodological “interventions”, including setting up a record label and intercepting network traffic, is indeed a good example for what we have in mind.

Our proposal is thus both more limited and more expansive than “traditional” science and technology studies. More limited, since we are specifically interested in articulations of normativity that emerge around and within technical systems, which help us explain and account for the behavior of these systems. This comes at the price of losing other dimensions of analysis, but allows us to connect more directly to the normative debates that surround AI, not least questions of regulation. The goal, eventually, is to critique as well as analyze. We also leave to the side “effects”, such as social impacts, focusing on what goes into the machine, again to understand how and why it does what it does. More expansive, because we are specifically interested in the knowledge traditions and broader modes of reasoning that inform the work on recommender systems in general, across sites and cases, before concrete systems come into play. And, on the other end, we consider concrete systems themselves to be central “informants” about the normative commitments they incorporate, foregrounding methods that make their behavior amenable to critical examination.

While the specific methods mix has to be calibrated to the case at hand, the last two points suggest a more general setup, organized around three layers, that has broader applicability. Informing both conceptual reflection and empirical research, we move from the more general to the more concrete and distinguish ambient technical knowledge, local design conditions, and materialized values.

Ambient technical knowledge addresses the “archives” and traditions technologists draw on when designing a system; the field of recommender systems has a history and a “substance” that includes technical knowledge, best practices, evaluative procedures, and readily available code. As Agre (1997b) argues, computer science overall and specific subfields in particular come with their own “cultural forms”, that is, with “linguistic forms, habits of thought, established techniques, ritualized work practices, ways of framing questions and answers, genre conventions, and so forth” (p. 150), which technical practitioners are not necessarily fully aware of. The engineers and scientists working on YouTube’s recommender systems enter the company not as blank slates, but are already immersed in cultures of knowledge and practice. In addition to providing the first normative cues, reconstructing these cultural forms has two other objectives: on the one hand, it helps researchers from non-technical disciplines acquire some level of technical knowledge, combating what Van Veeren (2018) calls “invisibility as inexpertise”; on the other hand, it provides elements and dimensions to look for when analyzing what comes and, crucially, what comes not into play in concrete production settings. Local design conditions then stand for the particular circumstances and considerations that influence how a concrete system is designed. While access to companies’ internal reasoning is often not an option, there are other materials, publications, and informants to draw on to further understand the choices and normative commitments made in the construction of a technical object. Analyzing the economic and legal context can inform on incentives and other conditioning forces that weigh on these choices. Attending to materialized values, finally, responds directly to the call “to complement ethnographic approaches with novel methods to observe technology” (de Goede et al., 2019, p. 15). It addresses technical systems themselves as normative agents, in the sense that they act and provide opportunities for action in ways that are directed and prescriptive. Contemporary systems’ internals may be opaque, but they are “observable” (Rieder & Hofmann, 2020) to a certain degree, affording the opportunity to investigate their normative thrust by analyzing their interfaces and technical operations.

This separation lays a conceptual grid over the manifold and distributed ecologies of normativity surrounding and pervading technical systems and serves to organize the analytical approaches we discuss in what follows. Our primary goal, here, is not to perform a full-fledged study of YouTube but to sketch a methodological toolbox and to propose an adaptation or recalibration of encircling to the domain of computational artifacts. However, taking a specific case as an example allows us to flesh out how one can apply and combine the various approaches, pointing to concrete materials and strategies for analysis as well as highlighting some of the difficulties and limitations that studying proprietary systems inevitably entails. The heterogeneous and multi-disciplinary nature of this endeavor means that parts of the analysis will vary in style and depth. Where possible, we focus on creating what could be called “analytical stubs”, contained prototypes for how a larger analysis could look like and what one may be able to learn from it. For example, we briefly look at key papers from the history of recommender systems and publications coming out of YouTube through the lens of discourse analysis. For other methodological elements, however, we proceed more in the form of an evaluative literature review since setting up our own ethnographic study or data-driven analysis is beyond the scope of a single paper. While the particular weighing of elements is in part an effect of their varying degree of difficulty and cumbersomeness, it is also an effect of the exploratory process suggested by encircling itself: rather than follow a clear blueprint of methodological steps, it asks how we can “describe and analyse objects and terrains that are not directly visible for multiple reasons” (de Goede et al., 2019, p. 14) and the specific configurations of visibility and invisibility will depend in large parts on the case at hand.

Coming from three different disciplines — media studies, law, and computer science — that subscribe to wildly different epistemic cultures, we are not suggesting that our approach serves as some kind of meta-methodology that can integrate these disciplines into a singular perspective. Instead, the question of how technical objects integrate and operationalize normativity has created a productive space of encounter and exchange where different concepts and methods could be brought into conversation. When we argue that the combined presentation and discussion of these methods allows for drawing a more “holistic” picture of the various normative forces or inputs going into complex technical objects, we do not suggest that every research project has to combine our three layers and associated methods, but rather that there is value in realizing the many different dimensions that come into play. There is no single answer to the question this special issue asks, namely how to “explain machines” that resist explanation in fundamental ways (Burrell, 2016).

In his critical discussion of recommender systems as “traps”, Seaver (2019) frames these systems not merely as socio-technical artifacts but argues that “epistemic and technical infrastructures come together to produce encompassing, hard-to-escape cultural worlds, at a moment when the richest companies in the world dedicate most of their resources to getting people hooked” (p. 423). Our approach seeks to contribute to our understanding of this world-making process, focusing on the production side and on the register of normativity, inquiring into the materialization of values across different spheres. Each of our layers adds to that process in specific ways and there is no singular “key” that unlocks the whole “secret”. The specific combination of methods and disciplines we put forward, then, does not fuse into a singular perspective, but traces a fragmented progression across a number of dimensions.

While readers with different disciplinary backgrounds will have different levels of familiarity with the methodological strategies discussed in what follows, we believe that articulating them together can serve audiences in several practical and research contexts: for computer scientists and technical practitioners, a broader horizon with regard to normativity and how to examine it can alter the view of their own role as value-setting agents and potentially open pathways towards what Agre (1997b) called “critical technical practice”, with “one foot planted in the craft work of design and the other foot planted in the reflexive work of critique” (p. 155). For legal practitioners and regulators, an understanding of values as spread over wider ecologies can help identify relevant points of contestation besides the algorithmic level. For researchers in the humanities and social sciences, the fields this paper most closely associates with, the different approaches discussed over the following sections can facilitate connecting with artificial systems that are notoriously difficult to study due to their complex technical character and corporate embedding. While several of these approaches require a certain level of technical skill, they point towards possibilities for transdisciplinary collaboration similar to the discussions that led us to write this paper.

3.1 Critical reconstruction

While a general historical anthropology of recommendation would provide an even more comprehensive starting point, a somewhat less expansive way to approach the field of recommender systems is what Agre (1997a) called “critical reconstruction”, that is, the attempt to identify, examine, and interpret the “fundamental ideas and methods” of a field, in a way that “eases critical dialogue between technology and the humanities and social sciences”. For us, this first means to historically trace and expound the various attempts to make the cultural gesture of recommendation a computable task. Recent examples of archeological or genealogical work in this direction (Mackenzie, 2017; Rieder, 2020) show how approaches sensitive to historical unfolding can draw on key publications and debates to reconstruct technical trajectories and narrow the gap between often forbidding technical detail and broader conceptual understanding. These texts are not simply taken as stepping stones toward the “state of the art”, but as primary sources or materials that need to be submitted to critical analysis and interpretation.

In the case of recommender systems, one could start from Elaine Rich’s (1983) Users Are Individuals, which presents Grundy, a personalized fiction literature recommender system that draws on explicit input in the form of self-descriptions and on implicit feedback based on the idea that “[o]ne of the simplest ways to derive information about a user is to look at the way he uses the system” (p. 205). Grundy was organized around a “user model” consisting of several facets, such as “interests” (e.g. “sports”), “politics” (e.g. “liberal”), or quantified psychological traits such as “tolerate-violence”. These facets served as the features of a user-vector that was compared to the manually attributed book-vectors in the collection.⁴ The system then began to suggest the book titles with the greatest overlap. As users provided feedback on recommendations, their profiles were updated accordingly. Jussi Karlgren’s (1990) An Algebra for Recommendations is another early contribution and although it received relatively little attention, it proposed a forward-looking experiment involving a multi-user system and discussed implicit “grading” of documents through “behavioral criteria, like numbers of visits to it, access time, or other observable factors” (p. 3), anticipating the heavy reliance on empirical metrics of contemporary recommenders.

The system that is often seen as inaugurating “modern” recommender systems is GroupLens by Resnick et al. (1994). Unlike Grundy, which relied on competent staff for book classification, GroupLens was fully automated, in the sense that recommendations were made on the basis of comparing user behavior, leaving content characteristics to the side. This method, called “collaborative filtering”, defines the basic setup of many systems to this day: “Collaborative filters help people make choices based on the opinions of other people. […] The rating servers predict scores based on the heuristic that people who agreed in the past will probably agree again” (p. 175).

“Agreed in the past” is again established by comparing vectors, this time representing user profiles that register engagement with content items. If two profiles are found to be “similar”, items “liked” by one user but unknown to the other can be recommended. This eliminates the need for editorial content classification and operationalizes recommendation as a purely behavioral function, as a market where users’ actions determine the relative value of circulating items. This shift explains, at least in part, why contemporary recommender systems have great difficulties dealing with “unwanted” content, including misinformation or political extremism.

In 1997, Resnick co-edited a special issue of Communications of the ACM on recommender systems together with Hal Varian, a well-known microeconomists and Google’s longtime chief economist. Their introduction not only summarizes different approaches to data collection and aggregation, but discusses social implications, business models, and the competitive advantages of having the largest user base possible. As is often the case in computer science, there is an “intuitive” justification and metaphorization of what is essentially a set of similarity calculations between users:

In everyday life, we rely on recommendations from other people either by word of mouth, recommendation letters, movie and book reviews printed in newspapers, or general surveys such as Zagat’s restaurant guides. Recommender systems assist and augment this natural social process. (Resnick & Varian, 1997, p. 56)

The identification of problems to be solved is indeed a locus of normative work in technical disciplines. The citations above indicate how technical publications formulate “problematizations” (Foucault, 1990) of the settings they intervene in and, in doing so, define their own specific ideas, techniques, standards, or “truth criteria”. Key narratives, here, are often built around notions such as “information overflow” and “information need”, which have dominated the field of information retrieval for decades (Rieder, 2020). But we also find elements that are more specific to recommender systems, e.g. that past behavior predicts future behavior or that people who have agreed in the past will again do so again, ideas that risk operationalizing a largely inertial view of people’s preferences.

While a more stringent scientometric analysis is beyond the scope of our paper, identifying key publications is the backbone of “reconstructive” and hermeneutic historical work. This can include a survey of the dominant techniques in the field, which are broadly divided into content-based, collaborative, and hybrid methods. But as a field matures, it often diversifies and singles out problems that arise in specific application contexts. When looking at very large systems like the one at work in YouTube, issues like scalability (how to make “good” recommendations quickly as user base and content archive grow?) and sparsity (most users have only seen a tiny percentage of available videos) have received much attention. Techniques based on matrix factorization and deep learning have been particularly successful at dealing with these problems.

Technical literature surprisingly often relies on psychological and sociological lay theories, often ignoring established knowledge from these disciplines. Evaluative practices that are organized around empirical measures have played a central role in keeping the need for deeper domain theorization at bay: success has most often been defined in terms of some form of congruence or overlap with actual user behavior, for example through lab experiments or in-situ measurement (Herlocker et al., 2004). How evaluation practices of recommender systems have evolved over time can thus be particularly revealing. In a recent paper, Seaver (2019) shows how “measures of success” have come to rely mostly on “captivation metrics” that are built on concepts like user retention and engagement rather than more contained definitions of “successful” prediction. Here, we find at least implicit affinity with concepts such as Samuelson’s (1938) notion of “revealed preference”, where behavior is taken as a direct expression of desire or volition, making behavioral signals unproblematic signs of user “satisfaction”. Reconstructing these movements allows us to appreciate variation and contingency in technical fields and provides a structured way to work up to present day debates.

3.2 Discourse analysis

Clearly compatible with a historical perspective, (critical) discourse analysis (Foucault, 1969; Wodak & Meyer, 2016) can also be productively used to identify central tenets in contemporary recommender system thought and practice. Based on the idea that “language functions in constituting and transmitting knowledge, in organizing social institutions or in exercising power” (Wodak & Meyer, 2016, p. 7), discourses — understood as “relatively stable uses of language serving the organization and structuring of social life” (p. 6) — can be analyzed to understand how a group or field constructs and mobilizes a certain “worldview” that defines problems, concepts, norms, modes of admissible argumentation, and so forth. Taking sets of documents as material or data to be analyzed allows researchers to describe and analyze such discourses.

While we focus on academic material in our short dive into the field of recommender systems, industry publications and developer websites such as StackOverflow or Kaggle provide interesting windows into practitioner spaces, and local meetups could add an ethnographic dimension. But even analysis limited to academic literature already has to deal with huge quantities of published material. Anthologies and textbooks, such as the Recommender Systems Handbook by Ricci et al. (2015), which seek to package the “state of the art” into a single volume, are thus not only highly referenced and used heavily in teaching, but also helpful for analysts interested in examining a field from the outside. There are clearly many different aspects that one could take under scrutiny, but evaluation again stands out as a place where normative concerns are most explicitly formulated. One of the three chapters in the Handbook dealing with recommender system evaluation (Gunawardana & Shani, 2015), for example, specifies 14 properties or criteria that could be used as guiding values to design and optimize for: user preference, prediction accuracy, coverage, confidence, trust, novelty, serendipity, diversity, utility, risk, robustness, privacy, adaptivity, and scalability. The discussion of these properties yields a space of reasoning that structures normative concerns into addressable parameters and provides concepts and methods to talk about, justify, and evaluate design decisions, including potential trade-offs, for example between accuracy and diversity (Gunawardana & Shani, 2015, p. 280). Value articulations inform value practices, and metrics play a crucial role in turning broad ideals into operationalized yardsticks.

While such lists of concerns can be useful, we also find more open-ended discussions in the literature, e.g. when Herlocker et al. (2004) debate why it is so difficult to evaluate recommender systems or when Jannach and Adomavicius (2016) survey and discuss different understandings of “purpose”. The latter explicitly distinguish between “value for the consumer” and “value for the provider” (Figure 2), indicating that there is some level of contradiction or compromise between two “sides”. Such lists can be used as conceptual frames for the analysis of concrete systems, including the purposes that play at best a subordinate role, e.g. the lack of effort to help users “understand the item space” on YouTube. In these areas, we also find particularly revealing instances of overlap and exchange between “cultural” and “technical” arguments, including a straight line linking technical features to product design and business planning.

These somewhat less visible domains of value articulation and arbitration are increasingly complemented by more explicit discussions of normative principles. Terms like “accountability”, “transparency”, “ethics”, or “explainability” have become common within AI discourse and there is growing interest in what Morley et al. (2020) call “ethics tools”, that is, “methods and tools available to help developers, engineers and designers of ML reflect on and apply ‘ethics’” (p. 7). While it is hard to say how far into the field these efforts reach, they are showing a certain level of awareness for problems that were previously passed over and they are producing concrete techniques to identify and counteract bias and discrimination. At the same time, the problematizations that fuel this work are themselves circumscribed and limited to specific sets of issues. This has led commentators like Powles & Nissenbaum (2018) to argue that “the preoccupation with narrow computational puzzles distracts us from the far more important issue of the colossal asymmetry between societal cost and private gain in the rollout of automated systems”. Here, we are confronted with the distributed and layered character of normativity, making the question of which aspect or echelon to focus on a central part of value negotiation itself. Our goal is not to chart or adjudicate these discourses, but rather to highlight that technical fields like recommender systems mobilize normative reasoning on a number of different levels. Recent debates on AI ethics only capture the most visible parts of networks of normativity that reach deeply into all aspects of engineering practice. There is a need to investigate these less explicit layers, the conventions and “deep knowledge” that shape resulting technical systems in myriad ways, to gain a more comprehensive understanding of the full range of “ethical work” (Ziewitz, 2019, p. 713) that comes into play.

3.3 Technical experimentation

When dealing with technical artifacts, it is crucial to reserve space for technicity itself, to avoid a framing of technology as a mere epiphenomenon or transposition of cultural values. Here, we draw on Simondon’s (1958) understanding of technicity as having meaning independent of use, that is, functional meaning that signifies through operation itself. If there is indeed something like properly “cultural” values, we can investigate how they structure professional practice, inform design decisions, and provide frameworks for evaluation. But “machine behavior” (Rahwan et al., 2019) — i.e. what a system actually does when embedded within a concrete application setting — does not follow teleologically from human intentionality. In large recommender systems, the sheer mass of participants and items has the effect that outcomes are best understood as metastable arrangements that depend on technical design, available content, and actual use practices, all evolving over time. The behavior of the system emerges from these different inputs and knowing how it works on the level of code is not enough to understand how it “reacts” when in use.

Researchers may not be able to gain access to a platform like YouTube, and almost certainly not in ways that permit hands-on experimentation, but there are open source libraries that — combined with public datasets — allow for forms of “vivisection” that can yield important insights. Projects like LensKit⁵ or Facebook’s DLRM (Deep Learning Recommendation Model)⁶ may not be exact copies of real-world implementations, but they are part of the knowledge archives developers draw on and often used as models or starting points when creating new systems. For critical researchers, these libraries are prototypical examples of working systems, enabling a “hermeneutics of screwing around” (Ramsay, 2010) that can help develop intuitions for possibilities and limitations by playing with parameters, datasets, or optimization targets. Burell (2016), for example, tinkers productively with a simple spam filter based on a Support Vector Machine to support her argument that human and machine perception can differ in fundamental ways. LensKit for Python, “an open-source toolkit for building, researching, and learning about recommender systems” (Ekstrand, 2020), is particularly suited for similar research experiments, as it provides easy access to standard datasets and facilitates comparison between common algorithms. It implements many of the technical strategies and evaluation metrics discussed above, mapping that space of reasoning onto a modular technical artifact. While beyond the scope of this paper, technical experimentation can be a method for investigating how broader normative concerns are operationalized and materialized into technical form.

4.1 Ethnographic investigation

While workplace ethnographies have a long tradition and STS scholars have dedicated considerable attention to studying technical projects (e.g. Latour, 1992), research focusing on the practices and particularities of designing complex algorithmic systems are still rare. Seaver’s (2017) study of a recommender company, which included interviews with about 40 people, is an exception that showcases the great potential of ethnographic methods in this field. What stands out in this work is how distributed, collaborative, and ultimately elusive “the algorithm” is, even in a company dedicated to technical production. There is no single person that has a complete grasp of the precise workings of the system. Recent work by Ziewitz (2019) targets a very different kind of technical field, search engine optimization, but pays particular attention to the way practitioners negotiate and reflect on their own, often morally ambiguous practices. We would argue that normativity is embedded in broader and often implicit considerations and processes, but we agree that “focusing on the work of ‘being ethical’ opens up a novel way of looking at the politics of algorithmic systems” (p. 725) in the sense that practitioners are not disconnected from or oblivious to material circumstances. While not the only “source” we can draw on, they are central informants for reconstructing the underlying constellations, including their material dimension.

But as Christin (2020) argues in her discussion of different ethnographic strategies for studying algorithmic systems, “the question of access remains crucial and complicated for ethnographers studying algorithms, especially on the construction side”, since “[t]echnology companies and their engineering departments are deeply cautious and secretive, not only about ethnographers and academics but more broadly about all kinds of public discourse and reporting on their inner workings” (p. 913). We cannot count on companies like YouTube agreeing to anything close to the level of access Seaver (2017) was able to broker — although the work of Klonick (2018, 2021) on Facebook’s moderation practices and governance, which involved far-reaching access and peripheral participation, shows that openings do exist. But in most cases, former employees and whistleblowers are probably the closest one can get to genuine “internal” information.

What we learn from ethnographic work in related areas is that technical production comes with its own social complexities and, more often than not, bureaucracies and organizational idiosyncrasies. The different processes and negotiations that happen within design teams and the relationships with other organizational units are crucial to understanding how different factors — technical, economic, pragmatic, and properly “ethical” — are coming together. This includes broader ideological frames that are often left unexamined, for example the idea that more information is necessarily better or that every problem has a (technical) solution. While this points beyond what we are able to discuss in this paper, it should be clear that YouTube and similar companies are part of a larger “culture”, whether one wants to call it “internet culture” or “californian ideology” (Barbrook & Cameron, 1996).

4.2 First-party documentation and “algorithmic gossip”

Even if “traditional” ethnographic methods such as interviews or participant observation are off the table for many of the most interesting cases, there are other sources to draw on that can provide insights into local circumstances, including first-party publications, official statements, and reports, but also documents and depositions shared during court cases. The people working on complex algorithmic systems are often (former) academics and many continue to publish. In fact, as Ahmed & Waheed (2020) have shown, large internet firms have come to dominate AI conferences, the central publication outlets in the field. In the case of YouTube⁷, there are at least three papers (Davidson et al., 2010; Covington et al., 2016; Zhao et al., 2019) that provide substantial insight into the platform’s recommendation system, how it evolved over time, and what kind of normative reasoning it draws on. While a deeper hermeneutics of these papers would merit another article, even a short discussion can demonstrate the potential of such an exercise for an analysis sensitive to value articulation.

First, the overall framing — or problematization — the system subscribes to is largely in line with the standard literature in understanding recommendations as helping users discover “relevant” (Davidson et al., 2010, p. 293) or “high-utility” (Zhao et al., 2019, p. 44) items. Interestingly, more philosophical musings, such as the idea that personalized recommendations address an “unarticulated want” (Davidson et al., 2010, p. 293), are no longer present in the later papers. There is overall a clear process of specialization in the sense that the initial view of the whole system, including interface aspects, yields to a focus on specific sub-problems and optimization puzzles. Second, this process of narrowing and refinement continues on the level of chosen objectives. While “freshness”, i.e. the tendency to recommend recently uploaded videos, still features prominently in the 2016 paper, “diversity” disappears. But rather than a shift, this is more of a doubling down on the dynamic character of recommendations, which are no longer calculated in regular intervals, but when a user loads a page, increasing recency and making it possible to introduce “churn” in the sense that recommendations change when a page is reloaded (Covington et al., 2016, p. 6). Supporting viral spread becomes an explicit objective in this context. In the 2019 paper, the authors introduce the important distinction between “user engagement objectives” (e.g. clicking on video) and “user satisfaction objectives” (e.g. liking a video), which indirectly acknowledges the tensions between the consumer and provider viewpoints discussed above. A more complete analysis could indeed draw on texts like the Recommender Handbook to trace which desirable properties are not explicitly articulated. Third, similar kinds of normative structuring appear in the discussion of the user signals to take into account. The reference to “privacy” disappears after 2010, giving way to the “data voraciousness” suggested and enabled by deep learning, where basically any kind of user behavior can be added productively to the model. Hundreds of features are being taken into account this way and the initial idea that users should “understand why a video was recommended to them” (Davidson et al., 2010, p. 294) is dropped. Fourth, the question of signal choice is particularly salient when it comes to the evaluation metrics used to assess and optimize the system, and one can again observe an evolution in the target measures. A/B testing on the live website is mentioned in all publications, but the 2016 paper now emphasizes watch time over click-through rate to combat “clickbait” videos (Covington et al., 2016, p. 5). The 2019 split between engagement and satisfaction, both captured via specific metrics, confirms the realization that a unique focus on the former may have detrimental impact for the site. The attention given to measuring and mitigating “representation bias”, i.e. the tendency for highly recommended videos to accumulate more views and thus even more recommendations, again shows how the addressed issues are becoming more fine-grained (Zhao et al., 2019, p. 43). Here, we also notice the increased willingness to engineer “fixes” or “corrections” into the system, often driven by more nuanced modes of evaluation, resulting in a more complex architecture where the coordination of several independent modules generates the desired behavior. Fifth, these different considerations are constantly in conversation with the technological choices that translate desiderata into achievable features of the system. YouTube’s move from pure collaborative filtering to matrix factorization and further to deep learning is first and foremost a testament to the immense importance that scalability, speed, and processing time have for the platform. The split into a less signal- or data-intensive candidate-generation stage, where millions of possible videos are reduced to sets of hundreds, and a much richer ranking stage, where many more signals are taken into account for a personalized ranking of these candidates, becomes the backbone of a system that prioritizes recency and reactivity.⁸

Taken together, these five points show a tightly connected problem space where engineers are not only “improving” recommendations, but developing an increasingly sophisticated and specialized understanding of a modular system that depends on their own technical work as well as on the behavior of very large quantities of users. Discussions of watch time, recency, virality, and so forth point to ideas about quality and desirability that are hardly at odds with economic success, even if the identification of risk and pitfalls (e.g. clickbait) leads to more nuanced apprehensions. Normative choices are operationalized and spread over a growing number of metrics that are discussed and balanced as “trade-offs”. The cost of this specialization and modularization may well be reduced comprehension of the overall workings — and thus control — of the system, for both users and the engineers themselves.

A comprehensive analysis of first party statements would include not only academic publications but also official policy documents, technical documentation, media interviews, or financial reports as material to be analyzed. In line with the logic of “encircling”, special attention should be paid to what is missing from these documents, how secrecy is operationalized and possibly changes in scope over time. For example, YouTube used to provide some basic insight into the main parameters of their recommendation engine (Google, n.a. a), before removing the page in 2015 and falling silent on the topic. While company statements need to be read with critical distance, they can reveal characteristics of working systems — or at least public justifications for design decisions. Facebook, for example, released a set of “recommendation guidelines” in 2020 (Fabeook, n.a.), which single out five types of content that “are allowed on our platforms, but that may not be eligible for recommendations”. YouTube (2019a) follows a similar logic, even if policies are less detailed when it comes to defining “problematic” content. Here we learn that while the company has long deleted videos or channels that violate content policies, recommendation has become an important lever for acting on “content that comes right up to the line” (Youtube, 2019b). This includes limiting “recommendations of borderline content and harmful misinformation, such as videos promoting a phony miracle cure for a serious illness, or claiming the earth is flat”. While these policies and their consequences are communicated in (strategically) vague language (“watch time that this type of content gets from recommendations has dropped by over 50% in the U.S”), they indicate a general shift towards explicit editorial intervention in systems that used to take pride in pristine automaticity. Here, we see how companies (sometimes) revise their value priorities in response to rising external pressure, which flags yet another area for future research, namely the question how far governmental scrutiny and critical commentary from civil society affect business practices and systems design.

Beyond first-party statements, we find another stream of material to study in absence of broad access, which comes from industry observers, consultants, and, most importantly, from participants that are directly affected and develop their own “folk theories of algorithmic recommendations” (Siles et al., 2020). In the case of YouTube, there are websites like Search Engine Journal⁹ that aggregate and document the distributed knowledge production happening in the industry; analytics and marketing companies publish guides on how to use the platform’s algorithms for growing one’s audience (e.g. Cooper, 2021); and creators themselves continuously accumulate and share both deep practical knowledge of how recommender systems function and tactics for bending them to their own advantage. While this “algorithmic gossip” — “collaborative and directive processes used to formulate and sustain algorithmic expertise” (Bishop, 2019, p. 2589) — presents interpretative challenges, it can help generate singular insights into mechanisms and their effects, to the point of functioning “as a tool for exposing platform discrimination and bias” (Bishop, 2019, p. 2603). In her work on YouTube vloggers, Bishop not only shows how creators perceive a not-so-subtle preference for beauty content in YouTube’s recommender systems, but also highlights the severe lack of communication and accountability when it comes to explaining how these systems work. While not the emphasis of our paper, this points towards the important question of how design choices and other corporate practices affect the different user groups gathering around algorithmically mediated platforms.

4.3 Business practices and legal context

A third approach to understanding a concrete recommender system in terms of value commitments and effects centers on the question of what or who it is working for. In the context of for-profit companies, this involves an analysis of the organization that designs and maintains the system, in particular its business model and the incentive structures that model generates (van Dijck et al., 2018). We certainly accept that the “bottom line” is not the only motivation for a firm and there are many ways to drive and develop sources of revenue; YouTube’s boardrooms and legal offices are indubitably ripe with debate about how to react to external demands, for example. But it should be clear that private companies are mainly driven by a profit motive that exercises constant pressure on all design decisions. For YouTube, this means starting with the main source of revenue, advertising, which amounted to $15B in 2019, when Google broke out numbers for their video-sharing site for the first time (Statt, 2020). On the one hand, advertising pushes towards a logic of “more” — more viewers, watch time, engagement, etc. — and the revelation that 70% of traffic is steered by AI-based recommendations (Solsman, 2018) emphasizes the tight connection between these systems and economic success. The move toward “captivation metrics” (Seaver, 2019) that seek to tease users into staying on site is thus not surprising. On the other hand, recent years have seen multiple moments of public outrage and flashes in what has come to be known as the “Adpocalypse”, where advertisers retreated from YouTube, forcing the company to adapt both policy and technology (Caplan & Gillespie, 2020; Kumar, 2019). The much greater willingness to make editorial decisions and the turn to more nuanced evaluation metrics are two direct reactions to this more complicated situation. Ranking, recommendation, and how these systems are communicated are indeed part of a balancing act that has to cater to different constituencies, including users, creators, advertisers, and regulators:

Intermediaries like YouTube must present themselves strategically to each of these audiences, carve out a role and a set of expectations that is acceptable to each and also serves their own financial interests, while resolving or at least eliding the contradictions between them. (Gillespie, 2010, p. 353)

A critical analysis of the contradictory forces at play and the dynamics they produce will again not be able to produce definitive answers but serves to create an analytical scaffolding that connects the various other aspects coming into play to the broad imperative to generate profit.

An important part of a more theoretically informed analysis, here, is the realization that corporate practice unfolds within negotiation structures that are shaped by many different kinds of law. Competition law has recently received much interest in both Europe and the US, as commentators have argued for new yardsticks beyond effects on consumer prices (Khan, 2018). Labor relations are crucial for managing a workforce, in particular the army of low-paid moderators that sift through content at social media companies. Rules for data collection and ownership directly affect what signals a recommender system can draw on. Transparency requirements may force a company like YouTube not only to divulge specifications, but to change the way the system works in order to make it “communicable”. Telecom regulation on issues related to net neutrality may affect bandwidth prices and the much-coveted speed at which recommendations can be served. These are just some examples from a panoply of rights and obligations that circumscribe companies’ capacities to negotiate their participation within a market. When we ask what a system is “working for”, we must thus add the question what it is “working with”, that is, which background conditions shape its space of action. This includes a temporal component, as platforms like YouTube have been able to secure their position over time and under changing conditions, for example, drawing on personal data that is no longer legally available or issuing patents covering core components of their technical systems into the future.

Authors like Cohen (2019) have analyzed these complicated relationships between legal institutions and information technology in great depth, but even a more limited exploration can point toward instances where legislation has affected recommender systems very directly. Section 230 of the U.S. DMCA, in effect since 1996, and similar “safe harbor provisions” in other legislations relieve “interactive computer services” from being liable for what users are sharing or posting on their services and explicitly allow for much leeway in content moderation, which recommendation is an increasingly important part of. More restrictive legal provisions, however, put limits on what data is collected and how it is made actionable. The U.S. Children’s Online Privacy Protection Act (COPPA), for example, has led YouTube to introduce its “made for kids” feature, which excludes channels and videos targeting younger audiences from a number of features that rely on data collection. Recommendation is affected in the sense that videos made for kids are “more likely” to appear next to other such videos (Google, n.a. b). This means that once within the “kids sphere” users are no longer pulled into broader YouTube through recommendations. Here, the platform is effectively “deputized” by a regulator to enforce action against unwanted user-content encounters.

5.1 Interface analysis

In recent years, the notion of “affordance” (Gibson, 1986) has come to play a central role in the conceptualization and analysis of how material properties of objects and environments provide and suggest possibilities for action, linking them to human behavior. While the term has been used in design discourse to discuss artifacts and design decisions in great detail, more recent takes have emphasized “higher-level” affordances, such as the broader social and communicative possibilities offered by social media services (Bucher & Helmond, 2018). Postigo (2016, p. 1), for example, uses the concept to inquire how “technological features designed into YouTube create a set of probable uses/meanings/practices for users while serving YouTube’s business interests”, connecting specific features and properties of the platform to its central goal of revenue generation. In practical terms, the analysis focuses on the interface and the various functionalities that allow for “distribution of video, advertising, communication between commentators and subscribers, subscriber recruitment and retention, and community participation” (Postigo, 2016, p. 13), creating an “architecture of digital labor” (p. 9) that extracts value from user practices. Approaches like “discursive interface analysis” (Stanfill, 2015), which seeks to examine “the assumptions built into interfaces as the normative or ‘correct’ or path of least resistance” (p. 1060), or the “walkthrough method” (Light et al., 2018) have attempted to organize similar forms of analysis into more structured methodologies. In both cases, the goal is to scrutinize the interfaces of apps and websites to understand how norms are constructed and user behavior is guided and shaped.

While these approaches do not focus on AI components, they draw our attention to the crucial question of how a recommender system is linked to use practices in functional terms. For YouTube, this means first of all localizing instances of recommendation in the interface, which is less straightforward than it may seem. While sections on the “home” and “trending” pages are clearly marked, it is less easy to understand which part of the “up next” column and even the search function are actually personalized to the user. We may also want to look at the leeway users have to configure, manage, or even circumvent the system. In line with previous observations, we notice that YouTube has hardly any way for the user to adapt the interface or tweak recommendations. For example, it is currently not possible to replace the heavily processed “Home” page with the “Subscriptions” page as default and giving feedback on recommendations is limited to a “not interested” button for individual videos, which can hardly be seen as a comprehensive and transparent form of steering the system. Note that affordances concerning interface components are relevant as they directly relate to users’ actions on the system, and then to the production/transformation of value in different forms depending on the various stakeholders coming together around the platform (e.g. video creators, advertisers, etc.). These outer layers can and should be included in a more general study of (behavioral) affordances enabled by the system.

5.2 Data-driven observation

The second direction for studying materialized values draws on the idea that even proprietary systems are at least somewhat “observable” (Rieder & Hofmann, 2020) if their outputs can be scrutinized. Such “scraping audits” (Sandvig et al., 2014) can yield insights into the workings of AI components, even if a number of caveats apply. Other than the complexity of the underlying technical architecture, which may include many modules working together, and possible attempts to thwart data collection, the difficulty to “nail down” the mechanisms at work comes from the already mentioned recognition that “[m]achine behavior […] cannot be fully understood without the integrated study of algorithms and the social environments in which algorithms operate” (Rahwan et al., 2018, p. 477). The behavior of YouTube’s recommender system depends not only on purpose-driven and value-laden engineering feats but also on the practices of billions of users that upload, watch, like, comment, and so forth. How the system acts, what it recommends, how it evolves over time, and so forth, are emergent properties, especially when machine learning techniques are used. While the company’s employees, crucially, define optimization targets and reward functions, actual outputs are not fully determined by these decisions, making value-focused analysis less straightforward than in the case of interface analysis. The presence of radical political content in video recommendations, a perpetual point of contention (Ingram, 2020), may variably be read as an indicator of YouTube’s ideological preferences or as an effect of an editorial laissez-faire approach combined with the higher engagement propensity of divisive content. The question, then, is whose value commitments are at work here, a question that is exceedingly difficult to answer.

But if we approach the question from the perspective of performativity, we can decide to leave distributed agency entangled and focus on the recommender system itself as normative agent, examining its behavior in terms of normativity and decision-making. What is actually being recommended? Can we observe trends, for example, a preference for popular content or a tendency to drive users toward niches? How do recommendations evolve over time? A number of recent studies (e.g. Ribeiro et al., 2019; Alfano et al., 2020; Matamoros‐Fernández et al., 2021) have analyzed recommended videos in this way, generally finding evidence in support of the radicalization hypothesis. Rieder et al. (2018), for example, focused on YouTube’s search function and found oscillations between two ranking “modes”, one prioritizing recency, where often divisive YouTube-native content dominates, and one that gives prominence to less polarized perspectives and more traditional media voices. While these forms of analysis come with their own caveats and are often difficult to implement, they allow us to investigate normativity as performativity, that is, as “values in action” rather than idealized input into design decisions. At the same time, we can conjecture that YouTube’s engineers are well aware of how their system broadly behaves, which means that there is intentionality behind what we can observe.