How Content Creators Craft Algorithmic Personas

TL;DR

We engaged with YouTube content creators to understand how they make sense of the YouTube algorithm. We discovered that YouTube content creators craft three main algorithmic personas to navigate the algorithm: Agent, Gatekeeper, & Drug Dealer. For YouTubers, these personas help facilitate and augment their discussions in the process of collective sense-making of the algorithm and affect what and how they post. Through algorithmic personas, we can enrich our understanding of algorithms and their impact in the real world.

Background

I conducted this research during my Master’s year at UC Berkeley. I worked with Assistant Professor Niloufar Salehi from the UC Berkeley School of Information (iSchool) and Eva Wu, a Master’s student at the iSchool. Our paper was accepted to CSCW ’19 and published in the Proceedings of the ACM on Human Computer Interaction.

Supplementary materials: Here is our presentation for CSCW ‘19 and blog post published in Medium by ACM CSCW about our research. Please feel free to take a look our interview script, and wiki surveys.

Problem Statement

Let me start off by providing some context about YouTube and the YouTube algorithm. Around 400 hours of video are uploaded to YouTube every minute — or 65 years of video per day. Isn’t that insane? To deal with the large amount of content on the platform, YouTube deploys algorithms to customize users’ video feeds, provide content recommendations and return search results. YouTube now has over 1.8 billion users every month. And that’s just people who are logged in.

Increasingly, online content creators have to navigate opaque, unpredictable, and proprietary recommendation algorithms that mediate the worker-employer relationship. These algorithms largely decide what content to promote and what to take down. 

We focused on YouTube because of its widespread use, content creators’ (YouTubers’) dependence on a real-world algorithm that impacts their creative work, and the spaces for collective sense-making and mutual aid that YouTubers have built within the last decade.

Research Questions

In our research we wanted to understand:

• How do YouTube content creators make sense of an algorithm that manages their creative work?

• How do perceptions of the algorithm affect their attitudes and actions?

• If they could, what would content creators change about the algorithm?

Research Methodology

Timeline

We worked on this research project for about a year. Here is an overview of our timeline:

• Brainstorm research questions: 1 month

• Literature review: 1 month

• Recruit participants: 2 weeks

• Conduct first round of interviews: 2 weeks

• Reflect on findings, refine 2nd round interview questions, and create paper prototypes: 2.5 weeks

• Recruit participants: 1 week

• Conduct second round of interviews (engage participants with paper prototypes): 2 weeks

• Reflect on findings, refine 3rd round interview questions, and iterate on prototypes: 1.5 weeks

• Recruit participants: 1 week

• Conduct third round of interviews (engage participants with paper prototypes and card sorting exercise): 2 weeks

• Conduct analyses of native formats of online information sharing: 2 weeks

• Informal share out of findings with team: 1.5 weeks

• Formal open coding/thematic analysis of interview results and native formats of online information sharing data: 1.5 months

• Create wiki surveys: 1 week

• Share wiki surveys on online forums/collect responses: 1 month

• Analyze wiki surveys results: 1.5 weeks

• Write final paper/create final presentation: 3 months

RecruitMent Process and Criteria

We found participants in three ways: First, our personal connections (5 participants). Second, we searched YouTube for content related to our university and reached out to the content creators (3 participants). Third, we posted a notice on our university’s various Facebook pages. We recruited one person from our Facebook post and reimbursed them with $10 for their time. We interviewed a total of 9 people (6 male, 3 female; 3 White/Caucasian, 3 Asian, 2 South Asian, 1 Hispanic; aged 18 to 30, M = 21). At the time of conducting our research, our interview participants had an average of 5 years creating YouTube videos and ranged from posting content weekly to yearly on their channels, and had between 56 to 257,000 subscribers (average=38,100, median=4,950). Most of our participants primarily make “lifestyle videos” with one participant making music videos.

We chose to focus on hobbyist YouTubers, which we defined as those actively producing content for YouTube and who have fewer than 1 million subscribers. This covers the majority of YouTube content creators.

Study Design

We gathered data in two ways: directly and indirectly. Our direct methods allowed us to probe deeper into content creators perceptions and behaviors around the algorithm. Our indirect methods allowed us to go to where they gather organically and observe unfiltered discussions of how they navigate the algorithm. This triangulation enabled us to verify our findings with more confidence. Let’s first discuss our direct methods.

1. Direct Methods

We began our study by conducting interviews with local YouTubers like Oh No Nina (Figure 1) who was a UC Berkeley student with around 300,000 subscribers at the time, and makes fashion and lifestyle videos. Nina now has close to 1 million subscribers and makes a living off the platform. We conducted 3 rounds of interviews, interviewing a total of 9 YouTubers. We used this interview guide. Our interviews enabled us to establish a basic understanding of content creators’ attitudes towards the YouTube algorithm. In our interviews, we engaged our participants through card sorting and speculative design exercises. Participants sorted features that the YouTube algorithm cares about by speculating their importance (e.g. thumbnail, click through rate, length of video, etc.) and added features they believed to be important that we had missed. The card sorting exercise proved to be an effective ice breaker. It met participants at the level that they usually are when discussing the algorithm and eased them into a more complicated and abstract discussion of the algorithm’s goals and behavior and toward imagining alternatives.

In the absence of a physical representation of an algorithm to discuss and manipulate, we created physical representations of algorithmic effects, design provocations. For example, after our first round of interviews, we learned from our participants that they think the algorithm promotes dumb and senseless content. Thus, we created a “smart mode” mock-up (Figure 2) in which users can switch between smart and dumb modes of content recommendation. We also created a “Diversify” prototype (Figure 3) because interviewees expressed the wish to escape the rabbit hole effect of the algorithm. These are just a couple of the design provocations we used. I want to note that we iteratively created mock-ups of alternative YouTube homepages and recommendation playlists that addressed issues our interviewees had brought up. These mock-ups are close to the way people experience algorithms in the real world. The goal was not for us to create the best possible YouTube algorithm, but to use design as provocation to elicit reactions from our interviewees. We used our designs to prompt participants to imagine a different YouTube and learn how they form understandings of how an algorithm operates and affects them.

Towards the end of our study, we created wiki surveys to understand how our findings would resonate with a larger group. We posted the wiki surveys on online forums such as YouTube subreddits and YouTube creator Facebook groups.

2. Indirect Methods

Our second source of data was native formats of online information sharing. We watched videos of YouTubers talking about the algorithm, mostly with the goal to explain the algorithm to other creators. We found these videos by searching “YouTube Algorithm”, “YouTube algorithm explained”, “YouTube algorithm hack”, “YouTube algorithm rant” on YouTube. We also reviewed information available online about VidCon, the major convention for YouTubers, and reviewed YouTuber forums, such as yttalk.com and YouTube subreddits.

Findings

Thematic Analysis

We analyzed our data through open coding and categorization, using Dedoose. Through data analysis, one theme emerged as the most prevalent and surprising. 

Algorithmic Personas

We found that content creators make sense of the algorithm by assigning human characteristics and goals to the algorithm to explain its behaviors, in what we have termed as algorithmic personas. Our research provides evidence that content creators make sense of the YouTube algorithm through three personas: Agent, Gatekeeper, and Drug Dealer. Content creators invoke characteristics of these personas when managing their relations with the algorithm, rationalizing algorithmic outcomes, deciding on courses of action, and engaging in conversations with other content creators. YouTubers used these personas interchangeably, so someone might speak of the algorithm as an agent in one context and as drug dealer in another. These personas are helpful conceptual tools, but are not completely distinct and often overlap. 

1. Algorithm as Agent

First, we have Algorithm as Agent. The most distinctive characteristic of the Algorithm as Agent is that is perceived as scanning, choosing, and promoting individual people’s channels. Algorithm as Agent is similar to a talent agent in the entertainment industry. Content creators are the talent and the algorithm is the talent agent. For example, one YouTuber invoked the Algorithm as Agent by saying:

“YouTube will favor you in the algorithm which would then lead to more views and more subscribers”.

This quote illustrates that when an agent-talent relationship is successful, the agent supports the talent by providing them with coveted gigs and the means to grow. 

2. Algorithm as Gatekeeper

Second, we have Algorithm as Gatekeeper. The most salient theme in this relationship was the power imbalance, with YouTubers feeling themselves at the whim of the Gatekeeper.  The real-world analogs are college admission officers or bouncers at a club.  Content creators are people trying to get into college or the club. One interviewee said:

“There is [an] algorithm between you and the viewers. You need to try to understand the algorithm and play to its strengths, or kinda get lucky.”

3. Algorithm as Drug Dealer

The first two personas focus on the algorithm’s relationship with the content creator. This third persona was one of the ways YouTubers make sense of the worst aspects of the algorithm’s behavior. The Algorithm as Drug Dealer has one nefarious goal: to make viewers addicted to YouTube so that they stay on the platform for as long as possible. The algorithm is the "drug dealer,” and the viewers are the “drug buyers.” One YouTuber said:

“The algorithm is really good at keeping us here.” 

This is a table of algorithmic persona descriptions (Table 2) gathered from our wiki survey about what role the algorithm should play. Our wiki survey received 572 total votes, and 43 unique voters. We seeded the poll with 6 themes from our field work and the participants added 6 new ones, which are the starred descriptions. The analysis process uses responses to construct an opinion matrix, and summarizes that matrix to calculate the probability that any one response would be chosen over a randomly chosen option. I’ll just call out a couple of ones added by the participants that we found interesting. For Agent: “A regulator who makes sure people don’t grow too quickly on the platform”; Gatekeeper: “A curator that decides what will and will not be seen by the viewers”; Drug Dealer: “A strategist for increasing user engagement with the platform.”

How Personas Shape Creators’ Behaviors

In our research, we also sought to understand how these personas affected what YouTubers did on the platform and how they factored the personas into their decisions. Sometimes, YouTubers oriented themselves toward the personas to achieve their goals, however, sometimes they acted against them. 

1. Algorithm as Agent

When invoking the Algorithm as Agent, YouTubers used phrases such as blessed by, build a relationship with, to please, and work with to explain how they interact with this algorithmic persona. For example, a YouTuber recognizes the importance of building a friendship with the algorithm as Agent, saying:

“You wanna be friends with the YouTube algorithm which decides to push your video or not.”

This quote illustrates the perennial dance in “being friendly” vs. “being friends” with the talent agent.

2. Algorithm as Gatekeeper

From the perspectives of the gatekeeper persona, YouTubers described actions such as to bribe, circumvent the gatekeeper, or to fit in. One YouTuber said:

“I ended up getting a lot of views because I actually piggybacked a very popular trend at the time”

3. Algorithm as Drug Dealer

Perceiving Algorithm as Drug Dealer led YouTubers to either rebel against the drug dealer or become complicit in the drug dealing, sometimes becoming unwillingly addicted themselves as a viewer. One interviewee said:

"My model is slow disperse growth. Trying to go the other way against the click-bait, viral algorithm. My goal is to not to follow that method. It is dangerous path — it’s luck.”

This participant worked against this persona by producing content he/she knew would be not be favored by the algorithm but would instead be true to his/her morals and creative integrity.

Algorithmic Wishes

In our interviews, we engaged with YouTubers about how they would want the algorithm to change. Again, we found YouTubers invoked algorithmic personas as a way to describe their preferred roles for the algorithm. Participants described their wishes in the shapes of Advocator, Custodian, Diversifier, Educator, Impartial Judge, and Revenue Sharer personas. The Advocator would promote smaller channels, the Custodian would take a role to moderate content, and the Diversifier to get people out of YouTube rabbit holes, the Educator would promote more meaningful and productive content, the Impartial Judge would provide explanations about algorithmic decisions, and lastly the Revenue Sharer would advocate for higher share of revenue to go back to the content creator. This last wish was more broadly about business model, showing that participants sometimes referred to the algorithm, the platform, and the company interchangeably. 

Summary

To summarize our findings, below is a table (Table 1) listing the 3 personas, what the personas describe, how people behaved when they invoked it, and what role they wished the algorithm played instead. 

Impact

Now that we learned that content creators personify the algorithm in these ways, what is the larger impact?

1. Typically, when we talk about algorithms, very abstract images of code and networks come to mind.  But with this conceptual framework, we can easily talk about algorithms, and enrich our understanding of algorithms and their impacts. 

2. Algorithmic personas use roles that we are familiar with, which we can leverage when talking about algorithms’ roles in socio-technical contexts. 

3. We can use these personas to develop our understanding of algorithmic power relations and potential accountability mechanisms. 

For example, what would it look like for Algorithm as Agent to procure a professional license in its role of locating employment opportunities for talents? What if content creators could ask the Algorithm as Gatekeeper to explain why their video got demonetized? How can we address the public health concerns of the Algorithm as Drug Dealer? The shapes of those explanations might be open questions that algorithmic personas could provide insight to.

Through the lens of the algorithmic personas crafted by YouTube content creators, we can enrich our understanding of algorithms and their impact in the real world. Algorithmic personas, invoking human characteristics in code-base algorithmic artifacts, enable designers and policy makers to design for human-to-computer systems with human-to-human relations as guides.