This study aimed to evaluate the emotional responses of Amazon Prime Video users after they encounter paid content and ads on the platform. This study was motivated by dissociations identified between the designers' intended user experience, which states that Prime subscribers would be given full ad-free access to the platform, and the actual user experience where users interact with in-app purchases and ads. The research question and hypothesis that guided this study focused on what extent the presence of ads and paid content affected the emotions of Prime subscribers. I completed this case study with 3 colleagues of mine - Akinola, Oluwatosin, Oluwatomi, on May 2024.
To carry out this study, a heuristic and usability test was carried out on Amazon Prime Video, where a multi-method approach consisting of quantitative and qualitative methods was applied. A pilot study was conducted where methods including questionnaires, affective walkthrough sessions of the Prime Video platform, and semi-structured interviews were employed to gather data on participants' emotional state. While there were limitations to the study like a small sample size and issues with retrieving data from eye-tracking software used during affective walkthrough, this study still revealed patterns supporting the hypothesis that users experience negative emotional responses when interacting with paid content and ads on Amazon Prime Video. Future studies to be carried out on Prime Video can consider testing with a larger and more diverse sample of users to gain more comprehensive insights.
Previously known as Amazon Instant Video, Amazon’s Prime Video is a subscription-based video-on-demand (SVOD) service by Amazon, with a large library of content, that includes movies, TV shows, live TV, and sports. It is a platform that serves as the video streaming offering of the Amazon Prime subscription, where users have access to stream movies and TV Shows. Amazon Prime itself is a paid membership subscription introduced by Amazon in 2005. This membership subscription offers services such as Prime ‘Next day' Delivery, Ad-free music streaming with Prime Music, Prime Reading, unlimited photo storage with Prime Photos, and Prime Video which is the focus of this study. Prime Video on its own is also available for subscription for just £5.99 per month, however, users won’t get the other benefits that come with Prime.
Amazon Prime Video offers a multitude of key features and functions to enhance users' streaming experience, some of the key aspects are;
Amazon Prime Video intends to provide users who have subscribed to its membership with access to a large catalogue of movies and shows. Still, this subscription is rendered redundant with the implementation of in-app purchases, which were not initially communicated with the users, such as prime members having to pay for an ads-free feature on movies, as seen in figure c below and having to pay to “rent” or “buy” movies as seen in figure d below.
Prime Video users are likely to experience feelings of frustration, disappointment, sadness, and even shame. This is based on the situation being appraised as unexpected stimuli that interrupt their original goal of watching a movie or show in this case. Quotes like "They duped us" and "I might as well just use Netflix" are likely to come from them, as seen in Figure e below, after they were promised full access to Prime Video after signing up for Amazon Prime and then being asked to pay even more money to eliminate ads.
Users are left with no option but to have 2 or 3 recurring subscriptions that were not initially planned just to watch their favourite show without interruption. They are left asking questions like “Do I cancel my Prime subscription?” but, then they lose access to other Amazon product offerings like Amazon Music and next-day deliveries.
Users are almost driven into a corner with this dark pattern and have no choice but to pay the extra fees for premium Prime Video service.
We have identified the dissociations in the Prime Video app to be the presence of In-app purchases in an offering that has already been financially subscribed to and the presence of ad interruption in movies and shows is bound to disrupt their cognitive process at the time of the interruption. Research (Sparrow & Knight, 2006) carried out in neuroscience has shown that the emotional brain and cognitive brain are linked to each other, and a person cannot memorize, reason, learn, and act without a working emotional brain. We can argue that these interruptions and gaps in cognition will lead to emotional responses by the users.
Given such a research purpose, our research question was:
To what extent does the presence of ads and paid content in Amazon Prime Video affect Prime subscribers emotionally?
Based on the personal experiences of the team, the reviews on social media platforms as seen in Figure e above, and research carried out on cognitive interruptions (Giakoni et al.,2022; Lewandowska et al., 2022), we hypothesized that:
The presence of paid content in Amazon Prime Video will have a negative emotional impact on users, making them alienate Prime Video while using other product offerings under Prime subscription.
These are the variables that we manipulate to see their effects on the dependent variables.
They are the outcomes we are interested in measuring.
These variables are to be kept constant or monitored to minimize their impact on the outcome of the study. They also act as a shield against our confounding variables.
They are external factors that can skew the results of the study if not identified and controlled properly.
Norman (1986) explained the gulf of execution and evaluation to represent the distance between the psychological goals of a user (e.g. I want to listen to music in my car while driving) and the actions necessary to be taken with a specific system to achieve those goals(e.g. Press the power button on your radio to turn it on).
The gulf of execution is measured by how well the action possibilities of the system match the intended actions of the user; e.g. Does my car radio have a power button that works by pulling the button instead of pushing?
The gulf of evaluation is measured by how well the system provides information about its state in a form that is easy to get, is easy to interpret, and matches the way the person’s mental model of the system (Norman, 2013); e.g. Does the car radio have a ‘pull’ tag on the button? Does the car radio respond to the pushing of the radio button, even if it is wrong, giving the user a prompt to pull?
We have identified a few gulfs while interacting with Prime Video and listed them below.
The circumplex model, specifically the interpersonal circumplex, is a framework that has been used in usability studies to analyse and describe the interpersonal dynamics between designers and participants during usability testing (Li et al., 2022). This model defines interpersonal behaviour and relationships along two orthogonal axes - a vertical axis representing dominance/control and a horizontal axis representing affiliation/warmth.
To ensure unbiased application of the model, we split the users into ‘New’ and ‘Experienced’ and formulated a task for each. We utilized the circumplex model differently by analyzing the designer’s intended experience for the user and the experience the user went through from both their perspectives. Rather than having one definite user journey and analyzing it from both the designers' and users’ perspectives, we highlighted the tasks the designer intends for the user to achieve their goal, and then we highlighted the actual user experience in achieving that same goal. The figures below show how both perspectives were mapped out and the assumed emotional responses at each point of the journey.
Task: Sign up for Amazon Prime membership and browse through content offers.
Task: Subscribe for ad-free content and access all movies in the catalogue.
Our mapped circumplex model was able to highlight the emotional gulf between the designers and users of Prime Video, with a lot of the emotions in the users' experience argued to match those of the Low valence-high arousal section of the circumplex model i.e. emotions like anger, irritation, fear. These have been researched to result from negatively appraised situations and will in turn lead to a worse-than-intended user experience.
In line with our research question and hypothesis, we decided to test for the presence of these negative emotional responses in the user journey. The next sub-section highlights how we use user research methods to observe these measures and test our hypothesis.
The global measures we decided to focus on are the range of emotions that fall into the Low Valence-High arousal quadrant of the circumplex model. These emotions include Anxiety, anger, fear, and frustration and they would be studied as sub-measures (i.e. we would test for any of these emotions). For this study, we will be examining the instances where any of these emotions are experienced by participants during their interactions with the Prime Video Platform.
The reason behind choosing these emotions as key measures is that when we carried out a preliminary analysis of the gulfs present in the Amazon Prime Video user experience, we found that the user experience when interacting with Amazon Prime Video evoked emotional responses that aligned with the fourth quadrant of the circumplex model.
By focusing on the range of emotions in this quadrant, we aim to gain a deeper understanding of the emotional responses or challenges users encounter while interacting with the Amazon Prime Video Platform.
For this study, the team decided the following methods: Affective-Cognitive Walkthrough, Questionnaires, and Interviews, would be the most appropriate for measuring the range of negative emotions observed in users when interacting with Amazon Prime Video.
To recruit participants, we randomly reached out to another group in the same master's program as ours, agreeing on a date and time for a pilot study. This group comprises 3 members,1 female and 2 males who were all within the age range of 20-35 years old.
The following equipment were utilized to conduct this pilot study;
The major materials used to carry out the study were;
While these are the results of a small sample size of just three (3) participants and limited data due to technical issues encountered with the eye tracker, we were still able to identify certain patterns and trends and propose a conclusion that addresses our hypothesis.
We used the information obtained by our lead observer and the results obtained from pre-study questionnaires to obtain the user's emotional state to get their baselines and isolate the effects of our independent variable on our measures.
Our Pre-questionnaire results allowed us to get details of our participants as quantitative data represented as tables and figures as seen below in Figure j and Figure k. We then used the mean measure of the central tendency to understand the average time spent, average streaming platform comfortability rate and average age of our participants, and we used the measures of variability in variance and standard deviation to see how variable these figures are.
The variability of our data highlights two main user demographics, and we were able to identify a correlation between their ages and frequency of use of streaming platforms. This allowed us to identify two personas to accurately represent the demographics of our participants and possibly identify future trends and patterns when analyzing the qualitative data.
The correlation scatter graph between the ages of our participants and their frequency of use of streaming platforms, although a small sample size, also shows two main demographics of participants and the patterns between them (e.g. the participants above the age of 30 use streaming platforms less regularly than those aged 30 and below), assisting us making sense of the demographic data got.
The technical issues experienced with the eye-tracking stimuli and screen recordings made it impossible for our team to perform a complete analysis of the data obtained from the emotion recognition software. Figures n and o below show the difference between what our data was supposed to look like versus what we got.
Although a major issue, we decided to use the facial emotion recognition data we got using the signals from the emotions we were measuring. The chart below displays the maximum spike point for our range of emotions between the three participants during their interaction with Prime Video. This data was paired with the thematic analysis data obtained from the post-study interview to produce the conclusions we discuss in the next section.
With the information transcribed from the semi-structured interviews with the participant and our interviewer, we used a thematic analysis approach to identify the major and common themes noticed among the participants and presented them in the form of Figure q below.
Our results display multiple trends and patterns we have highlighted.
Through our pilot study, we were able to delve deeper into our participants' experiences. Although we encountered some issues with the eye-tracking data and inconsistent display of ads, we were still able to gather valuable insights that allowed us to support our hypothesis.
We are recommending the following changes in the Prime Video and Amazon Prime platforms to improve the overall usability and bridge the dissociation between users and designers.
The goal of this study was to gain practical experience in applied usability and to learn how to effectively design and conduct a usability study. Our team aimed to develop an understanding of the emotions experienced by users when they interact with the Amazon Prime Video platform.
This study was overall a valuable learning experience for us, we were able to put all the theoretical knowledge into a hands-on project. We aimed to gain the skills to conduct a good usability study and through the stages of identifying a usability issue, analyzing the intended and actual user experience, and formulating and testing our hypothesis, we were able to do this.
While we faced some technical challenges in our study and worked with a limited number of participants, we were still able to learn how to draw valuable data that validated our hypothesis. In the future, we will apply the lessons learned from this study to create optimal usability studies.
-p-800%201-min%20(1).png)
