Ranked Play and Positive Systems
DESIGNING FOR DIGITAL THRIVING IN ONLINE GAMING ENVIRONMENT
OPENIDEO’S DESIGN CHALLENGE
In the fall of 2022, OpenIDEO hosted a design challenge on digital thriving, with co-sponsors including Riot Games, Fair Play Alliance, Sesame Workshops, and IDEO. Our Informatics class at the University of California Irvine used OpenIDEO’s design question as our inspiration for UX research & design:
“How might we design healthy, inclusive digital spaces that enable individuals and communities to thrive?”
Challenge Theme 1
Welcoming and Supporting Underrepresented Voices
Challenge Theme 2
Fostering Healthy and Resilient Digital Space
Challenge Theme 3
Building a Digital Future for Youth, Kids, and Families
PROJECT OVERVIEW
Scroll down to explore how our team used UX research and design methods to iteratively explore the design challenge. Throughout, our team used the Design Thinking Process to create a third-party application with the goal of rewarding players who consistently communicate positively in games. It can be added to team-based, online multiplayer games by game developers, or run concurrently as a stand-alone application. The application runs a sentiment analysis on the user's recorded communication at the conclusion of each match. An honor score and honor points are then given to the user based on the results of that sentiment analysis. This system can incentivize gamers to support less experienced players and create a collaborative environment for all to play.
About the Project
Team Members
- Di-Yun Cheng (myself)
- Conor Timothy Murphy
- Victoria Winn
- Chase Katsuji Hashimoto
- Gurpreet Singh
- Doyun Kim
Project Duration
- January - March 2023 (3 Months)
Tools Used
- Miro, Google Workspace, Figma, Google Meet, Zoom, Storyset
THE CURRENT CONTEXT
Currently, many gamers play online games as a way to cope with their stress, socialize with friends, or simply enjoy the gaming experience. However, the
growth of toxicity in games has been an underlying problem that many gamers have faced in an online competitive gaming environment, causing them to be frustrated, furious, and irritated. These
"antisocial actions" known under the broader category of "
disruptive behavior and harmful conduct", are so embedded in gaming culture that many gamers believe it's impossible to find a solution. Although there is no one magical solution to this critical problem in the gaming world, our team wants to help gamers digitally thrive by
reducing the occurrence of antisocial actions in games.
OUR VISION FOR IMPACT
When competitive players are playing team-based multiplayer games, we want to support them in collaborating with their teams. Specifically, we want them to feel safe and empowered to queue for games knowing that they are more likely to have a positive, collaborative experience. This can help them meet their goals of belonging to a community and improving their own gameplay. Our vision:
"We aim to design a system used by game developers to support players in experiencing collaborative and positive matches in multiplayer, competitive video games."
We aim to design a system that goes beyond pure censorship or pure rewards. We want it to be multi-faceted and customizable to many types of games within the multiplayer, team-based, competitive genre. Specifically, we are designing for Theme 2: Fostering Healthy and Resilient Digital Spaces.
OUR UX Methods
Using the Design Thinking Process, our team conducted our UX research & design with methods including Semi-Structured Interviews, Affinity Diagramming, Storyboarding, Thematic Analysis, Wireframing, Prototype Testing, and many more.
OUR DESIGN QUESTION
Our team cares deeply about collaborative, gaming environments because of the frequent use of antisocial or hostile communication in online, ranked matches. And so we wondered:
"How Might We develop a system that rewards collaboration amongst gamers in online ranked play and reduces antisocial actions?"
OUR DESIGN SPACE
We used affinity diagramming techniques to iteratively brainstorm our design space. We chose to focus on the 5 W's (Who, What, When, Where, Why) of the design space.
OUR CORE STAKEHOLDERS
Through iterative rounds of research, we identified 3 core stakeholders in our design space: Streaming Gamer, Personal Identities, and Game Developer.
We decided to choose streaming gamers as one of the stakeholder groups because they typically have a degree of influence on their viewers and other players. We believe if we are able to spread positivity through the gaming entertainment industry, it would have a positive impact on the overall gaming community.
We chose Personal Identities as one of the stakeholder groups because we believe minority races, women, and LGBTQIA+ players are targeted with more hostile communication than other players. As one of our core stakeholders, our group aims to welcome and support them into the gaming community.
We chose Game Developers as one of the stakeholder groups because they are the ones that are responsible for producing games. Their impact on the gaming industry is significant and it is important for our group to consider what we can produce to convince and support game developers in designing for digital thriving.
SECONDARY RESEARCH
We needed to learn about what causes disruptive behaviors in competitive gaming spaces. We researched the effects of collaboration in gaming and the rising prevalence of toxicity in gaming. We learned different gamers' goals and constant exposure to disruptive behaviors are factors that influence toxicity in competitive gaming spaces.
A main takeaway we had from this article was the concept that optimal collaboration may be interfered with by individual goals. Although many matchmade teams have a collective goal of winning a match, the individual players may have differing ideas of how to get there which can result in a "sub-optimal" collective outcome.
A main takeaway we had from this article was that being exposed to toxic behaviors leads to the acceptance of toxic behaviors. This ultimately causes a cycle where multiple being toxic in an online community creates this idea that being toxic is okay which causes more and more people to continue being toxic.
A main takeaway we had from this article was that multiplayer games can encourage players to work together, build trust, and develop social skills. Although there is a negative stigma that surrounds competitive multiplayer games, it is a space that can provide benefits to its players.
PILOT OBSERVATIONS & INTERVIEWS
We conducted both semi-structured interviews and user observations in order to collect quantitative data and better understand toxicity in game.
One challenge we had during this process is we don’t know where to start. We were unsure of how to conduct user observations and interviews to achieve our goal of understanding disruptive behaviors in the game. However, we overcame this problem by constructing interview protocols and user observation protocols iteratively based on the feedback given by our instructors.
We conducted 2 user observations using online game streaming platforms such as Twitch and Youtube. We observed ranked games of Counter-Strike: Global Offensive and Call of Duty: Modern Warfare 2 specifically looking for team communication, player attitudes, and player performances.
We recruited 2 participants through colleagues and conducted semi-structured interviews using Zoom or in person. We asked questions related to toxicity in order to understand what triggers disruptive behaviors in games.
POINTS-OF-PAIN SCENARIO
Using our thematic analysis and research, we created a points-of-pain scenario that encompasses a typical experience in a ranked game.
In this scenario, Bob, a 19-years old aspiring pro player, decides to join an online ranked match in order to improve his own gameplay skills. When he ends up making mistakes, instead of encouragement or constructive criticism, his teammates curse and berate him. Bob exits the game mid-match and is left frustrated, depressed, and unconfident in himself.
With this scenario, we're able to focus on the emotions our players may feel to help us understand where our prototype should help make an impact.
Scenario Settings
Scenario Scenes
PRIMARY RESEARCH:
SEMI-STRUCTURED INTERVIEWS WITH STAKEHOLDERS
We iterated on our pilot research in order to better understand the different stakeholders of our design space. We wondered what types of groups encounter toxicity. One challenge we had recruiting streamer gamers in addition to our stakeholder groups -- personal identities and gaming developers. Next time, we will better research candidates through either our social networks, LinkedIn, or recruiting at gaming centers.
We recruited 6 participants through personal connections and LinkedIn. Then, we conducted semi-structured interviews using Zoom or in-person and asked questions related to toxicity (disruptive behavior) in order to understand what triggers toxicity in the game.
PRIMARY DATA ANALYSIS:
THEMATIC ANALYSIS
With the 6 semi structured interviews with stakeholders including game developers, beginning gamers, and underrepresented groups, we found that many of them liked the idea of cosmetic items as rewards, such as player skins, for being positive.
Some themes we found were:
- Acceptance of toxicity
- Players' emotion affecting other teammates
- Specific communication methods
- Insight for solutions
STAKEHOLDER PROBLEM STATEMENTS
From our primary research, we iteratively constructed problem statements to understand our stakeholder's needs, challenges, and goals.
Problem Statement for Stakeholder Group 1: Streaming Gamers
We found that this group want to have fun playing ranked games, improve their skillset, and work as a team. However, these gamers are met with disruptive behavior and harmful conduct when they make mistakes.
Problem Statement for Stakeholder Group 2: Underrepresented Gamers
This group of gamers wants respect from other players and the ability to be themselves when gaming. However, these gamers feel ostracized from better-represented groups and have a lack of control over these toxic players in their matches.
Problem Statement for Stakeholder Group 3: Game Developers
This stakeholder needs to use their available resources to build an application to improve the gaming community and management to approve and support this development. However, a limited number of resources, an inability to find a solution, and a lack of managerial support prevent gaming developers from obtaining their core needs.
STAKEHOLDER REQUIREMENTS
From our primary research, we iteratively constructed stakeholder requirements to understand their functional and contextual requirements. Originally, we selected beginning players as our primary stakeholders but later decided to change the stakeholder to streaming gamers. We felt that this stakeholder group brought a new perspective, whereas beginning players' insights were similar to those of the underrepresented gamers.
Functional Requirements
Contextual Requirements
BRAINSTORMING DESIGN IDEAS
We used mind mapping to help brainstorm our design ideas for three different components of our design:
Types of Prototype - we brainstormed different forms a prototype can take.
Raise Awareness - we thought of different mediums that could bring attention to this issue.
Influencing behavior - we thought of different incentives and punishments that could affect a player's behavior.
CONCEPT IDEAS
As a team, using the insights from our user research, we came up with multiple ways to prevent/punish disruptive behaviors in games as well as reward positive behaviors.
Reward/Deterrent System
A system that incorporates deterrents and rewards for communication in ranked matches. For example, a player will have a number of points to receive based on their communication score (Communication score will be directly affected by what is deemed a positive and negative comment throughout the course of a match.
Text and Voice Filtration
A system that filters hostile and inappropriate language from verbal and written communication during matches. Hostile and inappropriate language will be replaced or omitted from communications if detected by an AI system.
Developer Protocol/Standards
A protocol/standard for game developers to follow that can help them design games to reduce antisocial actions. Guidelines will be created for developers to follow as well as standards they must meet in order to regulate toxicity within their games.
CONCEPT TESTING
We recruited 6 participants and conducted semi-structured interviews using Zoom and in-person sessions. We asked questions related to how our concept ideas may help our interviewees in their own gaming experiences in order to understand what are the pros and cons of each concept idea. We also want to know if there is a clear concept that is liked by the majority of interviewees.
We then conducted thematic analysis to discover themes of what is wanted from a solution that tackles disruptive behaviors in competitive gaming environments.
From our thematic analysis, some key takeaways include:
- Players in general were concerned about competitive points being taken away from them if they had a poor communication score.
- Interviewees also seemed to generally be concerned about some of our concepts being abusable by the player base.
- Players favored concepts that punished poor communication
- Game developers favored concepts that rewarded good communication
- Both groups favored concepts that resulted in immediate feedback for players
- Both groups believed the problem required a multi-faceted approach
- Overall we found that our interviewees had a preference for our concept where players gain rewards for positive/collaborative communication.
PROTOTYPING:
STORYBOARDING
We created a storyboard and scenario to better understand how users may interact with our prototype.
In the storyboard, it shows how our system is implemented in the games and how positive and negative behaviors are detected!
- Imagine you log into your favorite game that uses our Positivity Perks System.
- You got into a game and are faced with both toxic and supportive comments from your teammates.
- The AI system in our Positivity Perks detects both positive and negative behaviors.
- After the game ends, points are rewarded to supportive players and points are deducted from toxic players.
You log into your favorite game.
You make a mistake and die.
One of your teammates becomes mad and send toxic comments.
This comment is detected as “negative behavior” by the AI system.
Another teammate supports and defends you.
This comment is detected as “positive behavior” by the AI system.
After the game ends, you and the supportive teammate get 50 Sportsmanship Points.
The toxic teammate received 0 points.
You are able to redeem in-game cosmetics items using Positivity Perks.
In the scenario, it shows how our main character, Amy, can interact with our product, Positivity Perks!
- She was able to signup and connect her game account to our product.
- She uses our product to find players with high honor ratings to play on the community page.
- After playing with her new friends, she was able to redeem Sportsmanship Points for in-game cosmetics in our product.
- She was also able to learn about instances of supportive behaviors on our match history page.
PROTOTYPING:
INTERACTION FLOW DIAGRAMMING
Our group constructed a user flow diagram to better understand the effect of a user's decision when using the application.
In this interaction flow diagram:
- The user starts a game, opens and enables the application, and plays an online match.
- During the match, if a user uses hostile communication, then a sportsmanship point is deducted.
- The continued use of this communication will result in no sportsmanship points given and no positive change in the user's honor rating.
- The user can view their instances of hostile communication and decide to learn from those mistakes.
- If the user does learn from their mistakes, then the next time they play a match and refrain from hostile communication there will be no deductions to their sportsmanship point total.
- The full total of points will result in a boost to their honor rating, more gamers will want to play with them, and more viewers might watch their streams.
- Similarly, the user could use those points to purchase merchandise within the application.
PROTOTYPING:
LOW-FIDELITY WIREFRAME
In the low-fidelity wireframe, we designed a total of 5 different pages: Login Page, Profile Page, Reward Shop Page, Community Page, and History Page.
Before we introduce the pages, there are two important features that calculate a player’s friendliness and sportsmanship.
The first one is SP points. SP points are currencies that can be earned by players based on their communication and actions in the game. Each player will be eligible to earn 50 SP points per game. However, if they are hostile to other players, their SP points will be deducted.
The other feature is Honor Percentage. The player's Honor Percentage will be calculated based on their SP points. The system will calculate the player’s average SP points earned per game and divide it by 50. For example, if a player plays 2 games and received a 50 and 45 SP in those two games, their Honor Percentage will be 95%.
Login Page
The first page is the Login Page, users are prompted to enter their account information or they can connect their account with their gaming account. They can also create an account manually.
Profile Page
In the Profile Page, user information such as their profile picture, honor percentage, SP points, and games that they have connected to their accounts are displayed. SP Points are points that can be earned by users based on their
Reward Shop Page
In the Reward Shop Page, there is a list of game cosmetics that can be purchased using the SP points the user gained from their past games. If needed, there is also a filter on top to help users filter items from a specific game.
Community Page
In the Community Page, we have a system where we help players find other players to play with. Furthermore, there is a filter on top to allow users to select the desired game. Then, a list of users will be displayed with their username, honor percentage, description, and an add friend button. The purpose of the page is to help players find others based on their game interests and honor percentage.
History Page
The History Page shows the player’s match history and instances of either supportive or hostile communication. Players will be able to learn from the past about what increases or decreases their SP points and Honor percentage.
PROTOTYPE ITERATION
From our prototype testing, we learned that most players and developers prefer a desktop version since it will be easier for them to use while playing games. Based on this insight, we created an additional desktop version that shares the same feature as our mobile version. Furthermore, we also learned that some players prefer a resources page that can help them understand positive and negative behaviors. We decided to implement a new page to help educate our users in order to better support gamers in understanding how to tackle toxicity when it is presented to them.
Using Figma, we iterated on our design. We created a desktop homepage to start our desktop prototype by expanding upon our mobile home page prototype. We also created a resource page in our mobile application for to better educate themselves about what to do in certain scenarios or understand why certain behaviors are happening.
Desktop Prototype
Resource Page
KEY QUESTIONS & NEXT STEPS
In our project, we aim to create a solution that effectively tackles disruptive behaviors in competitive gaming spaces by promoting positive behavior, collaboration, and inclusivity. Our design goal is to create a safe and welcoming environment for all gamers, regardless of their background or skill level. We hope that our solution will have a positive impact on the gaming community by promoting positivity and reducing the prevalence of toxic behavior. Ultimately, our vision is to create a better gaming experience for everyone. Here are our Next Steps and Key Questions.
Next Steps:
- We need to understand what specific features and functionalities our prototype should have to effectively tackle toxicity in competitive gaming spaces.
- We need to identify the potential challenges that may arise during the implementation of our solution and come up with strategies to overcome them.
- We need to determine how we can measure the success of our prototype in terms of reducing toxicity and improving collaboration in gaming environments.
- We can conduct usability testing with different stakeholder groups, such as game developers, streaming gamers, and personal identity groups, to discover the strengths and weaknesses of our design.
- We can conduct impact testing to measure the effectiveness of our solution in reducing toxicity and promoting positive behavior in gaming environments. By testing our prototype with diverse groups, we can ensure that our solution is inclusive and meets the needs of all stakeholders.
Key Questions:
- Which kinds of toxicity are new and marginalized players facing the most?
- What kinds of API functions will developers need?
- How can we test our prototype on a smaller scale first?
.png)
.png)
.png)
.png)
.png)
