User Research Design

Research Goals:

1. Understand shifts in attitudes towards social connection during the pandemic

2. Understand consistencies and changes in user experiences with digital social connection

3. Understand new opportunities and challenges via the strategies that users have developed

Target User Group:

Young adult professionals and students - two groups that have been disproportionately impacted by COVID-19. These users share similar needs after being disconnected from their physical work and academic communities in the transition to remote working/learning.

Methodology:

• Phase 1: Diary study and user interviews to understand users’ needs for social connections

• Phase 2: Design fiction sessions to narrow down design focus

• Phase 3: Brainstorm relevant interventions to storyboard, prototype, and conduct user testing.

 Method 1: Diary Study

For our first method, we conducted a feedback diary study to explore how users connected during the pandemic, as well as their emotions, thoughts, and experiences. Participants were asked to fill out an online questionnaire after each significant social interaction. They were allowed to individually define “significant,” which we then explored in follow-up interviews.

Duration:

2 weeks with daily dairy entries

Participants:

College distance learners and young professionals who work from home

Procedures:

Contextual Interview

Gather background information and fill out screener survey

⬇

Filling Out Questionnaire

Participants to answer a Google Forms questionnaire after every significant social connection

⬇

Follow-Up Interview

Participants were asked to define “significant social interaction” to capture their perspective on what they considered to be important

Data Analysis:

 Findings:

• Although less frequent, in-person interactions continue to be the most enjoyable for participants.

• During the pandemic, participants developed a greater appreciation for mundane activities and casual conversations, although they found it harder to initiate these interactions.

• Virtual technologies have reduced certain types of work in connecting (e.g., commutes) but have increased work in other areas. Video is the preferred method for virtual interactions for most participants, but ironically, Zoom is one of the most negatively perceived tools.

Design Themes Identified:

• Re-imagining school-related interactions to be more interactive and engaging

• Creating opportunities for shared activities and casual conversation

• Introducing greater spontaneity into social interactions

• Strengthening the benefits of virtual interactions while counteracting the work required to set them up and run them 

• Reducing the feeling of unnaturalness in virtual interactions

 The varied nature of these findings suggests that social connection is complex, multi-dimensional, and possibly broader than the scope of a technological intervention. We continued to explore these themes through design fiction sessions with our participants.

 Method II: Design Fiction Sessions

For our second method, we conducted two anti-solutionist Design Fiction sessions with 4 participants (2 participants per session). From our first design method, we had uncovered multiple topics for exploration, but there were not enough details to prioritize specific challenges. To address these gaps, we used Design Fiction activities to drive discussion that later informed our ideation.

Duration:

~ 1 hour over Zoom and Miro for each session

Participants:

4 college distance learners and young professionals who work from home: 3 participants were distance learning students (2 female, 1 male) and 1 was a young working professional (male).

Recruited through student forums such as Ed Discussion, as well as social and professional platforms like LinkedIn and Facebook.

Session Plan:

Design Session Intro

Gave participants a detailed overview of un-useless designs, the process of creating and defending magic machines, and explained the design themes

⬇

Brainstorm and Design

Participants worked as a group to select one or more themes to design for, brainstorm ideas, and pick one to digitally sketch

⬇

Follow-Up Discussion

Participants were asked to walk us through a use scenario in their hypothetical future, as well as consider challenges that their designs may not address

Data Analysis:

Qualitative Findings:

• Shifts in user attitudes towards social connection during the pandemic

  — Participants from both Design Fiction sessions focused on solutions that enabled side conversations to chat about their lives and commiserate.

  “In the moment, you get a little relief, knowing that someone else's having the same problems”

  — Participants preferred in-person interactions for social activities, but were able to find more meaning in virtual interactions with people that they felt close to.

  “[Before] we could choose who we want to sit next to...people have partners that they prefer to work with”

• Consistencies and changes in user experiences with digital social connection

  — Most participants preferred video tools such as FaceTime, which were familiar and provided the closest proxy for in-person interactions.

  “You actually feel there's someone. You know, like you're not just talking to the screen”

  — However, Zoom was often criticized and only associated with academic and professional work interactions. The randomization used in breakout rooms reduced participants’ autonomy to choose who they interacted with, creating friction and resentment.

  “If you get assigned to a breakout room that you're not happy with, you can’t argue with the professor”

• New opportunities and challenges via the methods/strategies that users have developed

  — In-person interactions remained the primary medium through which participants share experiences, meet spontaneously, and make new friends.

  — Participants from both Design Fiction sessions highlighted the difficulty of getting to know new connections virtually, alluding to a decrease in confidence that impacted the growth of new relationships.

  “I feel that during COVID, most interactions are virtual and it becomes harder for us to know others, so it takes more time for us to know what they like, what they don't like”

  — Virtual interactions simultaneously required more and less work. Participants saved effort from commuting and enjoyed the flexibility of virtual meetings. However, they spent more effort in navigating between and physically managing devices, setting up calendar invites and links, and clarifying communications misunderstandings.

  “I need to put effort and unmute myself, open my camera, and talk”

  — Setting up virtual interactions required a purpose and thus felt unnatural.

Design & Evaluation

Synthesis of our findings indicated that during the pandemic, users need greater opportunities for spontaneous, private interactions that support new relationships. To design for this need, our proposed idea focuses on creating opt-in, one-to-one support groups. Our design will help students find peers in their classes to develop connections with.

Evaluation:

We evaluated the storyboard by conducting a roleplay with 4 participants (1 male, 3 female) that were similar to our user demographic (3 college students, 1 recent graduate). During the roleplay, we asked participants to put themselves into the headspace of a student who is physically away from the campus and is currently struggling through a difficult class. Then we asked them to roleplay the scenes of the storyboard and imagine themselves as the main protagonist.

We evaluated the speed meeting interface by asking three UX designers to conduct heuristic analyses on our prototype to identify usability issues. The severity of these issues were ranked low, however, and the evaluators understood the overall user flow. From the list created by our evaluators, we prioritized and fixed 60% of the most pressing issues.

 Future Work

While we were able to develop an understanding of our original research questions, the results of our design evaluations suggest that there are still gaps in our translation of these insights into a successful design intervention. Some concerns were raised by the participants during the evaluation process:

•  How do we ensure that people are on the app at the same time for speed meetings?

• How does this design distinguish itself from discussion forums, which are current popular resources for academic help?

Participants’ lukewarm responses to our design suggest that social connection is complex, multi-dimensional, and possibly broader than the scope of our understanding. Of particular interest is participants’ natural association of our design with academic support, rather than social uses. One potential hypothesis is that the overuse of digital technologies for education and work purposes during the pandemic will continue to impact user perceptions of digital connection long after the pandemic has run its course. Further research and testing is needed to test this hypothesis, as well as brainstorm alternative ways to support users with the themes identified in our research. Non-technological tools and un-design may be relevant next steps to explore.