LuminAI.

Using AI-based tech interventions to help motivate senior adults to exercise more often

My Role

UX Designer

UX Researcher

Methods

Sketching

Wireframing

Iterative Prototyping

Usability Testing

Storyboarding

User Journey Flows

Interviews

Observations

Tools Used

Pen + Paper

Whiteboard

Sketch

Figma

Unity

Timeline

Aug 2019 - April 2020

Team

Swar Gujrania

Komal Hirani (me)

01. OVERVIEW

 

IDENTIFYING THE PROBLEM

My teammate had been previously working on a project called LuminAI that uses the Kinect and Artificial Intelligence to capture someone's body silhouette and move a virtual avatar based on that person's movements. She and I then first explored various areas and user groups where we could leverage this technology to help solve and particular problem that occurred in that user group.

After some preliminary research, we found that senior adults ages 65 and older have a tendency to decrease their levels of physical movement, which can lead to an increased risk of problems such as physical injury and depression.

Thus, we wanted to figure out:

How could we help motivate senior adults to increase their physical activity?

WHAT WE CAME UP WITH

LuminAI: an exer-gaming platform that uses Artificial Intelligence and the Microsoft Kinect to give senior adults an entertaining form of physical movement without risking injury. 

 

The system comprises of three main parts, a main projection screen, a Participant App and an Admin/Instructor App. The main screen is a wall-size projection screen which would display routines for people to follow. The contents of the main screen can be controlled by the mobile apps. 

System Overview Diagram

Features

1.

Exercise in 3 different ways

There are 3 different modes that users can choose from to start exercising: 

1. Exercise mode: Participants follow a

pre-defined exercise routine.

2. Game mode: Participants move their body to interact with virtual objects to play games.

3. Instructor mode: Participants would follow a physical instructor who completely leads the exercise session.

2.

Customize your workout based on your physical needs

Users can select how long and how intense they want their exercise session to be.

3.

Use your mobile device for easy control while moving!

Users can use their mobile device to control the main interface at any point of their workout, particularly if they want to take a break or adjust any settings.

4.

Exercise by yourself or with others!

The system allows for users to participate in an individual exercise session or join with others to work out together while an instructor/supervisor controls the system for them.

THE PROCESS

1.

Research

Observations

Interviews

Surveys

Literature Review

Affinity Mapping

2.

Ideation

Persona

Design Implications

Informed Brainstorming

Design Directions

3.

Design

4.

Evaluation

Sketching

User Flows

Storyboards

Wireframes

Feedback Sessions

High-fidelity Prototype

Usability Testing

Heuristic Evaluation

MY CONTRIBUTIONS

UX Design: Created a list of design implications from our findings, sketched potential design ideas for our solution, involved with creating user flows for all potential scenarios, created the initial wireframes, initial prototype, and final prototype of our solution, particularly the mobile app portion and the game mode feature on the desktop part of our solution.

 

UX Research: Conducted user observations of current physical activity classes for senior adults, conducted semi-structured interviews with senior adults living in senior living communities, created and distributed surveys, conducted feedback sessions on wireframes, conducted usability testing sessions on our initial and final prototype.

02. RESEARCH

 

Our first step was to attempt to understand our primary users, senior adults, and our secondary users, wellness instructors and administrators at senior living communities and their current experiences with fitness classes taught at these senior living communities. We also needed to explore previous research projects that have aimed to use artificial-intelligence technologies to encourage senior adults to exercise so that we could have a basis for our project and understand how to design technologies for senior adults. 

 

We used a variety of research methods in order to accomplish this step:

User Observations

(3) Observations led us to understand how wellness classes are currently conducted at senior living centers

Interviews

(11) Semi-structured Interviews led us to understand how senior adults currently feel about their current experiences with physical movement

(3) Semi-structured Interviews led us to understand how wellness instructors feel about their experiences with administering exercise instruction to senior adults

Surveys

(11) Surveys led us to understand how senior adults spend their time and how they feel about socializing with others around them.

Literature Review

We found that there were several projects in the past related to using the Kinect and AI-based exergaming for senior adults but none evaluated their systems with this user group.

ANALYSIS

Our primary form of analysis of our observation and interview data was affinity mapping. This allowed us to find high order needs from both our user's and stakeholder's viewpoints and develop design ideas for our solution.

More than 350 notes generated!

INSIGHTS

Based on our affinity mapping, we distilled out some major insights and recommendations for our eventual design.

Insights from Senior Adults:

Senior adults have a variety of physical limitations

Balance is an extremely important factor  

Want to learn to do exercises at a comfortable pace

Want independence to choose what they do

Need to frequently modify exercises

They don’t want to fail/feel like they will fail

Want to have a physical instructor present

Social aspect of exercise is essential

Senior adults are willing to try new things

They want control on tracking their own progress

Insights from Wellness Instructors:

They teach exercises for senior adults in a group setting

Changing exercises weekly helps seniors grow

Provide modifications all the time

“Be a helper, not a caregiver”

Make sure they are aware of how to prevent injury

Keeping track of progress is beneficial to them and to administration

03. IDEATION

 

Based on the insights we derived from our research data, we drafted an in-depth persona of our primary user group. We also came up with several design implications that our future system would have to be designed in accordance with.

PERSONA

DESIGN IMPLICATIONS

Afterwards, we did several rounds of rapid brainstorming individually and then came to together to consolidate our ideas for the design of the system. The chart presented here shows the design ideas that we came up with. There are 5 main areas that the ideas fell under: technology in props, gamification, movement to generate visualizations, AI support tools, and mirroring. We also explored different kinds of technology that the system could leverage and general features that the system would need. 

From these ideas, we identified 4 design directions for this system. The system could leverage tech enabled props, have an AI co-therapist that helps lead an exercise session, use games or gamification elements to encourage senior adults to move, and use the movements that senior adults make to generate media.

DESIGN DIRECTIONS

04. DESIGN

 

After consulting with our Thesis Advisor and considering our time and budget constraints, we decided to narrow our design directions and decided to combine an AI driven virtual co-therapist with gamification elements.

We then spent some time sketching out possible screens in order to figure out the user flow for our solution. Afterwards, we walked the wall with our individual sketches in order to combine our ideas into one.

SKETCHING PROCESS

Some of my sketches!

SYSTEM OVERVIEW

After our walking the wall session, we pulled our ideas to create LuminAI: an exer-gaming platform that uses Artificial Intelligence and the Microsoft Kinect to give senior adults an entertaining form of physical movement without risking injury. 

The system comprises of three main parts, a main projection screen, a Participant App and an Admin/Instructor App. The main screen is a wall-size projection screen which would display routines for people to follow. The contents of the main screen can be controlled by the mobile apps. 

System Overview Diagram

The system would have 3 modes - 1) exercise mode in which participants follow a pre-defined exercise routine and 2) game mode in which participants move their body to interact with virtual objects to play games. The 3rd is an instructor mode in which the system would take more of an observational role, allowing the instructor to completely lead the session.

FIRST ITERATION

In the first iteration, we designed an Admin-controlled scenario for group exercises in which an Admin/Instructor would have full control of the system, while senior adults would only use their mobile app for joining an exercise session or view their session results. 

User Flow

Storyboard

This storyboard here illustrates how the system would work in a group setting. We have some senior adults entering the space in which the system is deployed, and the admin starts up the system for them. The residents then connect to the system through the app on their phones. After consulting with the group, the admin selects the options for the current session, and starts the session. The seniors follow along the instructions provided on the main projection screen to perform the exercises. After the session is over, they can go back to their app to view their individual results.

Wireframes

Here are some wireframes of the first iteration that we came up with:

FEEDBACK SESSION

We then took our designs to 4 senior residents that we had previously interviewed and got their feedback on our initial proposed system for the participant-facing app and the projected screen. 

Some key findings from these sessions include:

  1. Need for an individual mode so that senior residents could exercise with this system by themselves

  2. Explicit instructions on how to use the system and how to prepare for the exercises, and 

  3. Continuous reminders about alignment and posture in order to minimize the risk for injury.

 

We also discovered that our definition of intensity was unclear and need to provide visibility to the user everywhere. The residents also expressed interest in game mode and wanted to be able to compete with each other.

FINAL ITERATION

After consolidating our findings from the feedback sessions on our wireframes, we made some key updates to our system designs:

Updated User Flow

Storyboards

To re-iterate, we have this scenario in which an admin is controlling the LuminAI system with a group of senior residents.

And we also now have this scenario in which a senior resident wants to use the system and exercise by themself. The resident enters the area in the living community in which the system is deployed, starts up the system, connects to and controls the system with their app, and then starts moving with the system.

Design Updates

Here are several screenshots of the main features that we either added or improved in this iteration.

1.

App Tutorials

We added several tutorials throughout the app that help the residents understand the purpose of this system and how to use it.

2.

Setup and Posture Help

We also re-designed and added several tutorials to remind the residents about maintaining proper exercise conditions.

3.

Redefined Intensity

We redefined intensity as a measure of time duration and speed levels for the exercise mode.

4.

Session Tutorials

We also improved the tutorials we had for exercise mode and game mode to ensure their safety.

5.

Improved AI-Based In-Session Layout

In exercise mode, the LuminAI character is able to detect the speed at which the residents are able to perform the exercise at and change the current speed setting so that everyone can keep up with the character. In game mode, the bubbles appear based on the senior’s position and change their position based on the senior’s range of motion.

6.

Revamped Results

We also revamped the results page in order to include more feedback on a senior adult’s workout sessions with the system.

Prototype Walkthrough

Here is a video of our prototype in action! This video shows the use case of a senior resident who wants to complete a session of Exercise Mode on their own. 

 

05. EVALUATION

As we started with the final round of evaluation on our prototype, it was around the time that the COVID-19 pandemic was taking roots and cases were increasing rapidly. Thus, we could not do any in-person evaluation on our prototype.

We were still able to do:

  • (4) Remote System Walkthroughs with administrators and instructors through Zoom

  • (2) Remote System Walkthroughs with seniors through Zoom and Google Hangouts

  • (4) Heuristic Evaluations with experts through Zoom

Findings - Admins

Positive:

  • System was overall straightforward and simple

  • AI capabilities are really great

  • Definition of intensity is solid

  • Game mode seems really fun

  • Love the reminders for support and posture

Areas of Improvement:

  • Self-rating is nice, but needs to be clearer

  • Content on exercises needs to be improved

  • Think about those with visual impairments as much as possible

  • Interacting with the technology initially could be challenging

  • Make things as simple as possible

Findings - Senior Adults

Positive:

  • Like having 3 modes to choose from

  • AI capabilities are really great

  • Ending screen for exercise mode on mobile is good

  • Game mode seems really fun

  • Tutorials seem pretty straightforward

Areas of Improvement:

  • Would like to be able to do more with body focuses

  • Content on exercises needs to be improved

  • Exercise List Preview needs work

  • Self-rating system is a little unclear

  • Would want more games in the future

Overall, both administrators and senior adults found our proposed system very intriguing and expressed positive interest in wanting to try it out in person! They gave us the following System Usability Scores (SUS): 

83.75

Administrators/Instructors

85

Senior Adults

System Usability Scores (SUS)

Thus, from a users' and stakeholders' perspective, we achieved our goal of designing an AI-based exergaming platform that senior adults would find intriguing and entertaining and would use this system to move more.

Heuristic Evaluation Findings

Overall, our usability experts didn’t find too many usability problems with our system; however, there were some areas in regards to visibility, user control, consistency, and flexibility that some of our experts found some issues with.

Positive:

  • Explicit & detailed instructions and illustrations were great

  • Very well thought out and covers many different scenarios

  • Feasible and easy to use

  • AI-based features are great

  • Games are very engaging

  • Can see senior adults enjoying the system

Areas of Improvement:

  • Language of instructions could be improved

  • Some visual inconsistency between mobile and web apps

  • Adding audio cues would make system more effective

  • Self-rating system is a little unclear

  • Visuals should be tested for accessibility

06. REFLECTION

 

Overall, I'm extremely proud of the solution that we were able to produce for this user group. I really enjoyed collaborating with several Senior Living Communities in the Atlanta area and seeing smiles on the senior adults who we worked with to create a system with the potential to encourage senior adults to move more. I had very limited knowledge previously about the challenges that senior adults face, so I learned a lot about this user group and how to design for those with limited technical know-how. In the process, I also saw the potential that AI holds when it comes to simplification of interactions for users with many limitations.

Of course, the pandemic situation hindered our testing process a little bit (we even started to develop the game mode in Unity that could be tested with the Kinect); however, I believe that we were able to successfully get some feedback despite this hinderance. I had done some remote user testing in the past, but this project helped me grow in that skill-set, especially since we had the constraint that our user group doesn't normally use video conferencing software. This situation also taught me how to quickly adapt to changing circumstances and how to function under extreme constraints.

If I could continue this project, I would:

  • Evaluate the system extensively with seniors, instructors and experts

  • Bridge the usability gaps discovered so far and re-design aspects of certain features

  • Develop the designs into an MVP

  • Study how seniors interact with AI and can such systems be used for AI literacy for the population

  • Curate the content of the system - exercise routines, language, music etc.

Thank you for reading about my journey with this project!

Email me: komalhirani7@gmail.com

Connect with me on Linkedin

Created with ♥ by Komal Hirani

 

© 2020