TeachXR

A desktop tool that empowers teachers  to implement, customize, and share XR storytelling in their classrooms.

Tools Used
Unity, Figma, Case Study
Date
Aug 2022 - Dec 2022
Type
AR mobile application & VR application
Role
UI design, research,
Tools
Figma, Unity
Teammates
Dr. Robert LiKamWa, Lauren Gold , Kathryn Powell, Shaun Xiong, Justin Hertzberg, Alexander Gonzalez
Duration
1.5 years
(Jan '19 - Apr '21)

Intro

Intro

Assisting NASA scientists in communicating their findings

JMARS AR Viewer makes complicated planetary terrain data understandable to anyone. It is used today to assist NASA scientists in communicating and disseminating their findings.
I designed high fidelity prototypes and developed interfaces for JMARS AR Viewer. The application received such a great response that my team and I were rehired to create a virtual reality version of JMARS AR Viewer.
Intro
Artifacts
Research
Problems
Designing TXR
TXR Prototype
Aaron's Ideation
Building Aaron's AR
Testing in Class
Conclusion

Intro

Intro

Empowering educators to teach in 3D

For my Master of Design thesis at UC Berkeley, I worked with Lucila Duloup to build TeachXR—a desktop tool that enables high school chemistry teachers to implement and customize XR activities as a medium for classroom storytelling.
This project speculates that if there were a tool to support teachers with implementing XR technologies in their classrooms, then teachers could explain microscopic science concepts in a way that is easy for students to understand. That hypothesis is tested through a case study of Aaron Glimme, a chemistry teacher at Berkeley High School.
Artifacts

Two Artifacts

I ideated and rapidly prototyped three possible solutions: Pull, Grab, and Rope. Then I met with my team in a VR collaborative environment to get feedback and discuss how these solutions address the current zoom problem.
TeachXR Desktop Tool
The TeachXR desktop tool guides teachers to create and share XR activities for their classrooms.
Aaron’s AR
Aaron designed an AR mobile activity using tools from the TeachXR desktop tool to illustrate microscopic concepts of phase change.
Research

Understanding teacher's needs

A case study is conducted to evaluate the hypothesis in a real-world context. The focus of the study is Aaron Glimme, a high school chemistry teacher at Berkeley High School.
The case study included a classroom observations, interviews and UX testing with Aaron. Feedback is also collected from other teachers and professionals from industry to confirm that design decisions are representative of the greater population.
Classroom Observation
User Interviews
Literature Review
Professional Consultation
Problems

Two main problems were identified

I ideated and rapidly prototyped three possible solutions: Pull, Grab, and Rope. Then I met with my team in a VR collaborative environment to get feedback and discuss how these solutions address the current zoom problem.
Students have difficulty learning 3d concepts when taught in 2d (but XR can help!)
Teachers struggle to learn XR—steep learning curve, lack of time, lack of community support.

How might we empower teachers to use XR technology to illustrate complex 3D topics?

Problem

Designing TeachXR

Increasing fidelity with feedback

JMARS data is too complicated

After sketching and storyboarding possible solutions for the TeachXR desktop tool....

Intro

I designed several iterations of the desktop tool in Figma. Each iteration was led by feedback from Aaron, other high school science teachers, and experts in the XR ed-tech space. The images below illustrate iterations of the TeachXR Explore page.
The most significant changes involve: editing types of search filters and increasing familiarity of navigation design.

Problem

Testing complex interactions

JMARS data is too complicated

The most complex portion of the TeachXR tool is the XR creation/editing feature. To better understand user needs, I built several iterations of a low fidelity desktop app that allowed teachers to use the creation/editing tools in real time.
I built each iteration of the desktop app in Unity, using C# scriptting and the Unity physics engine. This process allowed me to see what types of interactions were successful, and which portions needed improvement.
Classroom Observation
User Interviews

Problem

JMARS data is too complicated

The final iteration confirmed
  • Three zoom levels
  • Molecule selection from drop down menu
  • Description text input
  • Animation creation using formulas
These confirmed interactions are implemented in the XR creation/editing of the TeachXR Figma prototype.
Final TeachXR Prototype

Problem

JMARS data is too complicated

Aaron's AR Ideation

Observation + Ideation

Two problems were identified from observing Aaron's class. I ideated possible ways AR could solve each problem and discussed the solutions with Aaron.
I ideated and rapidly prototyped three possible solutions: Pull, Grab, and Rope. Then I met with my team in a VR collaborative environment to get feedback and discuss how these solutions address the current zoom problem.
1
Problem: Students in the back struggle to see lab demo
Solution: Replicate demo at students desks
2
Problem: Aaron struggled to explain invisible concepts
Solution: Augment invisible concepts atop live demo
 Feedback: Pull interaction is nauseating
Feedback: Showing more info is good, grab will not work for objects that are far away

Intro

We decided that digitally replicating what you can already see in the physical world is a worse version of reality.

Augmenting invisible content on top of a lab in the physical world, is a better version of reality.

Problem

Building Aaron's AR

Giving Aaron the Tools

JMARS data is too complicated

Based on discoveries from observation and ideation, I started building the AR experience with Aaron.
Aaron worked with the TeachXR Figma interface to build the the AR experience that would best help him tell the store of phase change.

Problem

JMARS data is too complicated

Aaron used the XR activity editor to create three states of h20 in the molecular zoom level.  By inputing different temperatures for each state, TeachXR created animations of h20 molecules as a solid, liquid, and gas.

Problem

Bringing Aaron's AR activity to life

JMARS data is too complicated

I built animations in Unity and used Vuforia image tracking to create an AR mobile app that enables Aaron to illustrate the structure and movement of h20 molecules in three states.
Three image trackers to show molecule structure of h20 at solid, liquid, and gas.
Animations depict molecule structure and particle movement of h20 at each state.
When an image is recognized, the associated molecule animation appears.

Problem

Testing TeachXR in Class

Aaron teaching with TeachXR Tools

JMARS data is too complicated

Design decisions were confirmed as Aaron was able to create an AR activity using TeachXR tools, and implement the activity into a lecture.

JMARS data is too complicated

Conclusion

TeachXR Success

In sum, I worked with Lula Duloup to create an engaging and effective tool to inform and empower high school teachers to use XR technologies as a storytelling tool.
To defend TeachXR as my graduate thesis project, I wrote 131 pages to capture this study and presented my work to four esteemed Berkeley professors and three leaders of industry.
We received incredibly positive feedback and are excited about a future that TeachXR creates. This future would make XR technologies accessible and exciting for teachers. With the power to create, share, and edit 3D content, a teacher’s storytelling potential would only limited to their imagination—No experience, whether dangerous, expensive, invisible, or impossible, is too challenging to illustrate.

Reflection

JMARS data is too complicated

JMARS AR Viewer was a huge success, me and my team were rehired to build a VR experience which allows users to fully immerse and explore planetary terrain in 3D.
JMARS AR Viewer is important in bridging the education gap between planetary scientists and the general public to increase accessibility to planetary data. This is the first project that I worked on that really focused on using mixed reality to make tools accessible to people who otherwise be unable to use them. Making tools accessible to everyone using XR is a huge passion of mine today, and I am grateful to have worked on this project where I could bring that passion to life.
Next Project
JMARS AR