Facial Palsy – Gian Dion Dubberke

AR - Medicine - Unity

Project
Facial palsy

Topic

How can we change Therapy to better support daly life of patients and doctors

Smartphone AR App with face & gesture recognition

Goal

Creating a smartphone App that offers interactive and playful guidance through physical Therapy

The application was developed as a proof of concept to demonstrate the potential for more guided therapy sessions with smartphone reminders, AR guidance markers and gamification elements.

Year

2023

Responsibilities

Research
Concept
Development

Patners

Solo project

Process

Definition & Context
Facial palsy is a paralysis of facial nerves that can affect patients of all ages, often between 25–55. Causes range from genetic factors to infections and flu. Current solutions are limited, and there is a need for more accessible, engaging, and effective training methods for patients.

Our Idea
We developed an AR-based training app that detects facial palsy and guides patients through exercises using face tracking. The system integrates gamification, automated training sequences, and 3D-animated exercise guides to make rehabilitation accessible, motivating, and easy to follow.

Current State & Future Features
The app currently supports an extendable exercise system, expression tracking, adjustable repetitions, and user onboarding. Future improvements may include notifications, progress visualization, consistency diagrams, and game modes to further enhance user engagement and therapy outcomes.