Research · AR + IoT
HoloKeys
Research · AR + IoT
When lessons moved online during COVID, music teachers had a real problem: you can't really learn piano from a laptop camera. HoloKeys lets a teacher physically play a student's piano from anywhere.
The teacher puts on a Magic Leap headset and a holographic copy of the student's piano appears in front of them. They play it with their hands, and every key they pinch is sent over the internet to the student's home, where little servo motors clipped onto the real keys press the same notes. The student sees and hears the demonstration on their own keyboard, just like a lesson in person.
The clips attach to any keyboard with no permanent changes, and the whole student side is cheap and connects over wifi. It was funded by a University of Florida College of the Arts grant.
We presented it at ACM VRST 2021 in Osaka, Japan, and it has become my most cited paper. I love that researchers in music education and mixed reality keep building on it! I made it with Austin Stanbury and Dr. Hyo Jeong Kang at the Digital Worlds Institute.
The project I care about most. Volunteers in Tunisia and around the world scan our endangered heritage in 3D, one object at a time. 100+ free models so far!
The first community-led virtual museum of Tunisian heritage. Real scanned objects, life-size, in hand-modeled whitewashed rooms. Built in Unity by our volunteers.
Every object our volunteers scanned, in your browser. Turn it with a finger, hear Nura tell its story, save your favorites, or put on a headset.
Let's work together
I'd love to hear about it! Whether you want to scan a collection or build something in XR, let's talk.